[SERVER-73418] repeat_queries passthrough leaks cursors Created: 27/Jan/23  Updated: 29/Oct/23  Resolved: 03/Feb/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: Engineering Test Priority: Major - P3
Reporter: David Percy Assignee: David Percy
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-73024 Re-enable time-series sharded passthr... Closed
Backwards Compatibility: Fully Compatible
Sprint: QO 2023-02-06, QO 2023-02-20
Participants:

 Description   

When jstests/libs/override_methods/rerun_queries.js runs a command more than once, it ignores all but the last result. Commands like find and aggregate can include a cursor in the response, so we should explicitly close the cursor if we're not returning it to the original caller.

This came up while fixing SERVER-73024: patch builds were failing with "System failure", we think because the open cursors are using a lot of memory.



 Comments   
Comment by Githook User [ 30/Jan/23 ]

Author:

{'name': 'David Percy', 'email': 'david.percy@mongodb.com', 'username': 'dpercy'}

Message: SERVER-73418 Make repeat_queries clean up its cursors

This override helper runs a given command several times, but only passes
ownership of the last result to the caller. The extra results before the
last one were being ignored.

Instead it can call killCursors on any response that it otherwise
ignores.
Branch: master
https://github.com/mongodb/mongo/commit/59e890e0aa8db10826e6d1931e4a68559541c225

Comment by David Percy [ 27/Jan/23 ]

To repro:

In one shell, use $currentOp to list open cursors:

> db.getSiblingDB('admin').aggregate([{$currentOp: {idleCursors: true}}, {$match: {type: "idleCursor"}}]).toArray()

In a separate shell, load the override helper. (Using the same shell would be confusing because the override helper will also affect the $currentOp query.)

> load('jstests/libs/override_methods/rerun_queries.js')
> db.c.drop()
> db.c.insert([ {}, {}, {} ])
> db.c.find().batchSize(2).itcount()

The batchSize smaller than the collection ensures the initial reply includes a cursor ID. Normally itcount() consumes the cursor.

But currentOp (in the other shell) will show some cursors were left open:

> db.getSiblingDB('admin').aggregate([{$currentOp: {idleCursors: true}}, {$match: {type: "idleCursor"}}]).pretty()
{
        "type" : "idleCursor",
        ...
        "planSummary" : "COLLSCAN",
        ...
}
{
        "type" : "idleCursor",
        ...
        "planSummary" : "COLLSCAN",
        ...
}

Generated at Thu Feb 08 06:24:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.