[SERVER-32212] DBCommandCursor is hard to use with tailable cursors Created: 07/Dec/17  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying, Shell
Affects Version/s: 3.6.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Charlie Swanson Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Participants:

 Description   

Currently it's pretty hard to get a correct, never-ending loop on a change stream cursor.

This is not right:

let cursor = db.foo.watch();
while (cursor.hasNext()) {
    printjson(cursor.next());
}

because the cursor will return false from hasNext() whenever it gets an empty batch back from the server, even though the cursor is still open and there will eventually be a next change. Fortunately, we added a isExhausted() which will only return true if there is nothing left in the cursor and the server has closed the cursor, so there will never be anything else. Sadly, this is still not quite right:

cursor = db.foo.watch();
while (!cursor.isExhausted()) {
    printjson(cursor.next());
}

because cursor.next() will first check whether hasNext() returns true, so if there is ever an empty batch, this will result in an error like the following:

foo:PRIMARY> cursor.next()
2017-12-07T17:05:04.624-0500 E QUERY    [thread1] Error: error hasNext: false :
DBCommandCursor.prototype.next@src/mongo/shell/query.js:843:1
@(shell):1:1

The correct way to do a never-ending loop like this over a tailable cursor is as follows:

cursor = db.foo.watch();
while (!cursor.isExhausted()) {
    if (cursor.hasNext()) {
        printjson(cursor.next());
    }
}

This seems overly confusing. It would be easier if the second example would work and cursor.next() would just automatically wait on cursors that are not exhausted but do not yet have a next result.



 Comments   
Comment by Charlie Swanson [ 05/Jan/18 ]

Another oddity is that it is perfectly legal to call one of several helpers that don't really make sense on a tailable cursor, such as .itcount(), .toArray() or .length().

Relatedly, if we change the behavior of .next() or .hasNext(), we should consider the impact on a user in the shell establishing a tailable cursor like so:

let changeCursor = db.foo.watch();

As is, this will cause the shell to iterate that cursor, printing any results from the first batch (likely none), then go until that cursor does not have a next result. If there is no activity on 'foo', that means issuing one getMore, which will wait for the default 1 second, then receiving an empty batch and reporting that there are no more results (returning false from DBCommandCursor::hasNext()). The cursor is still open, which isn't really obvious to the user. What the user sees is the prompt disappearing for that one second, then re-appearing one second later.

Generated at Thu Feb 08 04:29:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.