[JAVA-4849] MongoException: next() called after the cursor was closed Created: 18/Jan/23  Updated: 28/Oct/23  Resolved: 03/Feb/23

Status: Closed
Project: Java Driver
Component/s: None
Affects Version/s: 4.8.0
Fix Version/s: 4.9.0

Type: Question Priority: Major - P3
Reporter: Sam Weston Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Summary

When I run several async requests against Mongo and then cancel after receiving the first result, I get the error:

ERROR reactor.core.publisher.Operators - Operator called default onErrorDropped
reactor.core.Exceptions$ErrorCallbackNotImplemented: com.mongodb.MongoException: next() called after the cursor was closed.
Caused by: com.mongodb.MongoException: next() called after the cursor was closed.

I am using Rx Java .firstElement() in order to cancel unwanted requests.

 
This is using...
driver version: 4.8.0
Mongo version: 6
Topology: replica set

How to Reproduce

I cannot provide exact steps to reproduce the database but the query below matches a couple of entries in the database which are indexed. This code triggers the exception every time. 

 

Flowable.range(1, 200)
    .flatMapMaybe(
        iter ->
            Flowable.fromIterable(Vector.of(mainCollection, mainCollection))
                .map(
                    dataBaseName ->
                        mongoClient.getDatabase(dataBaseName).getCollection(collectionName))
                .flatMap(collection -> Flowable.fromPublisher(collection.find()))
                .firstElement())
    .collect(Vector.collector())
    .blockingGet();

  
 

Additional Background

It appears to be the double nesting of Flowables that helps trigger this bug. With a very large flat list of queries it doesn't trigger.



 Comments   
Comment by Githook User [ 03/Feb/23 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Ensure BatchCursorFlux does not drop an error

The BatchCursor class can be closed by the BatchCursorFlux class
but there is no signalling for the BatchCursor onNext publisher.

This leads to a next() called after the cursor was closed error and
that breaks the reactive streams spec as the error ends up being dropped.

JAVA-4849
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/617432e8aee1d7a272e15a8d330d042531336c73

Comment by Sam Weston [ 01/Feb/23 ]

Thanks Ross, we really appreciate you looking in to it.

Comment by Ross Lawley [ 01/Feb/23 ]

Hi sam@unlikely.ai,

is it not a fairly standard scenario to kick off a series of requests and then cancel some of them before they return?

Its the first report we've had of this scenario, so I would say its a little unusual or just under reported. However, I can see our use of the BatchCursor here allows it to signal an error that ends up being dropped. I was able to determine the cause and look to have a fix in the next release.

Ross

Comment by Sam Weston [ 01/Feb/23 ]

Hi Ross,

Thank you very much for looking into this for us. I understand the explanation and it's reassuring to know it's not functionally affecting the code. The error seems a little trigger-happy, is it not a fairly standard scenario to kick off a series of requests and then cancel some of them before they return? How do you suggest dealing with this, should I silence this particular error since it's always a false alarm? For context, we need the nesting and would like to keep things general, so putting the 

.first()

 at the lower level is not ideal. I do very much appreciate the suggested fixes though.

Many thanks,
Sam

Comment by Ross Lawley [ 31/Jan/23 ]

Hi sam@unlikely.ai,

Interesting question! And as often with async your code has uncovered a race condition, which results in an error being logged. The race condition is due to more data being requested from MongoDB than required and then cancelling the subscription. This ends up with the cursor being closed while data has been requested from MongoDB and then once the data has returned the error is logged. It is just a side effect and shouldn't impact the correctness of the code.

Why this happens is slightly in the weeds:

Flowable.range(1, 200)
  .flatMapMaybe(iter -> 
    Flowable.fromIterable(Vector.of(mainCollection, mainCollection))
            .map(dataBaseName ->mongoClient.getDatabase(dataBaseName).getCollection(collectionName))
            .flatMap(collection -> Flowable.fromPublisher(collection.find(bsonFilter)))
            .firstElement())
  .collect(Vector.collector())
  .blockingGet();

.flatMap(collection -> Flowable.fromPublisher(collection.find(bsonFilter))) ends up requesting 128 results from the publisher due to using rxJava's default buffer size.
This causes the MongoDB Java driver to make multiple calls to the server to fulfil this demand. However, firstElement() abruptly cancels the subscription as soon as it gets a result and this results in the MongoDB cursor being closed. If the results comeback from the server after the cursor has been closed an error is reported - which is what you are seeing logged.

The following code (non nested) does not report an error because the downstream request to the MongoDB driver is always one.

        List<Document> documents = Flowable.range(1, 200)
                .flatMapMaybe(i -> Flowable.fromPublisher(collection.find()).firstElement())
                .collect(Collectors.toList())
                .blockingGet();

If you force the MongoDB find publisher to only return a single item it removes the demand race condition even if its nested.

Flowable.range(1, 200)
  .flatMapMaybe(iter -> 
    Flowable.fromIterable(Vector.of(mainCollection, mainCollection))
            .map(dataBaseName ->mongoClient.getDatabase(dataBaseName).getCollection(collectionName))
            .flatMap(collection -> Flowable.fromPublisher(collection.find(bsonFilter).first()))
            .firstElement())
  .collect(Vector.collector())
  .blockingGet();

Note: collection.find(bsonFilter).first() will essentially return a Single rather than an unbound Observable.

I hope that helps,

Kind regards,

Ross

Comment by Esha Bhargava [ 20/Jan/23 ]

sam@unlikely.ai Thank you for reporting this issue. We'll look into it and get back to you soon.

Generated at Thu Feb 08 09:03:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.