[JAVA-3710] sporadic "com.mongodb.MongoException: state should be: open" Created: 15/Apr/20  Updated: 28/Oct/23  Resolved: 15/Jun/20

Status: Closed
Project: Java Driver
Component/s: Async
Affects Version/s: 2.8.0
Fix Version/s: 3.12.6, 4.0.5

Type: Bug Priority: Major - P3
Reporter: Yann Simon Assignee: John Stewart (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JAVA-3894 executeQuery (find or aggregate) that... Closed

 Description   

We are using a mongo query Observable, converted to a reactive stream Publisher based on this code: https://github.com/mongodb/mongo-scala-driver/blob/master/examples/src/test/scala/reactivestreams/Implicits.scala

By using this in a akka-stream flow, we have this error:
java.lang.IllegalStateException: Shutting down because of violation of the Reactive Streams specification.
at akka.stream.impl.fusing.GraphInterpreterShell.tryAbort(ActorGraphInterpreter.scala:654)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:770)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:783)
at akka.actor.Actor.aroundReceive(Actor.scala:534)
at akka.actor.Actor.aroundReceive$(Actor.scala:532)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:690)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:573)
at akka.actor.ActorCell.invoke(ActorCell.scala:543)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:269)
at akka.dispatch.Mailbox.run(Mailbox.scala:230)
at akka.dispatch.Mailbox.exec(Mailbox.scala:242)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)
Caused by: akka.stream.impl.ReactiveStreamsCompliance$SignalThrewException: It is illegal to throw exceptions from request(), rule 3.16
at akka.stream.impl.ReactiveStreamsCompliance$.tryRequest(ReactiveStreamsCompliance.scala:125)
at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary.dequeue(ActorGraphInterpreter.scala:164)
at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary.onNext(ActorGraphInterpreter.scala:194)
at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary$OnNext.execute(ActorGraphInterpreter.scala:97)
at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute(ActorGraphInterpreter.scala:53)
at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute$(ActorGraphInterpreter.scala:51)
at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary$OnNext.execute(ActorGraphInterpreter.scala:94)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:599)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:768)
... 14 common frames omitted
Caused by: com.mongodb.MongoException: state should be: open
at com.mongodb.MongoException.fromThrowableNonNull(MongoException.java:79)
at com.mongodb.async.client.AbstractSubscription.onError(AbstractSubscription.java:135)
at com.mongodb.async.client.AbstractSubscription.tryProcessResultsQueue(AbstractSubscription.java:174)
at com.mongodb.async.client.AbstractSubscription.request(AbstractSubscription.java:89)
at org.mongodb.scala.ObservableImplicits$BoxedSubscription.request(ObservableImplicits.scala:494)
at de.commercetools.sphere.persistence.mongo.MongoRxStreams$ObservableToPublisher$$anon$1$$anon$2.request(MongoRxStreams.scala:29)
at akka.stream.impl.ReactiveStreamsCompliance$.tryRequest(ReactiveStreamsCompliance.scala:122)
... 22 common frames omitted
Caused by: java.lang.IllegalStateException: state should be: open
at com.mongodb.assertions.Assertions.isTrue(Assertions.java:72)
at com.mongodb.operation.AsyncQueryBatchCursor.setBatchSize(AsyncQueryBatchCursor.java:145)
at com.mongodb.async.client.MongoIterableSubscription.requestMoreData(MongoIterableSubscription.java:88)
at com.mongodb.async.client.AbstractSubscription.processResultsQueue(AbstractSubscription.java:227)
at com.mongodb.async.client.AbstractSubscription.tryProcessResultsQueue(AbstractSubscription.java:172)
... 26 common frames omitted
This error is sporadic. It happens every few minutes, on a multi-tenant platform having several hundert requests per minute.

The mongo scala driver is using the java driver mongodb-driver-async in the version 3.12.2. We had this issue also with older versions. We tried updating without any success.

Any hint would be helpful.

I see that there is now a reactive driver but might not have this issue. We cannot upgrade now but might consider this in the future.

I'll looking for ways to fix the issue with the current drivers.



 Comments   
Comment by Yann Simon [ 24/Jul/20 ]

I can confirm this issue is fixed.

 

We are using the mongo scala driver that is still using an older version of mongodb-driver-async.

https://search.maven.org/artifact/org.mongodb.scala/mongo-scala-driver_2.13/2.9.0/jar

So we had to force an update of this transitive dependency.

Thanks a lot for the fix!

 

It'd be great to update the dependencies of the mongo scala driver so that users can profit from the fix automatically.

Comment by Githook User [ 08/Jul/20 ]

Author:

{'name': 'John Stewart', 'email': 'john.stewart@mongodb.com', 'username': 'jstewart-mongo'}

Message: Avoid calling batch cursor when the cursor is closed

JAVA-3710
Branch: 4.0.x
https://github.com/mongodb/mongo-java-driver/commit/89079b12f5fbf650480ac1b47b8ce6691b700ce0

Comment by Githook User [ 30/Jun/20 ]

Author:

{'name': 'John Stewart', 'email': 'john.stewart@mongodb.com', 'username': 'jstewart-mongo'}

Message: Avoid calling batch cursor when the cursor is closed

JAVA-3710
Branch: 3.12.x
https://github.com/mongodb/mongo-java-driver/commit/f015023db246b8cec15e7bef711c9d8be3e02828

Comment by Yann Simon [ 15/Jun/20 ]

Thx for the fix!

Will it be backported in the 3.x branch?

Comment by Githook User [ 15/Jun/20 ]

Author:

{'name': 'John Stewart', 'email': 'john.stewart@mongodb.com', 'username': 'jstewart-mongo'}

Message: Avoid calling batch cursor when the cursor is closed

JAVA-3710
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/69ef5be50c8ac47a989dcf92f230c837f4454df3

Comment by John Stewart (Inactive) [ 30/Apr/20 ]

yann.simon.fr@gmail.com It is a change that needs to be implemented in the driver.

Comment by Yann Simon [ 30/Apr/20 ]

Thx for the quick feedback.

Is it a change I could do myself, for example in the [ObservableToPublisher|https://github.com/mongodb/mongo-scala-driver/blob/master/examples/src/test/scala/reactivestreams/Implicits.scala?]

Or is it a change that has to be done in the driver itself?

Comment by John Stewart (Inactive) [ 30/Apr/20 ]

yann.simon.fr@gmail.com The fix found in JAVA-3487 needs to be applied to other files to prevent the exception.

Comment by Yann Simon [ 30/Apr/20 ]

For info, I've tried to use monix instead of akka-stream to check if that can fix the issue.

But sadly the issue is still occuring.

Comment by Yann Simon [ 30/Apr/20 ]

Today we had a similar issue:

java.lang.IllegalStateException: state should be: open
	at com.mongodb.assertions.Assertions.isTrue(Assertions.java:72)
	at com.mongodb.internal.connection.BaseCluster.selectServerAsync(BaseCluster.java:149)
	at com.mongodb.internal.connection.AbstractMultiServerCluster.selectServerAsync(AbstractMultiServerCluster.java:54)
	at com.mongodb.async.client.ClientSessionHelper.createClientSession(ClientSessionHelper.java:68)
	at com.mongodb.async.client.ClientSessionHelper.withClientSession(ClientSessionHelper.java:51)
	at com.mongodb.async.client.OperationExecutorImpl.execute(OperationExecutorImpl.java:68)
	at com.mongodb.async.client.MongoIterableImpl.batchCursor(MongoIterableImpl.java:167)
	at com.mongodb.async.client.MongoIterableSubscription.requestInitialData(MongoIterableSubscription.java:46)
	at com.mongodb.async.client.AbstractSubscription.tryRequestInitialData(AbstractSubscription.java:164)
	at com.mongodb.async.client.AbstractSubscription.request(AbstractSubscription.java:87)
	at org.mongodb.scala.ObservableImplicits$BoxedSubscription.request(ObservableImplicits.scala:494)
	at org.mongodb.scala.internal.FoldLeftObservable$$anon$1$$anon$2.request(FoldLeftObservable.scala:42)
	at org.mongodb.scala.ObservableImplicits$ScalaObservable$$anon$2.onSubscribe(ObservableImplicits.scala:367)
	at org.mongodb.scala.ObservableImplicits$ToSingleObservable$$anon$3.onSubscribe(ObservableImplicits.scala:445)
	at org.mongodb.scala.internal.SubscriptionCheckingObserver.onSubscribe(SubscriptionCheckingObserver.scala:28)
	at org.mongodb.scala.internal.MapObservable$$anon$1.onSubscribe(MapObservable.scala:28)
	at org.mongodb.scala.internal.SubscriptionCheckingObserver.onSubscribe(SubscriptionCheckingObserver.scala:28)
	at org.mongodb.scala.internal.FoldLeftObservable$$anon$1.onSubscribe(FoldLeftObservable.scala:47)
	at org.mongodb.scala.internal.SubscriptionCheckingObserver.onSubscribe(SubscriptionCheckingObserver.scala:28)
	at org.mongodb.scala.Observer.onSubscribe(Observer.scala:85)
	at org.mongodb.scala.Observer.onSubscribe$(Observer.scala:85)
	at org.mongodb.scala.internal.SubscriptionCheckingObserver.onSubscribe(SubscriptionCheckingObserver.scala:21)
	at com.mongodb.async.client.MongoIterableSubscription.<init>(MongoIterableSubscription.java:40)
	at com.mongodb.async.client.Observables$1.subscribe(Observables.java:47)
	at org.mongodb.scala.ObservableImplicits$BoxedObservable.subscribe(ObservableImplicits.scala:478)
	at org.mongodb.scala.FindObservable.subscribe(FindObservable.scala:344)
	at org.mongodb.scala.internal.FoldLeftObservable.subscribe(FoldLeftObservable.scala:27)
	at org.mongodb.scala.internal.MapObservable.subscribe(MapObservable.scala:25)
	at org.mongodb.scala.ObservableImplicits$ToSingleObservable.subscribe(ObservableImplicits.scala:437)
	at org.mongodb.scala.ObservableImplicits$ScalaObservable.headOption(ObservableImplicits.scala:359)
	at org.mongodb.scala.ObservableImplicits$ScalaObservable.head(ObservableImplicits.scala:345)
	at org.mongodb.scala.ObservableImplicits$ScalaObservable.toFuture(ObservableImplicits.scala:337)

by using ".toFuture()"

The calling code:

coll.find(new BasicDBObject("_id", new BasicDBObject("$in", ids)))
  .sort(new BasicDBObject("_id", 1))
  .toFuture()

Comment by Yann Simon [ 30/Apr/20 ]

Thx for the update.

https://jira.mongodb.org/browse/JAVA-3487 is fixed in 3.12.1

We are already using org.mongodb:mongodb-driver-async:3.12.2

 

Comment by John Stewart (Inactive) [ 29/Apr/20 ]

The fix will be similar to the one found in JAVA-3487.

Generated at Thu Feb 08 09:00:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.