[JAVA-2411] MongoDB: java.lang.IllegalStateException: state should be: open Created: 15/Dec/16 Updated: 27/Oct/23 Resolved: 28/Dec/16 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | None |
| Affects Version/s: | 3.3.0 |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Evgeny Grin | Assignee: | Unassigned |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Description |
|
I have an application that pulls data from one source and then upserts it to MongoDB. Sometimes the pulling process takes a lot of time and when app tries to push a row to MongoDB the following exception is raised probably to the timeout issue:
There is a wrapper for MongoCollection class that for each MongoException and now for java.lang.IllegalStateException thrown
tries to reload MongoClient:
However, when the mongoClient is updated after "com.mongodb.MongoSocketReadException: Prematurely reached end of stream" it fails with:
and continues failing all the time... I use mongoDB ver 3.2.9 Per request, the RetryingMongoCollection.java starts with:
Then we override each and every com.mongodb.client.MongoCollection<T> method the same way as we do with updateOne method. |
| Comments |
| Comment by Jeffrey Yemin [ 19/Dec/16 ] |
|
Hi Evgeny, Currently the driver never retries an operation, leaving it to the application to determine whether to do so. Regards, |
| Comment by Evgeny Grin [ 18/Dec/16 ] |
|
Hi Jeff, in addition to my last question: I get today an exception "com.mongodb.MongoNotPrimaryException: The server is not the primary and did not execute the operation". As I mentioned previously, I have provided all three replica set members to Java client. Should not Java client to treat this issue by itself? Is there any reason that this error message came up to my handling? |
| Comment by Evgeny Grin [ 16/Dec/16 ] |
|
The question that i have than is - if the driver knows how to treat replica set failover why we catch this exception? Should not it be treated internally in the driver code? |
| Comment by Jeffrey Yemin [ 16/Dec/16 ] |
|
Yes, you're right about the "Prematurely reached end of stream". It can certainly happen during a replica set failover. The driver handles replica set failover automatically. A single-threaded application should expect to receive a single failure for any operation that's in progress when the failover occurs. Subsequent operations should block until the driver discovers the new primary. The gory details are described in two specifications that we've published : Server Discovery and Monitoring and Server Selection. |
| Comment by Evgeny Grin [ 16/Dec/16 ] |
|
Furthermore, today I guess I figured out the reason for the initial issue - "Prematurely reached end of stream". |
| Comment by Evgeny Grin [ 16/Dec/16 ] |
|
Hi Jeff, The issue I posted occurred while running in one thread. Actually, I already did that and the log now looks like as in the attached file (logs_mdb.txt) As you can see it always reconnect. I am afraid that it can cause to a huge number of opened connections from both sides: server & client. I agree regarding the last point. That's the reason that we catch only MongoDB exceptions. In case it is network or system issue the re-applying won't happen. |
| Comment by Jeffrey Yemin [ 15/Dec/16 ] |
|
Are you using an instance of RetryingMongoCollection concurrently from multiple threads? If so, that would explain the second exception, which the MongoClient throws when an application attempts to use it after it's been closed. Basically there is a race condition in which one thread closes a MongoClient instance that is still in use by other threads. I think the simplest solution here is to simply not close the MongoClient when there is an exception. The MongoClient class is designed to recover from any sporadic failure to connect to the MongoDB cluster to which it is connected, so applications are free to simply retry an operation using the same MongoClient instance. One other note of caution: not all updates are idempotent. It's not safe in the general case to simply retry all updates until thy succeeds, as it may end up being applied more than once. Consider for example an update that uses the $inc operator. The application could write the update command to the socket, the server receives it and applies it, and connectivity is disrupted before the application reads the response. If the application simply retries that update, the increment will end up being applied twice. |