[JAVA-3608] updateOne blocked due to ?retryWriter=true and WriteConcern.UNACKNOWLEDGED combination Created: 27/Jan/20  Updated: 25/Feb/20  Resolved: 25/Feb/20

Status: Closed
Project: Java Driver
Component/s: Write Operations
Affects Version/s: 3.12.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Hendy Irawan Assignee: John Stewart (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Java 8, MongoDB Atlas M5 4.2.2


Issue Links:
Related
related to CSHARP-2970 Deadlock using Atlas Closed

 Description   

There is a very specific combination of factors that must be fulfilled for this to happen:

  1. Use mongodb-driver-sync 3.12.0 or 3.12.1
  2. Connect with ?retryWrites=true
  1. Do a write to a document with WriteConcern.UNACKNOWLEDGED. Driver will log that retryWrites=true cannot be used by unacknowledged.

e.g. 

userColl.withWriteConcern(WriteConcern.UNACKNOWLEDGED).updateMany(in("_id", ids), inc("impressionCount", 1));

db.user.updateMany({_id: [466]}, {$inc: {impressionCount: 1, {w: 0});}}

  1. Do another update with Java for the same document, this time with writeConcern of 1.

db.user.updateOne( {_id: 466}, { $set: { 'email': 'hendy@hendyirawan.com'}}, {w: 1});

  1. At this point the previous updateOne() is locked and will never return.
  2. Also, pressing Ctrl+C does not terminate the app. Java app cannot exit. Doing a kill -9 will make connections stuck on the MongoDB server (at least on MongoDB Atlas M10 v4.2.2 it does).

Note that this bug is not reproducible when using mongo shell, so I think this bug is specific to MongoDB Java Driver. At first I thought it is due to codecs or thread safety or server busy, but even just updating a string reproduces the issue.

Workaround:

Don't use WriteConcern.UNACKNOWLEDGED (with ?retryWrites=true).  As long as I update always with w=1 or w=majority, there is no issue.



 Comments   
Comment by John Stewart (Inactive) [ 25/Feb/20 ]

hendy@hendyirawan.com The cloud team was able to identify the cause of the issue and they have opened the ticket CLOUDP-58004 to work on this. You can follow that ticket for further updates. If that ticket fails to resolve this issue, I will reopen this bug.

Comment by Hendy Irawan [ 11/Feb/20 ]

Thank you John !

Comment by John Stewart (Inactive) [ 11/Feb/20 ]

hendy@hendyirawan.com I was able to reproduce the hang in the mongo shell with an M5 instance. After executing the update with w=0, I get the response acknowledged: false and then a hang:
 

MongoDB Enterprise Sandbox-shard-0:PRIMARY> db.restaurants.updateOne({_id: 666}, {$set: {'cuisine': 'Italian'}}, {w: 0})
{ "acknowledged" : false }

 I am then unable to execute any other commands. Entering Ctrl-C generates an exception and dumps a backtrace. I am using the mongo shell version 4.2.2.

Comment by Hendy Irawan [ 10/Feb/20 ]

Thank you very much @John ! And that is good news... I hope you'll find the root cause soon

Comment by John Stewart (Inactive) [ 10/Feb/20 ]

I am able to replicate this issue only with an M5 instance. I am investigating further.

Comment by Hendy Irawan [ 04/Feb/20 ]

Did you get the driver complaining "Driver will log that retryWrites=true cannot be used with unacknowledged."?

 

So far the difference is:

  1. I used Atlas shared M5 instance (my gut feeling is this may have an effect)
  2. my connection string has w=1 and uses SRV 
Comment by John Stewart (Inactive) [ 03/Feb/20 ]

hendy@hendyirawan.com I created a test based on your description and ran it successfully against a 4.2.2 replica set. Since retryWrites=true is the default, I ran the test with and without ?retryWrites=true in the connection string, but I could not reproduce your issue. Here is my test:

 

MongoClient mongoClient = MongoClients.create("mongodb://localhost:27017,localhost:27018,localhost:27019/?retryWrites=true");
MongoDatabase db = mongoClient.getDatabase("test");
db.getCollection("restaurants").deleteOne(eq("_id", 666));
db.getCollection("restaurants").insertOne(new Document("_id", new BsonInt32(666))
 .append("cuisine", "something"));
db.getCollection("restaurants").withWriteConcern(WriteConcern.UNACKNOWLEDGED)
 .updateMany(new Document("_id", new BsonInt32(666)),
   new Document("$set", new Document("cuisine", "Category To Be Determined")));
db.getCollection("restaurants").withWriteConcern(WriteConcern.W1)
 .updateOne(new Document("_id", new BsonInt32(666)),
   new Document("$currentDate", new Document("lastModified", true)));
mongoClient.close();

Could you point out the differences between my test and your scenario? Thanks.

Generated at Thu Feb 08 09:00:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.