[SERVER-38461] performSingleUpdateOpWithDupKeyRetry loops indefinitely Created: 07/Dec/18  Updated: 29/Oct/23  Resolved: 12/Dec/18

Status: Closed
Project: Core Server
Component/s: Querying, Write Ops
Affects Version/s: None
Fix Version/s: 4.1.7

Type: Bug Priority: Critical - P2
Reporter: Martin Neupauer Assignee: James Wahlin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-12267 Docs for SERVER-38461: performSingleU... Closed
Related
is related to SERVER-37124 Retry full upsert path when duplicate... Closed
is related to SERVER-38494 EphemeralForBtreeImplTestHarnessHelpe... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

db.testColl.createIndex({ x: 1 }, { unique: true })
db.testColl.insert({_id: 1, 'a': 12345 });
db.testColl.update({ x: 3 }, {}, { upsert: true } )

Sprint: Query 2018-12-17
Participants:
Linked BF Score: 50

 Description   

UpdateStage::shouldRetryDuplicateKeyException should not return true for the replacement style upserts. See the repro.



 Comments   
Comment by Ian Whalen (Inactive) [ 13/May/19 ]

Not changing driver specs here since driver specs state that errInfo is a BSON document and don't make assumptions about its structure.

Comment by Githook User [ 12/Dec/18 ]

Author:

{'name': 'James Wahlin', 'email': 'james@mongodb.com', 'username': 'jameswahlin'}

Message: SERVER-38461 Limit upsert retry to DuplicateKey violations with matching values
Branch: master
https://github.com/mongodb/mongo/commit/a34b0ac74038a86f22de1d70c58f2f7c51c1a42a

Comment by Andy Schwerin [ 09/Dec/18 ]

Thanks!

Comment by James Wahlin [ 09/Dec/18 ]

I believe the correct way to fix this is not to ban retry of replacement-style upsert. Instead we should ban retry of upserts that fail on DuplicateKey error where the to-be-written document does not contain all of the unique index fields with the same values used in the query predicate.

For example, here is a replacement-style upsert that can be retried. This is because the replacement document contains the unique index field and value:

db.testColl.createIndex({ x: 1 }, { unique: true });
db.testColl.insert({_id: 1, 'a': 12345 });
db.testColl.update({ x: 3 }, {x: 3}, { upsert: true } );

And the following is an update-style upsert that will trigger an infinite retry loop. This is because the update is writing a value of 'x' that already exists as part of another document.

db.test.createIndex({x: 1}, {unique: true});
db.test.insert({x: 3});
db.test.insert({x: 4});
db.test.update({x: 3}, {$inc: {x: 1}}, {upsert: true});

 

Comment by James Wahlin [ 09/Dec/18 ]

The problem encountered here is that with a replacement-style upsert, the field / fields in your query predicate ({x: 3} for this repro) is not what will be inserted into the collection. This breaks the assumption that our retry mechanism makes, which is that you are inserting to or updating a document with the same unique index fields as the query predicate. 

So to break down how the reproduction fails:

1) A unique index is created on field 'x'

 db.testColl.createIndex({ x: 1 }, { unique: true });

2) A document is inserted without an 'x' field, resulting in an index key with a value of undefined

db.testColl.insert({_id: 1, 'a': 12345 });

3) An upsert is performed with query predicate of {x: 3}, which does not exist in the index. Given this we attempt to insert a document. As this is a replacement upsert, with no 'x' entry we will attempt to insert a second index key with a value of undefined. This fails on DuplicateKey error. As the query predicate and unique index keys match, we will attempt to retry. We then end up in a loop where we fail repeatedly in the same manner.

db.testColl.update({ x: 3 }, {}, { upsert: true } )

 

Comment by Andy Schwerin [ 09/Dec/18 ]

Could you provide a little more explanation? Should the final upsert in the retro succeed or fail after this change?

Generated at Thu Feb 08 04:49:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.