[SERVER-79033] Image collection invalidation for missing namespace during initial sync always attempts upsert Created: 17/Jul/23  Updated: 29/Oct/23  Resolved: 19/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 7.0.0-rc7
Fix Version/s: 7.1.0-rc0, 7.0.1

Type: Bug Priority: Major - P3
Reporter: Kaitlin Mahar Assignee: Kaitlin Mahar
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
is related to SERVER-76774 Invariant that secondary oplog applic... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0
Sprint: Repl 2023-07-24
Participants:
Linked BF Score: 118

 Description   

SERVER-73532 added logic to invalidate the image collection during initial sync when the namespace is missing.

This logic unconditionally passes in true for upsertConfigImage on each write conflict retry attempt.

However, this can cause problems in cases like the following: consider an oplog batch containing two oplog entries requiring pre-images with the same LSID and these timestamps:

1. t1: {"t": 1686186824, "i": 50}

2. t2: {"t": 1686186824, "i": 54}

We can apply entries in a batch out of order, and so it's possible we apply entry 2 first. In that case, we will write an invalidate entry to the image collection for the LSID with timestamp t2. 

Then, when we try to apply entry 1, we will query for an image collection entry with ts < t1. 

We won't see one, and so we will attempt to insert such an entry, and get a duplicate key error, and retry indefinitely.

There is already logic in the code to handle this issue though. We can opt into it by pulling the boolean upsertConfigImage up out of the writeConflictRetry block so that we won't overwrite the value on retry attempts and will instead update rather than insert on retry.

See this comment for some additional context. 



 Comments   
Comment by Githook User [ 18/Aug/23 ]

Author:

{'name': 'Kaitlin Mahar', 'email': 'kaitlin.mahar@mongodb.com', 'username': 'kmahar'}

Message: SERVER-79033 Image collection invalidation for missing namespace should not upsert on retry due to write conflict

(cherry picked from commit 18dd9b734865fc92c8a3da3b3d60a3c02212f0cc)
Branch: v7.0
https://github.com/mongodb/mongo/commit/aaaea1402449c86c656f1b9deab315c5a2de4912

Comment by Githook User [ 19/Jul/23 ]

Author:

{'name': 'Kaitlin Mahar', 'email': 'kaitlin.mahar@mongodb.com', 'username': 'kmahar'}

Message: SERVER-79033 Image collection invalidation for missing namespace should not upsert on retry due to write conflict
Branch: master
https://github.com/mongodb/mongo/commit/18dd9b734865fc92c8a3da3b3d60a3c02212f0cc

Generated at Thu Feb 08 06:39:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.