[SERVER-48453] Lazily initialize a record store's auto incrementing counter on deletes Created: 27/May/20  Updated: 29/Oct/23  Resolved: 29/May/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.0.20, 4.4.0-rc8, 4.2.9, 4.7.0

Type: Bug Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Daniel Gottlieb (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-48603 Rollback via refetch can result in ou... Backlog
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4, v4.2, v4.0
Sprint: Execution Team 2020-06-01, Execution Team 2020-06-15
Participants:
Linked BF Score: 14

 Description   

Apologies for summarizing the solution and not the problem.

It's imperative that the update chain for a storage engine key goes in increasing timestamp order (with only a few explicit caveats). The lazy initializing of record ids only on inserts creates a scenario where a key can be reused, and in the case of the _mdb_catalog, can result in an update that can't be rolled back (intentional 0-timestamp value) sitting on top of an update that must be eligible for rollback. This sequence of updates "locks in" a non-majority committed update.

The scenario:

  • Create collection A. Inserts into _mdb_catalog (initializing the counter to "1") with RecordId(1), Timestamp(10)
  • Drop collection A. Timestamp(20)
  • Rollback to 15
  • Replay drop collection A (No more records in the _mdb_catalog)
  • Create a non-replicated collection (e.g: `local.replset.initialsync`). Initialize the counter to "1", Insert Record(1), Timestamp(0)
  • Rollback to 15. The drop at timestamp 20 is "locked in". Collection A is missing


 Comments   
Comment by Githook User [ 16/Jul/20 ]

Author:

{'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'username': 'dgottlieb'}

Message: SERVER-48453: Prevent reusing record ids within a process/rollback lifetime.

(cherry picked from commit 82be4b070e723683916733d903a678ae0e29ad91)
Branch: v4.0
https://github.com/mongodb/mongo/commit/dd37bf1c363dc073373c7d86ff55e76578ee97b3

Comment by Githook User [ 10/Jul/20 ]

Author:

{'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'username': 'dgottlieb'}

Message: SERVER-48453: Prevent reusing record ids within a process/rollback lifetime.

(cherry picked from commit 82be4b070e723683916733d903a678ae0e29ad91)
Branch: v4.2
https://github.com/mongodb/mongo/commit/70257260e5f4f3b44d9759e83f88ed2bd411159e

Comment by Githook User [ 01/Jun/20 ]

Author:

{'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'username': 'dgottlieb'}

Message: SERVER-48453: Prevent reusing record ids within a process/rollback lifetime.

(cherry picked from commit 82be4b070e723683916733d903a678ae0e29ad91)
Branch: v4.4
https://github.com/mongodb/mongo/commit/7be1baa039fa78f583171ea8cc630a93c038e719

Comment by Githook User [ 29/May/20 ]

Author:

{'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'username': 'dgottlieb'}

Message: SERVER-48453: Prevent reusing record ids within a process/rollback lifetime.
Branch: master
https://github.com/mongodb/mongo/commit/82be4b070e723683916733d903a678ae0e29ad91

Comment by Daniel Gottlieb (Inactive) [ 27/May/20 ]

Marking for backport through 4.0 as this only affects rollback_to_stable. I think the reconfigure project in 4.4 "accidentally" added test coverage which exposed this bug, which we can be used as inspiration for a targeted test that demonstrates a failure on earlier versions.

Generated at Thu Feb 08 05:17:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.