[SERVER-34494] Background index builds on secondaries should set the collection minimumVisibleSnapshot when they start Created: 16/Apr/18  Updated: 29/Oct/23  Resolved: 17/Apr/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 3.7.4
Fix Version/s: 3.7.5

Type: Bug Priority: Major - P3
Reporter: Louis Williams Assignee: Louis Williams
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Storage NYC 2018-04-23
Participants:
Linked BF Score: 18

 Description   

During batch application, starting a background index build on a secondary does not set the minimumVisibleSnapshot on the collection until the index build finishes. If a secondary reader without the PBWM lock reads at a timestamp A before an index build timestamped for B, a reader may iterate through the indexes for a collection and reach this invariant.

The problem is that _setupInMemoryStructures is called before the actual catalog entry is updated on disk, so a reader may see an in-memory state inconsistent with the state on disk. This can be prevented by updating the minimumVisibleSnapshot on the collection to force readers to wait for a later timestamp.



 Comments   
Comment by Githook User [ 17/Apr/18 ]

Author:

{'name': 'Louis Williams', 'email': 'louis.williams@mongodb.com', 'username': 'louiswilliams'}

Message: SERVER-34494 Background index builds on secondaries should set the collection minimumVisibleSnapshot
Branch: master
https://github.com/mongodb/mongo/commit/228106a7416288ad6263ab67878477cc23e9c23f

Comment by Daniel Gottlieb (Inactive) [ 16/Apr/18 ]

The problem is that _setupInMemoryStructures is called before the actual catalog entry is updated on disk, so a reader may see an in-memory state inconsistent with the state on disk. This can be prevented by updating the minimumVisibleSnapshot on the collection to force readers to wait for a later timestamp.

That's not how I had it. That paragraph suggests there's a lack of locking where a reader can come in and view an inconsistent state.

I had thought that a writer could create the index entry in its "ready: false" state at a time T, but not set the minimumVisibleSnapshot field. This allows a reader to come in on a collection with a readTimestamp of T-1, causing the crash (because of the in-memory/"on disk" state). The crash still happens because of a split state, but as far as I know, there's sufficient locking to avoid the low-level race, if only a minimum visible snapshot were set (which is what's being suggested).

Generated at Thu Feb 08 04:36:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.