[SERVER-45347] 2-phase index build on empty collection(implicit collection creation) can skip rebuilding the index on rollback/startup recovery. Created: 03/Jan/20  Updated: 29/Oct/23  Resolved: 22/Jan/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.3.3

Type: Bug Priority: Major - P3
Reporter: Suganthi Mani Assignee: Suganthi Mani
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-38736 Optimize simultaneous index builds to... Closed
Related
related to SERVER-44953 Secondaries should restart index buil... Closed
related to SERVER-45064 Stop replicating startIndexBuild and ... Closed
is related to SERVER-45201 Implicit collection creation from cre... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Execution Team 2020-01-27
Participants:
Linked BF Score: 31

 Description   

Currently 2-phase index build for empty collection does the following steps.

1) We reserve an oplog slot with timestamp say TS(10), and write the "startIndexBuild" oplog entry with that timestamp.
2) We reserve another oplog slot with timestamp say TS(11) and use that to write commitIndexBuild oplog entry. To be noted, As a result of this step, all future writes done to that WT recovery unit will use timestamp TS(11).
3) Now, we start the index build which would update the catalog table to write the index entry with "ready" field as false using commitIndexBuild oplog entry's timestamp (TS(11)).
4) On Index completion/success, we would again update the catalog table to write the index entry with "ready" field as true using commitIndexBuild oplog entry's timestamp (TS(11)).

 Now, suppose if my stable timestamp/ checkpoint timestamp is TS(10) (i.e, startIndexBuild oplog entry), then it would miss rebuilding the index while replaying the commitIndex oplog entry during rollback and startup recovery. Because, when the node tries to replay the "commitIndexBuild" oplog entry during rollback/startup recovery, commitIndexBuild Oplog entry will fail with ErrorCodes::NoSuchKey since it won't be able to find any corresponding running/in-progress index. Also, when commitIndexBuild oplog entry fails with ErrorCodes::NoSuchKey, we don't escalate it to the caller (oplog applier), instead we suppress it as "ErrorCodes:NoSuchKey" is an acceptable error code during oplog application.



 Comments   
Comment by Githook User [ 22/Jan/20 ]

Author:

{'username': 'smani87', 'name': 'Suganthi Mani', 'email': 'suganthi.mani@mongodb.com'}

Message: SERVER-45347 Primaries should generate createIndexes oplog entry (single phase index build) for empty
collections.
Branch: master
https://github.com/mongodb/mongo/commit/3d7cb72c831478e995ec6c521f428b66f8081d34

Comment by Benety Goh [ 13/Jan/20 ]

The empty collection optimization to be introduced in SERVER-45201 should be updated to use IndexBuildsCoordinator::createIndexesOnEmptyCollection().

Comment by Benety Goh [ 10/Jan/20 ]

suganthi.mani proposed one possible solution that would eliminate any ambiguity with start/commit timestamps with index builds on empty collections - IndexBuildsCoordinator::createIndexesOnEmptyCollection() would replicate index builds as single phased.

Comment by Benety Goh [ 10/Jan/20 ]

louis.williams observed while reviewing SERVER-45201 that IndexCatalogImpl::createIndexOnEmptyCollection() does not provide a buildUUID when it updates the catalog.

We should ensure that a buildUUID is provided as we work on this ticket to improve the startIndexBuild/commitIndexBuild semantics on empty collections.

Generated at Thu Feb 08 05:08:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.