[SERVER-49408] adjust startup to resume index build from phase 2 (bulk load index table from sorted keys) Created: 09/Jul/20  Updated: 29/Oct/23  Resolved: 06/Aug/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Task Priority: Major - P3
Reporter: Benety Goh Assignee: Gregory Noma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-50095 Augment ResumableIndexBuildTest to en... Closed
Related
related to SERVER-50221 Keep NoLimitSorter aware of its in-me... Closed
related to SERVER-49163 extend internal index build interface... Closed
is related to SERVER-50131 Extend SorterTests to cover construct... Closed
is related to SERVER-58197 Pass ranges by reference when constru... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-07-27, Execution Team 2020-08-10
Participants:

 Description   

Currently, index builds are restarted from the beginning. To be able to resume an index build, we require the temporary tables used by the hybrid index build to be preserved on startup and the progress information in the internal index build table (one table per index build).

This ticket implements the resumable index build logic for index builds that were shut down during the bulk load phase. The internal ident for the index build should have the following phase:

{ _id: <build uuid>,
 phase: "bulk load",
 ...
}

 

If shutdown is initiated in the second phase (where the bulk loader is being used to build each index table, serially), the bulk load will be aborted and the same information, other than the "phase" value, is saved as from the first phase. On restart, the bulk load of all the tables will begin anew.  We must ensure the unfinished index tables are empty before starting a new bulk load, by dropping and recreating the tables.

If the second phase completes successfully, each Sorter temporary file is deleted.



 Comments   
Comment by Githook User [ 06/Aug/20 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-49408 Enable resuming index builds from the bulk load phase
Branch: master
https://github.com/mongodb/mongo/commit/f67a91df05719630d7893140dbb3a5edd45a642b

Generated at Thu Feb 08 05:19:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.