[SERVER-37643] add createIndexes command logic to the index build interface Created: 15/Oct/18  Updated: 29/Oct/23  Resolved: 01/Feb/19

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.1.8

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-37940 Establish a code path for FCV 4.0 and... Closed
is depended on by SERVER-39068 Replication of simultaneous index bui... Closed
Gantt Dependency
has to be done before SERVER-37951 Break index builds into phases and ad... Closed
has to be done before SERVER-37963 Update metadata command behaviors to ... Closed
has to be done before SERVER-37966 Do not wait for index build completio... Closed
has to be done before SERVER-39086 Move startup recovery index creation ... Closed
has to be done before SERVER-39314 Update IndexBuildManager unit testing Closed
has to be done after SERVER-37636 Create an interface through which to ... Closed
Problem/Incident
causes SERVER-39368 Refactor IndexBuildsCoordinator::_run... Closed
Related
related to SERVER-38745 MigrationDestinationManager assigns i... Closed
related to SERVER-45378 IndexBuildsCoordinator::_setUpIndexBu... Closed
related to SERVER-45933 2 phase index build running with maxT... Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage NYC 2018-12-17, Storage NYC 2018-12-31, Storage NYC 2019-01-14, Storage NYC 2019-01-28, Storage NYC 2019-02-11
Participants:
Story Points: 20

 Description   

The index builder interface established is established in SERVER-37636.

This ticket will add a Threadpool and move all the instances of MultiIndexBlock (index builder class) that are all over the place behind the interface and running on the Threadpool.

We should be able to register index builds via the interface and then wait upon a condition variable to hear back on the Status result.

Keep in mind SERVER-37644, which is to make index builds joinable via the createIndexes command. The condition variable setup must be such that we can have multiple waiters who can all hear back about the same result. Maybe an interface internal helper function to get something to wait upon for a Status result.



 Comments   
Comment by Githook User [ 01/Feb/19 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-37643 IndexBuildsCoordinatorMongod task inherits CurOp and OperationContext info from caller thread

This supports copying an OperationContext's deadline and timeout error across
thread boundaries.

This also adds OperationContext::getTimeoutError().
Branch: master
https://github.com/mongodb/mongo/commit/8387824b3ac937b0489fcb94c590cc663b47348c

Comment by Githook User [ 01/Feb/19 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-37643 implement IndexBuildsCoordinator::_runIndexBuild()

This provides a common implementation for _runIndexBuild in
IndexBuildsCoordinatorMongod and IndexBuildsCoordinatorEmbedded.

This commit also moves fail points from the createIndexes command to the
IndexBuildsCoordinator.
Branch: master
https://github.com/mongodb/mongo/commit/9be581f05602f7956ea8f501cbc774a6172c40f9

Comment by Githook User [ 01/Feb/19 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-37643 implement IndexBuildsManager functions for createIndexes
Branch: master
https://github.com/mongodb/mongo/commit/2f2432067b55d5a41f3e4103ee3cbfc1754f5d58

Comment by Githook User [ 01/Feb/19 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-37643 update IndexBuildsCoordinator and IndexBuildsManager interfaces.

rename IndexBuildsCoordinator::buildIndex() to startIndexBuild()
add ReplIndexBuildState::dbName and use in IndexBuildsCoordinator registration
add IndexBuildsManager::isBackgroundBuilding()
remove unused IndexBuildsManager::finishConstraintPhase()
add IndexBuildsManager::drainBackgroundWrites()
remove nss argument from IndexBuildsManager::setUpIndexBuild()
Branch: master
https://github.com/mongodb/mongo/commit/066d681d4c0377da30aa0fc262826d67e9b1ad69

Comment by Githook User [ 24/Jan/19 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-37643 fix dbtests to avoid creating indexes using DBClient while holding locks
Branch: master
https://github.com/mongodb/mongo/commit/7106a8ed5f278c81e10b94a3501b8d3992b4a97e

Comment by Githook User [ 18/Jan/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 refine IndexBuildsCoordinator and IndexBuildsManager interfaces to support createIndexes refactor

extend IndexBuildsCoordinatorMongodTest::createCollection() to accept collection UUID
IndexBuildsCoordinator::buildIndex() returns index catalog stats
IndexBuildsCoordinator::buildIndex() accepts collection UUID instead of namespace
IndexBuildsManager::commitIndexBuild() accepts callback for index builds
Branch: master
https://github.com/mongodb/mongo/commit/94ca845476a61c9a97ae119bb4d0bf2a045fe16f

Comment by Githook User [ 13/Jan/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 add Collection::addCollationDefaultsToIndexSpecsForCreate()
Branch: master
https://github.com/mongodb/mongo/commit/1e699ce684d988008ec944f8c82d301f9a5cac2b

Comment by Githook User [ 12/Jan/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: Revert "SERVER-37643 add Collection::addCollationDefaultsToIndexSpecsForCreate()"

This reverts commit 190fef15ba515974c789b5a195a761b518fca46e.
Branch: master
https://github.com/mongodb/mongo/commit/4d223196abebac6df3a671709f07524eead85b67

Comment by Githook User [ 12/Jan/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 add Collection::addCollationDefaultsToIndexSpecsForCreate()
Branch: master
https://github.com/mongodb/mongo/commit/190fef15ba515974c789b5a195a761b518fca46e

Comment by Githook User [ 12/Jan/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 IndexCatalog::removeExistingIndexes() optionally throws on invalid non-duplicate indexes
Branch: master
https://github.com/mongodb/mongo/commit/19f8455248c54159339fbb07b4ffb8da8795e770

Comment by Githook User [ 02/Jan/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 add IndexBuildsCoordinator to dbtest initialization
Branch: master
https://github.com/mongodb/mongo/commit/b12400c3a732190c341d7ab99f44e35742e322d6

Comment by Githook User [ 27/Dec/18 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 remove unused multi_index_block.h includes
Branch: master
https://github.com/mongodb/mongo/commit/ff6163de0fa1d2758c1a182d4e5a10aa7ed50fa3

Comment by Githook User [ 27/Dec/18 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 DatabaseTest does not require MultiIndexBlock for unfinished index build
Branch: master
https://github.com/mongodb/mongo/commit/359b7be9b08d972978207815fb90f161d0144a64

Comment by Githook User [ 21/Dec/18 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 collection rename across databases no longer requires MultiIndexBlock
Branch: master
https://github.com/mongodb/mongo/commit/f7c5ff60981fd30043f42cc67df6ad2d772c6a94

Comment by Githook User [ 18/Dec/18 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 replace MultiIndexBlock reference in test with IndexCatalog::createIndexOnEmptyCollection
Branch: master
https://github.com/mongodb/mongo/commit/20cfa56cb3a8452f71489faee0cba4eebc9d20a7

Comment by Githook User [ 13/Dec/18 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-37643 move MultiIndexBlock::removeExistingIndexes() to IndexCatalog
Branch: master
https://github.com/mongodb/mongo/commit/41878d5036e76f215c02fca47e5e10919087d469

Generated at Thu Feb 08 04:46:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.