[SERVER-47182] Single-phase index builds should run synchronously on secondaries Created: 30/Mar/20  Updated: 29/Oct/23  Resolved: 05/Jun/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Improvement Priority: Major - P3
Reporter: Louis Williams Assignee: Benety Goh
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-47867 Remove ghost timestamping code Closed
Related
related to SERVER-48465 restore noop write msg format for sin... Closed
related to SERVER-38961 Remove override for causally consiste... Closed
is related to SERVER-46601 createIndexes on local database colle... Closed
is related to SERVER-39079 Add parallel IndexBuildsCoordinator c... Closed
is related to SERVER-41554 Remove applyOps support for createInd... Closed
is related to SERVER-47439 IndexBuildsCoordinator::createIndexes... Closed
is related to SERVER-48593 createIndexForApplyOps() should alway... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-06-15
Participants:
Linked BF Score: 10

 Description   

We can't remove single-phase index builds entirely, but we should consider forcing single-phase builds to run synchronously with oplog batch application. This enables them to stop using ghost timestamps. Single phase builds are only used for empty collections, and to rebuild indexes admin.system collections (see SERVER-47439).
Original description:

Use two-phase index builds by default. Evaluate and remove logic, where possible, that depends on IndexBuildProtocol::kSinglePhase, with the exception of creating indexes on empty collections.



 Comments   
Comment by Githook User [ 05/Jun/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47182 remove IndexBuildsCoordinator::IndexBuildOptions::replSetAndNotPrimaryAtStart

This option is only applicable to single-phase index builds on secondaries, which no longer
go through the IndexBuildsCoordinator.
Branch: master
https://github.com/mongodb/mongo/commit/4042389f588d64e9537253d7879450f3db5efcb1

Comment by Githook User [ 04/Jun/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47182 single-phase index builds should run synchronously on secondaries
Branch: master
https://github.com/mongodb/mongo/commit/f88b07048ed2641a673ac703bd2db9a9aeff7f90

Comment by Benety Goh [ 04/Jun/20 ]

Author: {'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47182 IndexBuildsCoordinator::createIndex() ignores certain indexing errors during oplog application

Branch: master
https://github.com/mongodb/mongo/commit/41b01cdece6f5db6f1a3e85f751b648e5bc0e2da

Comment by Githook User [ 04/Jun/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47182 IndexBuildsCoordinator::createIndexes() accepts single index spec
Branch: master
https://github.com/mongodb/mongo/commit/1722c72dde1ae8fa6559af6bc431d075324bd91b

Comment by Githook User [ 04/Jun/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47182 include ops in OplogApplierImplTest::runOpsInitialSync() error
Branch: master
https://github.com/mongodb/mongo/commit/bb3552868ac0c0698d83cb905db5f9667c26f10f

Comment by Benety Goh [ 04/Jun/20 ]

The createIndexes command is no longer supported as an {{applyOps} } operation. It should be fine to assume that constraints should always be relaxed in createIndexForApply().

Comment by Benety Goh [ 03/Jun/20 ]

The test IdempotencyTest::IndexWithDifferentOptions expects certain behavior from applying a createIndexes oplog entry that should be preserved.

Comment by Louis Williams [ 03/Jun/20 ]

Index builds on system indexes only happen once and on empty collections. There is a case where a system index would need to be rebuilt from a collection scan, but only if a user manually deletes it, so I think we should keep system index builds as single-phased.

Comment by Benety Goh [ 03/Jun/20 ]

SERVER-46601 ensures that index builds on standalone are always single-phased.

Comment by Benety Goh [ 03/Jun/20 ]

Index builds on some system indexes will probably remain single-phased. See SERVER-47439.

Comment by Louis Williams [ 06/May/20 ]

Single-phase index builds aren't going to be removed completely, but I think we can straighten out some logic and make assertions about the following:

  • Always used on standalones
  • In replica-sets, only used on empty collections
  • Use "createIndexes" oplog entry
  • On secondaries, are completed synchronously with oplog batch application (because they are only used on empty collections)
Generated at Thu Feb 08 05:13:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.