Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-49249

initial sync fails invariant during oplog application due to conflict with in-progress index build

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.9.0
    • Affects Version/s: None
    • Component/s: Storage
    • None
    • Fully Compatible
    • ALL
    • Execution Team 2020-11-30
    • 10

      Initial sync may fail this invariant in MultiIndexBlock::init() due to a conflict between single and two phase index builds using the same index name during the oplog application phase. This issue is believed to have been introduced by SERVER-47182 which affects 4.5 only.

      In SERVER-47182 (post-4.4), we started using an alternate code path to apply createIndexes on on secondaries (which also covers nodes running initial sync).

      This race may happen when we have built an index with the same name using both single-phase and two-phase methods on the  collection that is being initial sync'ed. The following order of operations on the collection may lead to the invariant failure:

      • secondary: initial sync starts
      • primary: create (empty) collection
      • primary: create index X (single-phased)
      • primary: insert documents
      • primary: drop index X
      • primary: start building index X (two-phased)
      • secondary: clones collection with index X still building
      • primary: complete building index X (two-phased)
      • secondary: completes collection cloning and starts applying oplog entries to catch up
      • secondary: applies first index X oplog entry (createIndex) and hits invariant because the two-phase index build for X is still not committed.

      This is not an issue in 4.4 because we would have filtered out existing indexes (including in-progress index builds) and ignored the createIndexes entry.

      The new code path introduced in SERVER-47182 does not filter out in-progress index builds, leading to the invariant observed in this ticket.

            Assignee:
            benety.goh@mongodb.com Benety Goh
            Reporter:
            benety.goh@mongodb.com Benety Goh
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: