Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-61427

Unique index builds can cause a loss of availability during commit due to checking many false duplicates

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 4.2.0, 4.4.0, 5.0.0
    • 5.2.0, 5.1.2, 5.0.6, 4.4.11, 4.2.18
    • None
    • None
    • Fully Compatible
    • v5.1, v5.0, v4.4, v4.2
    • Execution Team 2021-11-29

    Description

      While an index build is in progress, we take any concurrent writes and insert them into the side writes table. Once the collection scan and bulk load, we drain the writes from the side writes table into the index. However, it is possible for these concurrent writes to also be seen by the collection scan. In this case, we end up inserting the same index key twice: once during collection scan/bulk load, and once while draining side writes. This is not a correctness issue; for non-unique indexes we simply ignore that the key already exists, and for unique indexes we write down the key as a potential duplicate. For the unique case, we then check all of these false "duplicates" under a MODE_X collection lock before committing the index. This can lead to a loss of availability if the index being built is on a large collection with many writes during the collection scan.

      A couple potential solutions:

      • Initially check duplicates under a MODE_IX lock and then later use a MODE_X lock, similarly to what we do for draining the side writes table
      • When draining the side writes table for a unique index, if the entire key (including the record id, as opposed to only the key prefix) already exists in the table, do not count as a potential duplicate

      Attachments

        Issue Links

          Activity

            People

              gregory.noma@mongodb.com Gregory Noma
              gregory.noma@mongodb.com Gregory Noma
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: