Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-97723

[v6.0-only] Range deleter doesn't correctly synchronise with the critical section

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 6.0.0
    • Component/s: None
    • None
    • Catalog and Routing
    • ALL
    • CAR Team 2024-12-23, CAR Team 2025-01-06
    • 200
    • 2

      If a collection is unsharded, the range deleter will skip the deletion and complete the task (which will cause its deletion from disk). The check is performed here and it relies on the filtering metadata to be up-to-date. 
      The check does not serialize on the critical section and therefore it potentially accesses to stale filtering metadata if a concurrent ddl is running. The check might cause the a range deletion to be skipped for a collection that is actually sharded.

      Example

      • db.collB exists as unsharded
      • A migration for the collection db.collA moves the range [0,10] from shard1 to shard2 and succeed.
      • A range deletion task is scheduled for the range [0,10] on shard1 on db.collA
      • A rename operation starts, renaming db.collA to db.collB with dropTarget=true
      • collA is renamed locally to collB  is dropped.
      • The range deletion tasks is updated to target collB

      The filtering metadata in this example, would be actually cleared and therefore unknown. However, a stepdown + step-up would cause them to be reloaded as stale.

      • A step-down + step-up occurs
      • collB has stale filtering metadata
      • The range deletion task executes and find the collection unsharded (when it’s not) and skips the deletion.

       

      Consequences

      Migrations for a specific range [a,b] are blocked until a range deletion task with an intersecting range for [a,b] is pending (i.e at least 1 document overlaps between the 2 ranges). This forbids the user to move back the data until the range deletion completes.

      In this case, the range deletion task is gone and the user can potentially move back the chunk to the donor.

      Potential consequences are

      1. Best cases (i.e do not result in data inconsistencies)
        1. The documents in the recipient are never deleted. In that case, the _id with the orphans would always correspond. Migrating the chunk back to the donor would fail the cloning phase (with error Location6718402: operation was interrupted), causing the migration to be aborted
        2. The user has set a shard key with unique index. In that case, we have an extra protection for document that are potentially dropped and re-created on the recipient. Migrating the chunk back to the donor would still fail with Location6718402 due to duplicate values for the same shard key field. 
      1. Worse cases (i.e might result in data inconsistencies)
        1. The user has dropped documents on the recipient. In this case, migrating the chunk back to the donor would succeed but cause the deleted document to re-appear.

      Potential fixes

      The solution should either:

      • Synchronize with the critical section
      • Evaluate whether remove that check, which seems unnecessary. 

            Assignee:
            pierlauro.sciarelli@mongodb.com Pierlauro Sciarelli
            Reporter:
            enrico.golfieri@mongodb.com Enrico Golfieri
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: