Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38471

Concurrent mapReduce to same output sharded collection can trigger invariant

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.1.6
    • Component/s: MapReduce, Sharding
    • Labels:
      None
    • Sharding
    • ALL
    • 53

      Normal mapReduce finish flow:

      1. shardCollection is run on outputColl. outputColl is created in the shard with UUIDA. UUIDFinal is stored in config.collections for outputColl.
      2. mapReduce finish is sent to shard with UUIDFinal. Shard creates tempColl with UUIDFinal and outputs results to tempColl.
      3. outputColl (created at step#1) is dropped.
      4. tempColl is renamed to outputColl.

      Case that will trigger invariant:
      1. mr1 runs shardColl on outputColl. outputColl is created in the shard with UUIDA. UUIDFinal is stored in config.collections for outputColl.
      2. mr2 runs shardColl on outputColl. outputColl is already sharded with UUIDFinal.
      3. mr1 sends mapReduce finish to shard. tempColl1 gets created with UUIDFinal and becomes outputColl.
      4. mr2 sends mapReduce finish to shard. tempColl2 gets created with UUIDFinal (which has the same UUID as outputColl because of step#3!).
      5. mr2 drops outputColl which removes UUIDFinal from UUIDCatalog.
      6. mr2 tries to rename tempColl2 to outputColl but can't find UUIDFinal from the catalog and triggers invariant.

            Assignee:
            backlog-server-sharding [DO NOT USE] Backlog - Sharding Team
            Reporter:
            randolph@mongodb.com Randolph Tan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: