Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38471

Concurrent mapReduce to same output sharded collection can trigger invariant

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Won't Fix
    • Affects Version/s: 4.1.6
    • Fix Version/s: None
    • Component/s: MapReduce, Sharding
    • Labels:
      None
    • Operating System:
      ALL
    • Linked BF Score:
      53

      Description

      Normal mapReduce finish flow:

      1. shardCollection is run on outputColl. outputColl is created in the shard with UUIDA. UUIDFinal is stored in config.collections for outputColl.
      2. mapReduce finish is sent to shard with UUIDFinal. Shard creates tempColl with UUIDFinal and outputs results to tempColl.
      3. outputColl (created at step#1) is dropped.
      4. tempColl is renamed to outputColl.

      Case that will trigger invariant:
      1. mr1 runs shardColl on outputColl. outputColl is created in the shard with UUIDA. UUIDFinal is stored in config.collections for outputColl.
      2. mr2 runs shardColl on outputColl. outputColl is already sharded with UUIDFinal.
      3. mr1 sends mapReduce finish to shard. tempColl1 gets created with UUIDFinal and becomes outputColl.
      4. mr2 sends mapReduce finish to shard. tempColl2 gets created with UUIDFinal (which has the same UUID as outputColl because of step#3!).
      5. mr2 drops outputColl which removes UUIDFinal from UUIDCatalog.
      6. mr2 tries to rename tempColl2 to outputColl but can't find UUIDFinal from the catalog and triggers invariant.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-sharding Backlog - Sharding Team
              Reporter:
              renctan Randolph Tan
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: