Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30111

Mapreduce uses global lock when outputting to empty collection

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor - P4
    • Resolution: Duplicate
    • Affects Version/s: 3.2.15, 3.4.6, 3.5.9
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
      None
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      Create a mapreduce and output to an empty collection, and it will get the global lock.

      Show
      Create a mapreduce and output to an empty collection, and it will get the global lock.

      Description

      I want to efficiently mapreduce some data and output to an empty collection without touching the global lock.

      There is an optimization in mr.cpp in postProcessCollectionNonAtomic which forces "replace" mode on outputting to empty collections.
      Unfortunately, "replace" mode gets a global lock, even when writing to the same database.

      Both of these are unexpected.

      The workaround is to insert a dummy document into the empty collection, and output using "merge" mode. This is unintuitive and requires digging into the mongo core code.

      There is a blog post describing the issue and workaround here:
      http://venublog.com/2014/04/05/mongodb-map-reduce-how-to-avoid-global-locks

      The blog post is a few years old but this is still an issue in the master branch:
      https://github.com/mongodb/mongo/blob/master/src/mongo/db/commands/mr.cpp#L643

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              james@rabb.it James Mitchell
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: