Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30111

Mapreduce uses global lock when outputting to empty collection

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Minor - P4
    • Resolution: Duplicate
    • 3.2.15, 3.4.6, 3.5.9
    • None
    • MapReduce
    • None
    • ALL
    • Hide

      Create a mapreduce and output to an empty collection, and it will get the global lock.

      Show
      Create a mapreduce and output to an empty collection, and it will get the global lock.

    Description

      I want to efficiently mapreduce some data and output to an empty collection without touching the global lock.

      There is an optimization in mr.cpp in postProcessCollectionNonAtomic which forces "replace" mode on outputting to empty collections.
      Unfortunately, "replace" mode gets a global lock, even when writing to the same database.

      Both of these are unexpected.

      The workaround is to insert a dummy document into the empty collection, and output using "merge" mode. This is unintuitive and requires digging into the mongo core code.

      There is a blog post describing the issue and workaround here:
      http://venublog.com/2014/04/05/mongodb-map-reduce-how-to-avoid-global-locks

      The blog post is a few years old but this is still an issue in the master branch:
      https://github.com/mongodb/mongo/blob/master/src/mongo/db/commands/mr.cpp#L643

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              james@rabb.it James Mitchell
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: