[SERVER-30111] Mapreduce uses global lock when outputting to empty collection Created: 12/Jul/17  Updated: 14/Jul/17  Resolved: 14/Jul/17

Status: Closed
Project: Core Server
Component/s: MapReduce
Affects Version/s: 3.2.15, 3.4.6, 3.5.9
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: James Mitchell Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-13552 remove unnecessary global lock during... Backlog
Operating System: ALL
Steps To Reproduce:

Create a mapreduce and output to an empty collection, and it will get the global lock.

Participants:

 Description   

I want to efficiently mapreduce some data and output to an empty collection without touching the global lock.

There is an optimization in mr.cpp in postProcessCollectionNonAtomic which forces "replace" mode on outputting to empty collections.
Unfortunately, "replace" mode gets a global lock, even when writing to the same database.

Both of these are unexpected.

The workaround is to insert a dummy document into the empty collection, and output using "merge" mode. This is unintuitive and requires digging into the mongo core code.

There is a blog post describing the issue and workaround here:
http://venublog.com/2014/04/05/mongodb-map-reduce-how-to-avoid-global-locks

The blog post is a few years old but this is still an issue in the master branch:
https://github.com/mongodb/mongo/blob/master/src/mongo/db/commands/mr.cpp#L643



 Comments   
Comment by Ramon Fernandez Marina [ 14/Jul/17 ]

Thanks for your report james@rabb.it. This enhancement was previously requested in SERVER-13552, so I'm going to mark this ticket as a duplicate. Please watch SERVER-13552 for updates.

Regards,
Ramón.

Generated at Thu Feb 08 04:22:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.