[SERVER-2581] MR locks server for a long time in merge and reduce modes with many keys Created: 17/Feb/11  Updated: 12/Jul/16  Resolved: 16/Feb/12

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: None
Fix Version/s: 2.1.1

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: Antoine Girbal
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Participants:

 Description   

This is probably necessary to implement the current atomic semantics, but I think we could have an optional non-atomic mode



 Comments   
Comment by Antoine Girbal [ 01/Sep/12 ]

Doc was added:

  • { nonAtomic : 1} - MongoDB 2.1+ If true then the post processing step will not be executed under the write lock. This option makes sense only for the "merge" and "replace" output which take time to process and would lock the database. Using this flag means that partial results will be visible as they are processed.
Comment by Pavel Chertorogov [ 11/Jun/12 ]

Please add to DOCS information about this flag.
http://www.mongodb.org/display/DOCS/MapReduce#MapReduce-Outputoptions

Comment by Antoine Girbal [ 16/Feb/12 ]

@Nathan yes it will be supported with sharding.
There wasnt any bug in fixForShards, because nonAtomic is a subfield of "out".

Comment by auto [ 16/Feb/12 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-2581: added tests for nonAtomic and other MR
Branch: master
https://github.com/mongodb/mongo/commit/00679f7b828368c4a99dd621cc7a58ec4156d0d1

Comment by Nathan Ehresman [ 18/Oct/11 ]

Will this be supported in a sharded environment? In cf953c164861, I didn't see the "nonAtomic" output option added to MRCmd::fixForShards in commands_public.cpp, so I think it may cause an error ("unknown m/r field for sharding") if I'm reading the code right.

Comment by Antoine Girbal [ 13/Sep/11 ]

added a "nonAtomic" flag to output option

Comment by auto [ 13/Sep/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-2581: MR locks server for a long time in merge and reduce modes with many keys
Branch: master
https://github.com/mongodb/mongo/commit/cf953c164861f764eb7b4333cf97e2159a08ab3d

Generated at Thu Feb 08 03:00:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.