[SERVER-10980] MR parallel processing failed Created: 01/Oct/13  Updated: 11/Jul/16  Resolved: 18/Dec/13

Status: Closed
Project: Core Server
Component/s: MapReduce
Affects Version/s: None
Fix Version/s: 2.5.4

Type: Bug Priority: Major - P3
Reporter: Attila Incze Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux precise64 3.2.0-23-generic #36-Ubuntu


Operating System: ALL
Steps To Reproduce:

apt-get install mongodb-10gen
mkdir -p /data/

{configdb,datadb1,datadb2}

mongod --port 27019 --configsvr > configdb.out &
mongod --port 27017 --dbpath /data/datadb1 > datadb1.out &
mongod --port 27016 --dbpath /data/datadb2 > datadb2.out &
mongos --port 27018 --configdb 127.0.0.1:27019 > mongos.out &

mongo --port 27018

sh.addShard('127.0.0.1:27017');
sh.addShard('127.0.0.1:27016');
use test;
db.test.ensureIndex({'geo': 1});
db.test.ensureIndex({ geo: "2d", time: 1 });
db.test.ensureIndex({'time': 1});
sh.enableSharding('test');
sh.shardCollection('test.test', {'_id': 1});
for (var i=0; i<1000000; i++) {
	db.test.insert({time: Math.round(i/1000), geo: [Math.random()*90, Math.random()*90]});
}
db.runCommand({
	mapreduce: 'test',
	map: function() { emit(this.time, {count: 1}) },
	reduce: function(key, values) {
		var count = 0;
     	values.forEach(function(v) {
        	count += v['count'];
     	});
     	return {count: count};
	},
	query: { "geo": { "$within": { "$box": [ [-90,-90] , [90,90]] } } },
	out: 'inline'
});
{
	"ok" : 0,
	"errmsg" : "MR parallel processing failed: { errmsg: \"exception: can't get a DBWrite while having a read lock\", code: 16186, ok: 0.0 }"
}

Participants:

 Description   

When specifying a query for the map-reduce job, it fails. Without query it works flawlessly, unfortunately without the query it runs miserably slow on our dataset.

I am simply aggregating the records based on one of the fields, and filter them with the query based on a 2d index. It fails. Now I'm working with a workaround to filter the records in the map() function, which obviously sucks.

I have brought to you a full step-by-step reproduction guide. It seems there is no such problem on a single instance of mongodb, as well as with a smaller dataset.



 Comments   
Comment by David Storch [ 18/Dec/13 ]

This is fixed in development version 2.5.4. atimb, thanks again for the detailed report and easy repro steps.

Comment by Daniel Pasette (Inactive) [ 07/Oct/13 ]

Thanks for the very clear report and reproduction script. I was able to reproduce this with your script, but shrunk the number of documents to 200,000. I was able to workaround this by using a 2dsphere rather than a 2d index. Can you try that and see if it works? The lock yielding behavior of the two indexes is different.

In the meantime, we'll try to find the root cause.

Generated at Thu Feb 08 03:24:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.