[SERVER-5536] 2.0/2.1 sharded mapReduce compatibility Created: 06/Apr/12  Updated: 11/Jul/16  Resolved: 12/Jun/12

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 2.1.2

Type: Bug Priority: Major - P3
Reporter: Greg Studer Assignee: Antoine Girbal
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File features2_output.txt    
Operating System: ALL
Participants:

 Description   

Seems like there's an incompatibility between 2.0 mongod/2.1 mongos on sharded mapreduce.

To repro, replace 2.1 mongod with 2.0.4 mongod and run features2.js, results in :

ri Apr  6 15:59:00 exec error: src/mongo/shell/collection.js:661 map reduce failed:{
	"ok" : 0,
	"errmsg" : "MR post processing failed: { assertion: \"invalid parameter: expected an object ()\", assertionCode: 10065, errmsg: \"db assertion failure\", ok: 0.0 }"
}
throw "map reduce failed:" + tojson(raw);

Attached sample output (there was some additional spurious logging in the mongos in this output, but reproducible in multiple other tests).



 Comments   
Comment by auto [ 12/Jun/12 ]

Author:

{u'login': u'erh', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-5536 - fix case when input and output db are different
Branch: master
https://github.com/mongodb/mongo/commit/32f3e9a4d2f627c56094e6cc1dc3b5ab8cda4171

Comment by Eliot Horowitz (Inactive) [ 12/Jun/12 ]

I have a fix for this. will commit in a minute.

Comment by Eliot Horowitz (Inactive) [ 12/Jun/12 ]

tests failing
http://buildbot.mongodb.org/builders/Linux%2032-bit%20debug/builds/1862/steps/test/logs/stdio

Comment by auto [ 12/Jun/12 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-5536: remove use of inputNS field
Branch: master
https://github.com/mongodb/mongo/commit/fe9822e8d65948060f29fcc86349144877f376b2

Comment by auto [ 12/Jun/12 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-5536: 2.0/2.1 sharded mapReduce compatibility
Branch: master
https://github.com/mongodb/mongo/commit/b5661e1cc5cc2278aa9bd13de7e031ce02fe6d83

Comment by Antoine Girbal [ 07/Jun/12 ]

In addition to "shards", the "shardedOutputCollection" field needs to be passed for compatibility.

Comment by Antoine Girbal [ 07/Jun/12 ]

Problem is that mongod 2.0 expects a "shards" sub-object in the command, which infers the shards to pull from.
This info is redundant with other parts of the command.. But we have to add it back to ensure compatibility.

                    {
                        // parse per shard results
                        BSONObjIterator i( shards );
                        while ( i.more() ) {
                            BSONElement e = i.next();
                            string shard = e.fieldName();
 
                            BSONObj res = e.embeddedObjectUserCheck();
 
                            uassert( 10078 ,  "something bad happened" , shardedOutputCollection == res["result"].valuestrsafe() );
                            servers.insert( shard );
 
                        }
 
                    }

Generated at Thu Feb 08 03:09:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.