Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12949

MapReduce not doing incremental reduces when needed

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.6.0-rc2
    • Affects Version/s: 2.6.0-rc0
    • Component/s: MapReduce
    • ALL
    • Hide

      Original Title: mapreduce stats have changed between 2.4 and 2.6

      Start mongo.exe and execute the following script (succeeds against 2.4 server and fails against 2.6 server):

      var db = db.getSisterDB("TestDB");
db.dropDatabase();
 
var coll = db.getCollection( "foo.mrInput" );
coll.getDB();
 
for (i=1; i<50; i++) {
    for (j=0; j<10; j++) {
        coll.insert({i: i, j:j});
    }
}
 
function mapFn() { emit(this.j, 1); }
function reduceFn(key, values) { return Array.sum(values); } 

var out = coll.mapReduce(mapFn, reduceFn, { out: { replace: "mrOutput", sharded: true } });
assert.eq(out.counts.reduce, 30, "reduce count is wrong");
      Show
      Original Title: mapreduce stats have changed between 2.4 and 2.6 Start mongo.exe and execute the following script (succeeds against 2.4 server and fails against 2.6 server): var db = db.getSisterDB( "TestDB" ); db.dropDatabase(); var coll = db.getCollection( "foo.mrInput" ); coll.getDB(); for (i=1; i<50; i++) { for (j=0; j<10; j++) { coll.insert({i: i, j:j}); } } function mapFn() { emit( this .j, 1); } function reduceFn(key, values) { return Array.sum(values); } var out = coll.mapReduce(mapFn, reduceFn, { out: { replace: "mrOutput" , sharded: true } }); assert .eq(out.counts.reduce, 30, "reduce count is wrong" );
    • None
    • 0
    • None
    • None
    • None
    • None
    • None
    • None

      The mapreduce reduce count statistic has changed between 2.4 and 2.6. It is possible that it was incorrect before and has been fixed in 2.6, but this has to be verified.

      To reproduce, see the script in the repro steps section.

      In 2.4, the reduce count is 30 and in 2.6 it is 10. The value in 2.6 seems more reasonable, given that the number of distinct ids emitted is 10, but this has to be confirmed.

        1. Diff_2.4.9_2.6.0rc1.txt
          4 kB

            Assignee:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Reporter:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: