Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12949

MapReduce not doing incremental reduces when needed

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 2.6.0-rc2
    • 2.6.0-rc0
    • MapReduce
    • ALL
    • Hide

      Original Title: mapreduce stats have changed between 2.4 and 2.6

      Start mongo.exe and execute the following script (succeeds against 2.4 server and fails against 2.6 server):

      var db = db.getSisterDB("TestDB");
      db.dropDatabase();
       
      var coll = db.getCollection( "foo.mrInput" );
      coll.getDB();
       
      for (i=1; i<50; i++) {
          for (j=0; j<10; j++) {
              coll.insert({i: i, j:j});
          }
      }
       
      function mapFn() { emit(this.j, 1); }
      function reduceFn(key, values) { return Array.sum(values); } 
       
      var out = coll.mapReduce(mapFn, reduceFn, { out: { replace: "mrOutput", sharded: true } });
      assert.eq(out.counts.reduce, 30, "reduce count is wrong");

      Show
      Original Title: mapreduce stats have changed between 2.4 and 2.6 Start mongo.exe and execute the following script (succeeds against 2.4 server and fails against 2.6 server): var db = db.getSisterDB("TestDB"); db.dropDatabase(); var coll = db.getCollection( "foo.mrInput" ); coll.getDB(); for (i=1; i<50; i++) { for (j=0; j<10; j++) { coll.insert({i: i, j:j}); } } function mapFn() { emit(this.j, 1); } function reduceFn(key, values) { return Array.sum(values); }   var out = coll.mapReduce(mapFn, reduceFn, { out: { replace: "mrOutput", sharded: true } }); assert.eq(out.counts.reduce, 30, "reduce count is wrong");

    Description

      The mapreduce reduce count statistic has changed between 2.4 and 2.6. It is possible that it was incorrect before and has been fixed in 2.6, but this has to be verified.

      To reproduce, see the script in the repro steps section.

      In 2.4, the reduce count is 30 and in 2.6 it is 10. The value in 2.6 seems more reasonable, given that the number of distinct ids emitted is 10, but this has to be confirmed.

      Attachments

        Activity

          People

            kaloian.manassiev@mongodb.com Kaloian Manassiev
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: