[SERVER-2409] Error attempting to use "reduce" MapReduce Output Mode Created: 25/Jan/11  Updated: 12/Jul/16  Resolved: 26/Jan/11

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 1.7.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Brendan W. McAdams Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

1.7.4, 64 bit Mac OS X with Durability enabled


Attachments: File reproduceMRReduceOutIssue.js     File yield_historical_in.json    
Operating System: ALL
Participants:

 Description   

I'm running a series of tests (See casbah-core/src/test/scala/MapReduceSpec.scala in Casbah for some of the test routines I'm running) and unable to successfully run a MapReduce with a 'reduce' output.

'merge' works fine.

I'm attempting to create a "final" map_reduce collection by having multiple MapReduce jobs on different identical datasets run with 'reduce' output.

The collections I'm reading from are 'yield_historical.out.nineties' and 'yield_historical.out.aughts'. They contain data like such:

{ "_id" : 100, "value" :

{ "count" : 231, "sum" : 1380.3799999999985 }

}
{ "_id" : 101, "value" :

{ "count" : 248, "sum" : 1245.1300000000003 }

}
{ "_id" : 102, "value" :

{ "count" : 250, "sum" : 1153.27 }

}
{ "_id" : 103, "value" :

{ "count" : 250, "sum" : 1003.4699999999997 }

}
{ "_id" : 104, "value" :

{ "count" : 250, "sum" : 1067.8300000000008 }

}
{ "_id" : 105, "value" :

{ "count" : 250, "sum" : 1072.2200000000003 }

}
{ "_id" : 106, "value" :

{ "count" : 250, "sum" : 1198.7499999999989 }

}
{ "_id" : 107, "value" :

{ "count" : 251, "sum" : 1163.3000000000002 }

}
{ "_id" : 108, "value" :

{ "count" : 251, "sum" : 919.73 }

}
{ "_id" : 109, "value" :

{ "count" : 250, "sum" : 816.0300000000009 }

}
{ "_id" : 110, "value" :

{ "count" : 189, "sum" : 628.5199999999996 }

}

These collections were produced by running a normal mapReduce job with the same Map and Reduce function.

When I attempt to run the job however, I get the error:

{ "assertion" : "Not an index cursor" , "assertionCode" : 9011 , "errmsg" : "db assertion failure" , "ok" : 0.0}

The command being run is:

{ "mapreduce" : "yield_historical.out.aughts" ,
"map" : "function m() { emit( typeof(this._id) == \"number\" ? this._id : this._id.getYear(),

{ count: 1, sum: this.bc10Year }

) }" ,
"reduce" : " function r( year, values ) {
var n =

{ count: 0, sum: 0 }

for ( var i = 0; i < values.length; i++ )

{ n.sum += values[i].sum; n.count += values[i].count; }

return n;
}",
"verbose" : true , "out" :

{ "reduce" : "yield_historical.out.all"}

}



 Comments   
Comment by Brendan W. McAdams [ 25/Jan/11 ]

Works against MASTER, no error reported.

Comment by Eliot Horowitz (Inactive) [ 25/Jan/11 ]

Can you try with HEAD

Comment by Brendan W. McAdams [ 25/Jan/11 ]

Attached a test JS file and the source data I was using.

It looks like the issue may be trying to reduce into a collection which doesn't already exist.

Comment by Brendan W. McAdams [ 25/Jan/11 ]

These JavaScript files can be used to reproduce the issue.

Import yield_historical_in.json into a collection named 'yield_historical.in'

And then run the attached .js file against the mongo shell.

Merging into a non-existent collection works.

Reducing into an existing collection works.

Reducing into a NON existing collection fails.

Comment by Eliot Horowitz (Inactive) [ 25/Jan/11 ]

you should make a .js file that reproduces this

Generated at Thu Feb 08 02:59:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.