[SERVER-2286] slaveOk is ignored for group() and mapReduce(), these are allowed to run on slave always Created: 24/Dec/10  Updated: 12/Jul/16  Resolved: 05/Jan/11

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 1.7.5

Type: Bug Priority: Major - P3
Reporter: Antoine Girbal Assignee: Antoine Girbal
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

this is easy to reproduce:

foo:SECONDARY> db.test.find()
error:

{ "$err" : "not master and slaveok=false", "code" : 13435 }

foo:SECONDARY> db.test.group({initial:

{n:0}

, reduce: function(obj,out){out.n += obj.a;}})
[

{ "n" : 3 }

]
foo:SECONDARY>



 Comments   
Comment by auto [ 06/Jan/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-2286: change to allow operations on slave if not replSet
https://github.com/mongodb/mongo/commit/80fecd37e37aea6f55dae25b4a63174061fea598

Comment by auto [ 05/Jan/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-2286: slaveOk is ignored for group() and mapReduce(), these are allowed to run on slave always
https://github.com/mongodb/mongo/commit/f5e2323933025844cd76e9f31025f8dc1747e55d

Comment by Antoine Girbal [ 05/Jan/11 ]

added test in jstests/replsets/groupAndMapReduce.js

Comment by Antoine Girbal [ 05/Jan/11 ]

ok the error raise is now correct:

foo:SECONDARY> db.getMongo().slaveOk = true
true
foo:SECONDARY> db.users.mapReduce(map, reduce, "outcoll");
Tue Jan 4 22:49:51 uncaught exception: map reduce failed:

{ "errmsg" : "not master", "ok" : 0 }
Comment by Eliot Horowitz (Inactive) [ 05/Jan/11 ]

1) yes. if inline results are not wanted, then we should check _isMaster right at the start

2) not sure I understand. Can you send the full example?

Comment by Antoine Girbal [ 05/Jan/11 ]

For map/reduce, I had to tweak several calls to exist() and count() within the map reduce, to always have "slaveOk".

foo:SECONDARY> db.users.findOne()
{
"_id" : ObjectId("4d1009c7262bb4b94af1cea4"),
"fn" : "usroatjnmo",
"ln" : "lparjzmorx"
}
foo:SECONDARY> map = function()

{ emit(this.ln, 1); }

foo:SECONDARY> reduce = function(key, vals) { var sum = 0; for (var val in vals)

{ sum += val; }

return sum; }
foo:SECONDARY> db.getMongo().slaveOk = false
false
foo:SECONDARY> db.users.mapReduce(map, reduce, {out: { "inline" : 1}});
Tue Jan 4 16:43:40 uncaught exception: map reduce failed:

{ "errmsg" : "not master", "ok" : 0 }

foo:SECONDARY> db.users.mapReduce(map, reduce, "outcoll");
Tue Jan 4 16:43:48 uncaught exception: map reduce failed:

{ "errmsg" : "not master", "ok" : 0 }

foo:SECONDARY> db.getMongo().slaveOk = true
true
foo:SECONDARY> db.users.mapReduce(map, reduce, "outcoll");
Tue Jan 4 16:44:03 uncaught exception: map reduce failed: {
"assertion" : "replSet error : logOp() but not primary?",
"assertionCode" : 13312,
"errmsg" : "db assertion failure",
"ok" : 0
}
foo:SECONDARY> db.users.mapReduce(map, reduce, {out: { "inline" : 1}});
{
"results" : [
...

{ "_id" : "zxjhcniatu", "value" : 1 }

,

{ "_id" : "zxjjjuxgeb", "value" : 1 }

,

{ "_id" : "zxxwkjajsg", "value" : 1 }

,

{ "_id" : "zyamwcrtrr", "value" : 1 }

,

{ "_id" : "zzobixwpuv", "value" : 1 }

,

{ "_id" : "zzucdarlws", "value" : 1 }

],
"timeMillis" : 129,
"counts" :

{ "input" : 4000, "emit" : 4000, "output" : 4000 }

,
"ok" : 1,
}

Questions:
1) when slaveOk=false and trying to write to collection, is the current exception "logOp() but not primary". Most likely exception needs to happen before that right?
It seems mongod creates a new NS for tmp collection, then fails when logging it, which is too late.

2) if I run the mr on master and output to table, the table data looks like:

{ "_id" : "btkcfrljxh", "value" : "00" } { "_id" : "btmhnalouu", "value" : "00" }

whereas inline it is like

{ "_id" : "zzobixwpuv", "value" : 1 }

,

{ "_id" : "zzucdarlws", "value" : 1 }

why is value different?

Comment by Antoine Girbal [ 04/Jan/11 ]

fixed the group cmd by switching return value of slaveOk() in group.cpp

foo:SECONDARY> db.getMongo().setSlaveOk()
foo:SECONDARY> db.test.find()

{ "_id" : ObjectId("4d239463ce489e240201f9dc"), "a" : 1 } { "_id" : ObjectId("4d23946ece489e240201f9dd"), "a" : 2 }

foo:SECONDARY> db.test.group({initial:

{n:0}

, reduce: function(obj,out){out.n += obj.a;}})
[

{ "n" : 3 }

]
foo:SECONDARY> db.getMongo().slaveOk = false
false
foo:SECONDARY> db.test.find()
error:

{ "$err" : "not master and slaveok=false", "code" : 13435 }

foo:SECONDARY> db.test.group({initial:

{n:0}

, reduce: function(obj,out){out.n += obj.a;}})
Tue Jan 4 15:28:02 uncaught exception: group command failed:

{ "errmsg" : "not master", "ok" : 0 }
Comment by MattK [ 03/Jan/11 ]

My goal would be to keep the M/R results off of the master, when performing an ETL aggregation of data into a reporting/analytics store.

Comment by Eliot Horowitz (Inactive) [ 03/Jan/11 ]

@matt That can cause very weird behavior, especially with replica sets. one option we can add long term is doing the work on a slave and writing the result to the master.

Comment by MattK [ 03/Jan/11 ]

> for map/reduce, doing "out" to a collection should only work on master

Would there be benefit to allowing out operations to write into a slave, creating reporting collections from ETL operations that do not affect the master?

Could such behavior target a separate separate database, keeping the slave database a replica of the master, but allowing high load ETL operations to be isolated on a secondary that cannot be promoted to a master?

Comment by Eliot Horowitz (Inactive) [ 24/Dec/10 ]

When we do this we should add tests to verify

  • they are not allowed without slave_ok
  • they are allowed with slave_ok
  • for map/reduce, doing "out" to a collection should only work on master
  • out "inline" should work on master or secondary
Generated at Thu Feb 08 02:59:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.