[SERVER-2286] slaveOk is ignored for group() and mapReduce(), these are allowed to run on slave always Created: 24/Dec/10 Updated: 12/Jul/16 Resolved: 05/Jan/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 1.7.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Antoine Girbal | Assignee: | Antoine Girbal |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Participants: |
| Description |
|
this is easy to reproduce: foo:SECONDARY> db.test.find() foo:SECONDARY> db.test.group({initial: {n:0}, reduce: function(obj,out){out.n += obj.a;}}) ] |
| Comments |
| Comment by auto [ 06/Jan/11 ] |
|
Author: {u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}Message: |
| Comment by auto [ 05/Jan/11 ] |
|
Author: {u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}Message: |
| Comment by Antoine Girbal [ 05/Jan/11 ] |
|
added test in jstests/replsets/groupAndMapReduce.js |
| Comment by Antoine Girbal [ 05/Jan/11 ] |
|
ok the error raise is now correct: foo:SECONDARY> db.getMongo().slaveOk = true |
| Comment by Eliot Horowitz (Inactive) [ 05/Jan/11 ] |
|
1) yes. if inline results are not wanted, then we should check _isMaster right at the start 2) not sure I understand. Can you send the full example? |
| Comment by Antoine Girbal [ 05/Jan/11 ] |
|
For map/reduce, I had to tweak several calls to exist() and count() within the map reduce, to always have "slaveOk". foo:SECONDARY> db.users.findOne() foo:SECONDARY> reduce = function(key, vals) { var sum = 0; for (var val in vals) { sum += val; } return sum; } foo:SECONDARY> db.users.mapReduce(map, reduce, "outcoll"); foo:SECONDARY> db.getMongo().slaveOk = true , , , , , { "_id" : "zzucdarlws", "value" : 1 } ], , Questions: 2) if I run the mr on master and output to table, the table data looks like: { "_id" : "btkcfrljxh", "value" : "00" } { "_id" : "btmhnalouu", "value" : "00" }whereas inline it is like , { "_id" : "zzucdarlws", "value" : 1 }why is value different? |
| Comment by Antoine Girbal [ 04/Jan/11 ] |
|
fixed the group cmd by switching return value of slaveOk() in group.cpp foo:SECONDARY> db.getMongo().setSlaveOk() foo:SECONDARY> db.test.group({initial: {n:0}, reduce: function(obj,out){out.n += obj.a;}}) ] foo:SECONDARY> db.test.group({initial: {n:0}, reduce: function(obj,out){out.n += obj.a;}}) |
| Comment by MattK [ 03/Jan/11 ] |
|
My goal would be to keep the M/R results off of the master, when performing an ETL aggregation of data into a reporting/analytics store. |
| Comment by Eliot Horowitz (Inactive) [ 03/Jan/11 ] |
|
@matt That can cause very weird behavior, especially with replica sets. one option we can add long term is doing the work on a slave and writing the result to the master. |
| Comment by MattK [ 03/Jan/11 ] |
|
> for map/reduce, doing "out" to a collection should only work on master Would there be benefit to allowing out operations to write into a slave, creating reporting collections from ETL operations that do not affect the master? Could such behavior target a separate separate database, keeping the slave database a replica of the master, but allowing high load ETL operations to be isolated on a secondary that cannot be promoted to a master? |
| Comment by Eliot Horowitz (Inactive) [ 24/Dec/10 ] |
|
When we do this we should add tests to verify
|