[SERVER-12408] Map reduce temporary collection can not be removed or accessed Created: 18/Jan/14  Updated: 11/Jul/16  Resolved: 17/Mar/14

Status: Closed
Project: Core Server
Component/s: MapReduce
Affects Version/s: 2.4.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Teemu Ikonen Assignee: Stennie Steneker (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu@db:~$ uname -a
Linux ip-10-104-157-132 3.2.0-56-virtual #86-Ubuntu SMP Wed Oct 23 09:43:22 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Amazon EC2 with EBS


Operating System: Linux
Steps To Reproduce:

Can not reproduce.

Participants:

 Description   

Ran accidentally map reduce to a busy server and had to kill the client process as some client connections were not responding anymore. After killing all processes the temporary collection '_purge' was not removed and even after cleanly shutting down and restarting mongo it can not be removed.

ubuntu@db:~$ /opt/mongodb/bin/mongo
MongoDB shell version: 2.4.4
connecting to: test
> use mydb
switched to db mydb
> show collections
_purge
adminmessages
...

> db._purge.drop()
Sat Jan 18 06:12:06.024 JavaScript execution failed: TypeError: Cannot call method 'drop' of undefined
> db['_purge'].count()
Sat Jan 18 06:12:26.334 JavaScript execution failed: TypeError: Cannot call method 'count' of undefined
> db.getCollectionNames();
[
"_purge",
"adminmessages",
....
Can see this from mongo log

Sat Jan 18 04:50:54.451 [conn5226] CMD: drop war._purge
Sat Jan 18 04:50:54.566 [conn5226] command war.$cmd command:

{ drop: "_purge" }

ntoreturn:1 keyUpdates:0 reslen:117 140ms
Sat Jan 18 04:50:54.695 [conn5226] command admin.$cmd command:

{ renameCollection: "war.tmp.mr.battleinbox_118", to: "war._purge", stayTemp: false }

ntoreturn:1 keyUpdates:0 reslen:37 120ms
Sat Jan 18 04:50:54.714 [conn5226] CMD: drop war.tmp.mr.battleinbox_118
Sat Jan 18 04:50:54.761 [conn5226] CMD: drop war.tmp.mr.battleinbox_118
Sat Jan 18 04:50:54.761 [conn5226] CMD: drop war.tmp.mr.battleinbox_118_inc
Sat Jan 18 04:50:54.849 [conn5173] update war.inbox query:

{ read: false, uid: 202584 }

update: { $set:

{ read: true }

} nscanned:3 nupdated:2 keyUpdates:0 numYields: 1 locks(micros) w:783 107ms
Sat Jan 18 04:50:54.849 [conn4886] insert war.mapevents ninserted:1 keyUpdates:0 locks(micros) w:435 108ms
Sat Jan 18 04:50:54.903 [conn5145] update war.battleinbox query:

{ read: false, uid: 212933 }

update: { $set:

{ read: true }

} nscanned:39 nupdated:21 keyUpdates:0 numYields: 2 locks(micros) w:101363 186ms
Sat Jan 18 04:50:54.920 [conn5226] CMD: drop war.tmp.mr.battleinbox_118
Sat Jan 18 04:50:54.931 [conn5226] CMD: drop war.tmp.mr.battleinbox_118_inc
Sat Jan 18 04:50:55.131 [conn5136] query war.users query:

{ gcid: "G:625654150", account_type: 6 }

ntoreturn:1 ntoskip:0 nscanned:112460 keyUpdates:0 numYields: 251 locks(micros) r:1113617 nreturned:0 reslen:20 1933ms
Sat Jan 18 04:50:55.135 [conn5226] command war.$cmd command: { mapreduce: "battleinbox", map:
function()

{ emit(this.uid, 1); }

, reduce:
function(key, values)

{ return Array.sum(values); }

, out: "_purge" } ntoreturn:1 keyUpdates:0 numYields: 69182 locks(micros) W:302844 r:1682304504 w:28452182 reslen:130 1952453ms

...
Sat Jan 18 04:53:18.628 [conn5226] getmore war._purge cursorid:3953529293811394301 ntoreturn:0 k
eyUpdates:0 numYields: 23 locks(micros) r:119699 nreturned:56952 reslen:1879436 134ms



 Comments   
Comment by Teemu Ikonen [ 18/Mar/14 ]

Hi, the .stats() command works fine when called like in your example. I'll try to drop the collection when we have next maintenance break. Thanks!

> db.getCollection('_purge').stats()
{
"ns" : "mydb._purge",
"count" : 57053,
"size" : 2053916,
"avgObjSize" : 36.00014022049673,
"storageSize" : 3825664,
"numExtents" : 5,
"nindexes" : 1,
"lastExtentSize" : 2883584,
"paddingFactor" : 1,
"systemFlags" : 1,
"userFlags" : 0,
"totalIndexSize" : 1594320,
"indexSizes" :

{ "_id_" : 1594320 }

,
"ok" : 1
}

Comment by Stennie Steneker (Inactive) [ 17/Mar/14 ]

Hi Teemu,

Thanks for the update. I'm going to close this issue, as the suggested syntax should work with the given collection name (I tested on MongoDB 2.4.9).

If you want to test with a non-destructive command you could use:

db.getCollection('_purge').stats()

Please feel free to reopen if you have any further questions on this issue.

Thanks,
Stephen

Comment by Teemu Ikonen [ 17/Mar/14 ]

I have not touched it since. Database is in production and this does not seem to cause any problems so I rather not try to do anything about it.

Comment by Stennie Steneker (Inactive) [ 17/Mar/14 ]

Hi Teemu,

Were you able to drop the _purge collection?

FYI, the following syntax should work from the mongo shell:

db.getCollection('_purge').drop()

Regards,
Stephen

Comment by Teemu Ikonen [ 22/Jan/14 ]

Yes I have, but it's 14MB compressed and it might contain user sensitive data. I could put them on one our sites and give privately url to you for downloading,

What happened exactly was that cleanup tool did first map reduce, and then started removing individual documents in batches by _id using the results from map reduce.

Comment by Daniel Pasette (Inactive) [ 22/Jan/14 ]

Do you have the mongod log from the period when the map/reduce was killed?

Generated at Thu Feb 08 03:28:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.