[SERVER-3108] namespace too long for temporary collections in Map Reduce Created: 16/May/11  Updated: 12/Jul/16  Resolved: 27/Jun/11

Status: Closed
Project: Core Server
Component/s: MapReduce
Affects Version/s: None
Fix Version/s: 1.9.1

Type: Bug Priority: Major - P3
Reporter: Antoine Girbal Assignee: Antoine Girbal
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Operating System: ALL
Participants:

 Description   

http://groups.google.com/group/mongodb-user/browse_thread/thread/09c50faebe52831c

When I run mapreduce with output type merge on a collection, it
creates temporary collection named as a concatenation of the source
collection and result collection i.e. heres a line in teh verbose
server output showing the temp collection being removed

Wed May 11 20:31:23 [conn7] CMD: drop Events.tmp.mr.battlefield-
assault-pc.login_battlefield-assault-pc.login.region_0

(my source collection name is: battlefield-assault-pc.login
the output collection name: battlefield-assault-pc.login.region)

Would it be possible to either increase the max namespace length, or
to modify the temporary named tables to something less verbose? (since
I read we only have max ~128 chars in a namespace to work with, this
would mean we really only have less than half of that if we intend to
use mapreduce)
Also the postfix "_0" is an ever increasing index showing the version
of the merge. Since I run this mapreduce every 3 minutes it will
eventually consume more and more characters as well.. may want to
represent this as hex value instead?

I've renamed my collections to the above so with the concated temp
collections only using about 80 chars, however the problem now is when
deleting the temporary collection it asserts when trying to drop one
of its longer indices (gives the following error):

Wed May 11 20:31:23 [conn7] dropCollection: Events.tmp.mr.battlefield-
assault-pc.login_battlefield-assault-pc.login.region_0
Wed May 11 20:31:23 [conn7] d->nIndexes was 3
Assertion: 10348:$extra: ns name too long

The index giving me trouble is:
{
"name" : "id.time_1id.region_1_id.countryCode_1",
"ns" : "Events.battlefield-assault-pc.login.region",
"key" :

{ "_id.time" : 1, "_id.region" : 1, "_id.countryCode" : 1 }

,
"v" : 0
}

I'm assuming the drop() is failing cause the concatenation of the ns
and index name is greater than the ~128 character ns max length?

here's my build info:
Wed May 11 20:38:58 [initandlisten] MongoDB starting : pid=28845
port=27017 dbpath=/data/db/ 64-bit
Wed May 11 20:38:58 [initandlisten] db version v1.8.1, pdfile version
4.5
Wed May 11 20:38:58 [initandlisten] git version:
a429cd4f535b2499cc4130b06ff7c26f41c00f04
Wed May 11 20:38:58 [initandlisten] build sys info: Linux bs-
linux64.10gen.cc 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28
EST 2009 x86_64 BOOST_LIB_VERSION=1_41



 Comments   
Comment by Antoine Girbal [ 27/Jun/11 ]

removed the target collection part in the temp collection name.
It is now "tmp.mr." + srcCollection + "_" + increment

Comment by auto [ 27/Jun/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: SERVER-3108: namespace too long for temporary collections in Map Reduce
Branch: master
https://github.com/mongodb/mongo/commit/4fdb96d39226408e8e841a01d5534a432f7ea567

Generated at Thu Feb 08 03:02:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.