[SERVER-2231] Existing sharding rules seems to be immutable even if i drop the target collection and rebuild one. Created: 16/Dec/10  Updated: 30/Mar/12  Resolved: 02/Sep/11

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 1.6.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Guti.Deng Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS release 5.4 (Final) , mongodb-linux-x86_64-1.6.4.tgz


Operating System: Linux
Participants:

 Description   

I've made a mistake by sharding a GridFS chunks collection with it's '_id'.
Then i got told that, the chunks collection should be sharded with

{files_id: 1, n: 1}

.

I tried to runCommand({shardcollection: "xx.xx.chunks', key: {files_id: 1, n: 1}}), then mongo told me something like 'the collection has already been sharded'.
Cannot find a command to 'remove an existing sharding rule' so i drop the chunks collection, then rerun the command above, no errors.

Then, with in database 'admin', i runned 'db.printShardingStatus()'. The result showed that the sharding rules on "xx.xx.chunks" had been modified as i wish.

But when i run my script to load data into this GridFS, the same error message from pymongo raised.

I've already found a solution, quite ugly by renaming the GridFS....



 Comments   
Comment by Eliot Horowitz (Inactive) [ 16/Dec/10 ]

Yes - jira is not the best source of current information.

It should be sharded by files_id only

Comment by Guti.Deng [ 16/Dec/10 ]

The latter problem is solved by replace chunk key:

{files_id:1, n:1}

with

{files_id: 1}

.

In this issue(http://jira.mongodb.org/browse/SERVER-889),

{files_id:1, n:1}

is recommended.

Some mechanism changed in the past year?

Would it be nice if the drivers took care of sharding GridFSes ?

Comment by Guti.Deng [ 16/Dec/10 ]

Another strange thing happens.
I'm using a fresh new GridFS with fresh new sharding rules.
Seems that the first file was saved successfully, while the second failed.

From replica-set s002/primary:

Thu Dec 16 12:21:32 [conn32] building new index on

{ _id: 1 }

for weibo.gfs_msg_rt.files
Thu Dec 16 12:23:28 [conn32] info: creating collection weibo.gfs_msg_rt.files on add index
building new index on

{ _id: 1 }

for weibo.gfs_msg_rt.chunks
Thu Dec 16 12:23:28 [conn32] info: creating collection weibo.gfs_msg_rt.chunks on add index
building new index on

{ files_id: 1.0, n: 1.0 }

for weibo.gfs_msg_rt.chunks

Operations and results:

> use admin
switched to db admin
> db.runCommand({shardcollection: "weibo.gfs_msg_rt.files", key: {_id: 1}})

{ "collectionsharded" : "weibo.gfs_msg_rt.files", "ok" : 1 }

<ction: "weibo.gfs_msg_rt.chunks", key: {files_id:1, n:1}})

{ "collectionsharded" : "weibo.gfs_msg_rt.chunks", "ok" : 1 }

Process TreeLoader-1:5:
Traceback (most recent call last):
File "/usr/local/lib/python2.6/multiprocessing/process.py", line 232, in _bootstrap
self.run()
File "/data0/guti6/m/TreeLoader.py", line 84, in run
gfs.put(body, _id=name, chunkSize=csize)
File "build/bdist.linux-x86_64/egg/gridfs/_init_.py", line 116, in put
grid_file.close()
File "build/bdist.linux-x86_64/egg/gridfs/grid_file.py", line 220, in close
self.__flush()
File "build/bdist.linux-x86_64/egg/gridfs/grid_file.py", line 202, in __flush
root=self._coll.name)["md5"]
File "build/bdist.linux-x86_64/egg/pymongo/database.py", line 293, in command
msg, allowable_errors)
File "build/bdist.linux-x86_64/egg/pymongo/helpers.py", line 119, in _check_command_response
raise OperationFailure(msg % response["errmsg"])
OperationFailure: command SON([('filemd5', '20110071131055328'), ('root', u'gfs_msg_rt')]) failed: db assertion failure

> use weibo
switched to db weibo
> db.gfs_msg_rt.chunks.count()
2
> db.gfs_msg_rt.files.count()
1
> db.gfs_msg_rt.files.find()

{ "_closed" : true, "_id" : "20110071131055329", "chunkSize" : 64, "length" : 50, "md5" : "7ff8708d75c1a7c77cd5277a621e3e5f", "uploadDate" : "Thu Dec 16 2010 12:35:26 GMT+0800 (CDT)" }

> db.gfs_msg_rt.chunks.find()

{ "_id" : ObjectId("4d09970e5d4bb07a3f000001"), "n" : 0, "data" : BinData(2,"MgAAAHsiMjAxMTAwNzExMzEyMzE1NjUiOiB7fSwgIjIwMTEwMDcxMTMxMDYzNTUwIjoge319"), "files_id" : "20110071131055329" } { "_id" : ObjectId("4d09970e5d4bb07a3f000003"), "n" : 0, "data" : BinData(2,"SQAAAHsiMjAxMTAwNzExMzE2NTc0ODIiOiB7fSwgIjIwMTEwMDcxMTMxMjM2NzUxIjogeyIyMDExMDA3MTEzMTU2MzgwOSI6IHt9fX0="), "files_id" : "20110071131055328" }

weibo.gfs_msg_rt.chunks chunks:
{ "files_id" :

{ $minKey : 1 }

, "n" :

{ $minKey : 1 }

} -->> { "files_id" :

{ $maxKey : 1 }

, "n" :

{ $maxKey : 1 }

} on : s002

{ "t" : 1000, "i" : 0 }

weibo.gfs_msg_rt.files chunks:
{ "_id" :

{ $minKey : 1 }

} -->> { "_id" :

{ $maxKey : 1 }

} on : s002

{ "t" : 1000, "i" : 0 }
Comment by Guti.Deng [ 16/Dec/10 ]
{ files_id: 1.0, n: 1.0 }

looks strange. I'm sure what i typed is

{files_id: 1, n:1}

.

Comment by Guti.Deng [ 16/Dec/10 ]

I grep 'weibo.gfs_msg_rtree' from repset s002/primary, where printShardingStatus indicates that it is the only location of 'weibo.gfs_msg_rtree.files' and 'weibo.gfs_msg_rtree.chunks'.
May logs about this appeared in other servers? Config server? Mongos?

Wed Dec 15 22:22:37 [conn31] building new index on

{ _id: 1 }

for weibo.gfs_msg_rtree.files
Wed Dec 15 22:22:37 [conn31] info: creating collection weibo.gfs_msg_rtree.files on add index
Wed Dec 15 22:23:25 [conn31] building new index on

{ _id: 1 }

for weibo.gfs_msg_rtree.chunks
Wed Dec 15 22:26:49 [conn31] info: creating collection weibo.gfs_msg_rtree.chunks on add index
Thu Dec 16 11:28:18 [conn14] building new index on

{ files_id: 1, n: 1 }

for weibo.gfs_msg_rtree.chunks
Thu Dec 16 11:59:05 [conn29] CMD: drop weibo.gfs_msg_rtree.chunks
Thu Dec 16 11:59:06 [conn29] wiping data for: weibo.gfs_msg_rtree.chunks
Thu Dec 16 11:59:10 [conn29] CMD: drop weibo.gfs_msg_rtree.files
Thu Dec 16 11:59:10 [conn29] wiping data for: weibo.gfs_msg_rtree.files
Thu Dec 16 11:59:44 [conn30] building new index on

{ _id: 1 }

for weibo.gfs_msg_rtree.chunks
Thu Dec 16 11:59:44 [conn30] info: creating collection weibo.gfs_msg_rtree.chunks on add index
building new index on

{ files_id: 1.0 }

for weibo.gfs_msg_rtree.chunks
Thu Dec 16 11:59:57 [conn30] building new index on

{ _id: 1 }

for weibo.gfs_msg_rtree.files
Thu Dec 16 12:04:36 [conn30] info: creating collection weibo.gfs_msg_rtree.files on add index
Thu Dec 16 12:08:40 [conn31] CMD: drop weibo.gfs_msg_rtree.files
Thu Dec 16 12:08:40 [conn31] wiping data for: weibo.gfs_msg_rtree.files
Thu Dec 16 12:08:46 [conn31] CMD: drop weibo.gfs_msg_rtree.chunks
Thu Dec 16 12:09:11 [conn19] building new index on

{ _id: 1 }

for weibo.gfs_msg_rtree.chunks
Thu Dec 16 12:09:11 [conn19] info: creating collection weibo.gfs_msg_rtree.chunks on add index
building new index on

{ files_id: 1.0, n: 1.0 }

for weibo.gfs_msg_rtree.chunks
Thu Dec 16 12:09:35 [conn19] building new index on

{ _id: 1 }

for weibo.gfs_msg_rtree.files
Thu Dec 16 12:12:57 [conn19] info: creating collection weibo.gfs_msg_rtree.files on add index
Thu Dec 16 12:16:52 [conn19] CMD: drop weibo.gfs_msg_rtree.chunks
Thu Dec 16 12:16:59 [conn19] CMD: drop weibo.gfs_msg_rtree.files

Comment by Eliot Horowitz (Inactive) [ 16/Dec/10 ]

Do you still have the logs?
We have tests that check for exactly this.

Generated at Thu Feb 08 02:59:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.