[SERVER-889] sharding by "_id" dosen't work for gridfs chunks Created: 21/Oct/09  Updated: 12/Jul/16  Resolved: 29/Oct/09

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 1.1.3

Type: Improvement Priority: Major - P3
Reporter: Tomasz Drobiszewski Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

On two servers Linux scorpius 2.6.28-15-server #52-Ubuntu SMP Wed Sep 9 11:50:50 UTC 2009 i686 GNU/Linux
mongodb-linux-i686-2009-10-16


Attachments: File test_load_file.py    
Issue Links:
Depends
depends on SERVER-888 "remove all" via the java driver does... Closed
Participants:

 Description   

Initialization:
scorpius$./mongod --dbpath /data/db/a --port 2048
scorpius$./mongod --dbpath /data/db/config --port 2049
scorpius$./mongos --configdb localhost:2049
andro3$./mongod --dbpath /data/db/c/ --port 2048
scorpius$./mongo
scorpius ~/tmp/mongodb-linux-i686-2009-10-16/bin $./mongo
MongoDB shell version: 1.1.2-
url: test
connecting to: test
type "help" for help
> use admin
switched to db admin
> db.runCommand(

{ addshard : "andro3:2048" }

);

{ "ok" : 1, "added" : "andro3:2048" }

> db.runCommand(

{ addshard : "scorpius:2048" }

);

{ "ok" : 1, "added" : "scorpius:2048" }

> admin = db.getSisterDB( "test" );
test
> db.runCommand(

{ enablesharding : "test" }

)

{ "ok" : 1 }

> db
admin
> db.runCommand( { shardcollection : "test.fs.chunks", key :

{ _id : 1 }

} )

{ "collectionsharded" : "test.fs.chunks", "ok" : 1 }

$./test_load_file.py Dexter.S04E03.HDTV.XviD-SYS.avi
=========================================================================================
On mongos console:
Wed Oct 21 15:03:43 ./mongos v0.3- (alpha 3) starting (--help for usage)
Wed Oct 21 15:03:43 git version: 2660c0ca229a557692780784a2b8b1a8ba7afd1f
Wed Oct 21 15:03:43 sys info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686
Wed Oct 21 15:03:43 connecting to griddb localhost:2049...Wed Oct 21 15:03:44 waiting for connections on port 27017
Wed Oct 21 15:16:00 connection accepted from 127.0.0.1:7813 #1
Wed Oct 21 15:16:29 couldn't find database [admin] in config db
Wed Oct 21 15:16:29 put [admin] on: localhost:2049
Wed Oct 21 15:17:57 couldn't find database [test] in config db
Wed Oct 21 15:17:57 put [test] on: scorpius:2048
Wed Oct 21 15:18:35 no chunks for:test.fs.chunks so creating first: shard ns:test.fs.chunks shard: scorpius:2048 min:

{ _id: MinKey }

max:

{ _id: MaxKey }

before:

{ _id: ObjId(4adf0a2b581c741d1b0dcb4c) }

shard ns:test.fs.chunks shard: scorpius:2048 min:

{ _id: MinKey }

max:

{ _id: MaxKey }

after :

{ _id: ObjId(4adf0a2b581c741d1b0dcb4c) }

shard ns:test.fs.chunks shard: scorpius:2048 min:

{ _id: MinKey }

max:

{ _id: MaxKey }

Wed Oct 21 15:19:27 connection accepted from 127.0.0.1:43752 #2
Wed Oct 21 15:19:27 end connection 127.0.0.1:43752
Wed Oct 21 15:19:27 connection accepted from 127.0.0.1:44008 #3
Wed Oct 21 15:19:27 User Exception can't use unique indexes with sharding
Wed Oct 21 15:19:27 UserException: can't use unique indexes with sharding
creating WriteBackListener for: scorpius:2048
Wed Oct 21 15:19:27 UserException: can only delete with a non-shard key pattern if can delete as many as we find
Wed Oct 21 15:19:27 UserException: can't upsert something without shard key
Wed Oct 21 15:19:27 UserException: can't upsert something without shard key
=========================================================================================
$./mongo
> use test
switched to db test
> show collections
fs.chunks
fs.files
system.indexes
> db.fs.files.find()

{ "_id" : ObjectId("4adf0a5f8f299e42c9000000"), "chunkSize" : 256000, "filename" : "Dexter.S04E03.HDTV.XviD-SYS.avi", "length" : 0, "uploadDate" : "Wed Oct 21 2009 15:19:27 GMT+0200 (CEST)", "next" : null }

> db.fs.chunks.find()
=========================================================================================

Replace db.runCommand( { shardcollection : "test.fs.chunks", key :

{ _id : 1 }

} ) to db.runCommand( { shardcollection : "test.fs.chunks", key :

{ n : 1 }

} ) everything going to work.



 Comments   
Comment by Eliot Horowitz (Inactive) [ 16/Dec/10 ]

NOTE: currently it has to be sharded

{ files_id }
Comment by Eliot Horowitz (Inactive) [ 29/Oct/09 ]

fixed some of the underlying issues, but...

the root problem is that there are operationson gridfs that don't many sense when you shard by _id

chunks need to be sharded on

{ files_id , n }

since that's what it looks up chunks by, not _id

Generated at Thu Feb 08 02:55:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.