[SERVER-6342] indexes in sharding configuration Created: 07/Jul/12 Updated: 15/Aug/12 Resolved: 11/Jul/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Azat Khuzhin | Assignee: | Greg Studer |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
version |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
I have two shards: "shard0000", "shard0001" And I move collection from main shard ("shard0000") to "shard0001" |
| Comments |
| Comment by Greg Studer [ 11/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Resolved as duplicate for better triaging, thanks for your help tracking this down! Linked issue has a new test which reproduces - it's easier to trigger with multiple mongoses as the original mongos will "remember" that the collection started on a particular shard. There are a few workarounds for now - 1) use coll.stats(), which is more accurate, or 2) create the index using the same mongos as was used to do the collection setup - this should prevent the problem from happening at least initially. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Azat Khuzhin [ 11/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
"flushRouterConfig" didn't helps | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Greg Studer [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Sorry, you were correct earlier - the code recently changed, index writes go to all shards. Still not able to reproduce on the version you've specified, however. Does the problem go away if you run flushRouterConfig on the mongos before ensureIndex - wonder if this is a known issue with commands and config refreshing. > Why no data will be collected? The find part of a moveChunk is just a single "location", not a query, that identifies which chunk should be moved. If you know the full bounds, you can just specify the low-bound. > Also are you sure that "mongod" detect indexes right in this situation, and will not create duplicate indexes? | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Azat Khuzhin [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
About what's happening here: I thought in the same direction. Because some mongodb drivers, can not supporting "ensureIndex" command, just "createIndex", and must check manually. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Azat Khuzhin [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
> Not sure I fully understand - but the second parameter here is the "find" parameter in the moveChunk command - it won't create a field it's just used as a selector for the chunk. Since you only have one chunk, -Inf to Inf, any value of _id will match that chunk, and no data will be collected. Why no data will be collected? | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Greg Studer [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Oh right, just realized what's happening here - When you run ensureIndex, it's targeted only to shards which contain chunks. Since all the data for your collection, once moved, is on shard0001, an index is only created on shard0001. This isn't a problem, however, since when a chunks is moved to a shard, it is ensured that all indexes on the old shard are recreated on the new shard. If you create the index prior to moving the chunk, you'll end up with indexes on both shards. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Greg Studer [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Will see if getIndexes() is displaying something incorrectly. > I don't want to add separate field for this, so because of this I use "exists" Not sure I fully understand - but the second parameter here is the "find" parameter in the moveChunk command - it won't create a field it's just used as a selector for the chunk. Since you only have one chunk, -Inf to Inf, any value of _id will match that chunk, and no data will be collected. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Azat Khuzhin [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Also in your script you don't run 'disableBalancing', but this is insignificantly And I think, I git this, in your script you run's 'db.foobar.stats()' while I run 'db.foobar.getIndexes()'
| |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Azat Khuzhin [ 10/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
I don't want to add separate field for this, so because of this I use "exists" | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Greg Studer [ 09/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Also, in the script you've posted, I don't think you want to use : {_id: {$exists: 1}} - the query operator is interpreted as a portion of the query range. Assuming the collection isn't split, any constant value will work here. | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Greg Studer [ 09/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
I can't seem to reproduce this problem with the git hash you posted - attached a script above that creates a collection, shards and moves it, and creates an index. Are you issuing the "ensureIndex" command against just one shard? | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Azat Khuzhin [ 08/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
Using this function
Create index using "ensureIndex" function | |||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 08/Jul/12 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||
|
How did you move the collection? How did you create the index? Was it through mongos, or directly connected to the shard, and if direct which shard? |