[SERVER-7693] Data loss possibly related to composite-compound index invalid for splitting chunks when used for sharding.... new feature 2.2.0 Created: 16/Nov/12 Updated: 08/Mar/13 Resolved: 17/Jan/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance |
| Affects Version/s: | 2.2.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Robert Jobson | Assignee: | Stennie Steneker (Inactive) |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | nh-240 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Steps To Reproduce: | manual test logged directly to shard. > db.listing_fields209_763293.getIndexes(); , , > db.listing_fields209_763293.find().min( {hash_fk_id: "753af0f2"}).max( {hash_fk_ id: MaxKey}); max: { hash_fk_id: MaxKey }", executed on router.. mongos> db.listing_fields209_763293.ensureIndex( {hash_fk_id:1}); db.listing_fields209_763293.find().min( {hash_fk_id: "753af0f2"}).max( {hash_fk_id: MaxKey}); query completes successfully after index added. This collection is created, composite index added to hash_fk_id,fk_type_id then sharded on the leading edge hash_fk_id. Let me know if you need more information. |
| Participants: |
| Description |
|
found a problem where the leading edge of a compound key will not su error from log Thu Nov 15 18:16:12 [conn126507] problem detected during query over homes_didx.listing_fields209_763293 : { $err: "no index found for specified keyPattern: {} min: { hash_fk_id: "753af0f2" }max: { hash_fk_id: MaxKey }", code: 10367 } As a work around we are removing the compound index. |
| Comments |
| Comment by Stennie Steneker (Inactive) [ 17/Jan/13 ] |
|
Hi Robert, I'm going to close this issue as "Cannot Reproduce". If you do re-encounter this issue please let us know. Thanks, |
| Comment by Robert Jobson [ 11/Dec/12 ] |
|
We are currently on 2.2.1. I probably won't be testing and building an example script any time soon, very busy lately. edit I'm not sure this was clear. We found the error on 2.2.0 and upgraded to 2.2.1 at a later time. Also we removed the leading-edge compound indexes as shard keys. Unless this issue is corroborated by another source I would be content with the issue being closed. |
| Comment by Stennie Steneker (Inactive) [ 05/Dec/12 ] |
|
Hi Robert, Do you have any update on this issue? Can you confirm what version of MongoDB you are using? A test script which reproduces the issue would be helpful. Thanks, |
| Comment by Kevin Matulef [ 16/Nov/12 ] |
|
If you're sharding on a prefix A and using a compound index on (A,B), then for sharding purposes, internally we extend the chunk bounds on the interval [x,y) to match the index, i.e. we produce the interval [(x,MinKey),(y,MinKey)). The min() and max() query operators don't automatically extend the bounds, since they're currently only meant to be used with a matching index. Are you using 2.2.0 or 2.2.1? There were some issues related to sharding on a prefix that were fixed between 2.2.0 and 2.2.1, but as far as I'm aware, none of them would lead to data loss. If you're able to reproduce this on 2.2.1, would you mind posting your test script? |
| Comment by Robert Jobson [ 16/Nov/12 ] |
|
We have missing data from inserts to the collection around the same time (within a second) as the error in the log. These inserts were done in safe mode with a perl driver and no error was returned. We believe it is related but haven't verified and will continue to monitor now that we have implemented the work around (using single column indexes). I think it should be clear that I believe this error was from the internal mongo mechanisms for splitting a chunk. If it cannot use the leading edge of a compound index to maintain sharding then how does it work around this to split the chunk? |
| Comment by Eliot Horowitz (Inactive) [ 16/Nov/12 ] |
|
The min and max helpers have to be used on exact index fields, and thus can't work on just the first part of the index. |