[SERVER-12983] Restore from 2.4 DB into 2.6 DB fails due to index? Created: 01/Mar/14 Updated: 20/Jan/16 Resolved: 02/Mar/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Tools |
| Affects Version/s: | 2.6.0-rc0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Michael Kennedy | Assignee: | Unassigned |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Steps To Reproduce: | Use this command and see screenshot. mongorestore --drop --db DB ".\DB" |
||||||||
| Participants: | |||||||||
| Description |
|
It appears that if a 2.4 DB which was backed up using MongoDump is then restored via MongoRestore into 2.6 and that dump contains an index which is too large to index, then the restore stops and leaves the DB incompleted. Notice how the last line from MongoRestore is Error creating index LearningLine.Conversation: 17282 err: "Btree::insert: key too large to index, failing LearningLine.Conversation.$find_messages_by_activity 1426 { : { Attachments: [], LastAttachmentId: 0, C..." This is NOT the final collection to restore. |
| Comments |
| Comment by Daniel Doubrovkine [ 20/Jan/16 ] |
|
Yup, opened https://jira.mongodb.org/browse/SERVER-22251, the 2.6 version of the shell at least seems to be doing something. |
| Comment by Daniel Pasette (Inactive) [ 20/Jan/16 ] |
|
Pretty sure you've found a bug in the upgradeChecker in 3.2.1. Can you use the 2.6 mongo shell and try again. Also, upgradeCheckAllDBs() is just a convenience method to do the iteration for you. |
| Comment by Daniel Doubrovkine [ 20/Jan/16 ] |
|
Running 3.2.1 shell against a 2.4 MongoDB. $ mongo ... ^^^ lies I cannot run upgradeCheckAllDBs() because this is a Compose deployment, I don't have admin access. I assume that it just iterates over and does upgradeCheck? |
| Comment by Daniel Pasette (Inactive) [ 20/Jan/16 ] |
|
What exactly are you running for upgradeCheck? What version of the shell and mongodb are you using? |
| Comment by Daniel Doubrovkine [ 20/Jan/16 ] |
|
Finding these has been a painful trial/error process, mongodump, mongorestore, restart once the next one fails. db.upgradeCheck() returned nothing The error from the index creation from mongoimport just leaves you guessing, hunting for the document ID that causes the problem. It would be nice it it at least returned the ID of the document - the one below for example doesn't get found with a regex because it's not in the index (partial index)? Failed: gravity-staging.events: error creating indexes for gravity-staging.events: createIndex error: Btree::insert: key too large to index, failing gravity-staging.events.$name_1 6590 { : "Activities New York / Description Soonja Han's Paris Studio gets reconstructed within the de Buck gallery in Chelsea, NY: An interrogation of the ar..." }
|
| Comment by Daniel Pasette (Inactive) [ 20/Jan/16 ] |
|
Hi DB, This is documented here: |
| Comment by Daniel Doubrovkine [ 20/Jan/16 ] |
|
FWIW I spent the majority of the afternoon working around this error, collection after collection. We query data by the problematic indexe(s) so this restriction and the inability to ignore and continue came as a surprise on a mongorestore. Being able to run the server in an old(er) mode is fine, but we don't run our own, I would have liked to get (or be able to get) a warning instead of a hard error here. Related, I couldn't find in the documentation anything about the semantics of inserting a record when the index key produced would be too long - does the record get inserted or not? |
| Comment by Michael Kennedy [ 03/Mar/14 ] |
|
Hi Dan, Yes, that is what my understanding was. I filed the bug because I think this is wrong. It's fine for runtime. But I want a flag or something for *mongorestore* which says something like 'continue on index error' or 'drop indexes that are incompatible with 2.6' or something. I do not want to run the server is a downgraded mode. I just want to import the data, acknowledging the problems and changes. |
| Comment by Daniel Pasette (Inactive) [ 03/Mar/14 ] |
|
With the new restrictions on index key length, you have a couple options to restore databases with such documents. In either case, it is highly recommended that the documents are cleaned up to avoid condition if possible as results from indexes will not be usable for docs with this keys that are too long. |
| Comment by Michael Kennedy [ 02/Mar/14 ] |
|
Thanks for looking at this Dan. I'm not sure I would classify it as duplicate. I think there is a difference between attempting a DB restore and attempting to create an improper index and runtime or doing an insert which is invalid. I don't want to run the server in a degraded mode, I just want to restore data and acknowledge any index errors (but continue) |
| Comment by Daniel Pasette (Inactive) [ 02/Mar/14 ] |
|
duplicate of |