[SERVER-15912] Unable to restore dumped collection Created: 02/Nov/14 Updated: 12/Nov/14 Resolved: 12/Nov/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Tools |
| Affects Version/s: | 2.6.5 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Pavlo Grinchenko | Assignee: | Ramon Fernandez Marina |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Steps To Reproduce: | 1) Dump collection |
| Participants: |
| Description |
|
We have the following problem:
Error stack trace looks like this:
We can provide actual dump file upon request - it's 865MB |
| Comments |
| Comment by Ramon Fernandez Marina [ 12/Nov/14 ] | |
|
Hi paulgpa, glad to hear you found the root cause of the issue. In its defense, mongorestore does flag the corruption and aborts, although I would agree it doesn't do this in a very friendly manner. Thanks for keeping us posted, closing this ticket now. | |
| Comment by Pavlo Grinchenko [ 12/Nov/14 ] | |
|
OK - I think we found the source of the issue. On a Mac by default TGZ files are unpacked with Unarchiver tool. On the dump archive it failed and extracted file partially. Unfortunately your mongorestore tool doesn't message corruption properly. We used command line unarchive approach and it was restored properly. Sorry for confusion, but the issue is on my side, but not yours. Please close this one. | |
| Comment by Pavlo Grinchenko [ 12/Nov/14 ] | |
|
I tried to restore this data dump using 2.6.5 tools and got the same issue:
| |
| Comment by Ramon Fernandez Marina [ 06/Nov/14 ] | |
|
Thanks paulgpa, we'll let you know what we find. | |
| Comment by Pavlo Grinchenko [ 06/Nov/14 ] | |
|
Ramon - we uploaded data dump. | |
| Comment by Pavlo Grinchenko [ 04/Nov/14 ] | |
|
Thanks for the follow-up Ramon. 1) I will upload the dump file as soon as I can. I will work with my operations team to do this. Unfortunately I am on the road this week and doubt that hotel's connection will allow me to do this efficiently. 2) Our configuration: 6 shards - each represented by the replica set with 3 nodes. Fairly standard configuration. | |
| Comment by Ramon Fernandez Marina [ 04/Nov/14 ] | |
|
Thanks for the additional information paulgpa. Can you please upload your dump file so we can examine it? Here's how:
where <dumpfile> is the dump file produced by mongodump. When prompted for a password just hit enter. Also you say you have six shards, are these stand-alone or replica sets? If the latter, are you taking the dump from the primary? Can you please include the command line(s) you used to produce the dumpfile? Thanks for offering to try the new tools, that should provide useful information on our effort to troubleshoot this ticket. | |
| Comment by Pavlo Grinchenko [ 04/Nov/14 ] | |
|
Ramon Thanks a lot for your suggestion. This is a completely repeatable problem. We reproduced it 2 times with different operations team members. We will try your 2.7.7/2.7.8 tools proposal. Do you think it will work properly with a sharded environment that's on 2.6.5? | |
| Comment by Ramon Fernandez Marina [ 03/Nov/14 ] | |
|
paulgpa, note that the MongoDB tools have been entirely re-written for the 2.7.7 development release, so if a new dump+restore cycle shows the same problem I'd recommend you try with the new tools, which can be downloaded as part of the 2.7.7/2.7.8 development releases or cloned from github. If the new tools also show the issue please let us know, and I'll send you instructions to upload the dump file. | |
| Comment by Ramon Fernandez Marina [ 03/Nov/14 ] | |
|
paulgpa, is this a repeatable problem? Can you re-dump the collection? The error you're seeing could indicate data corruption on the dump file, so I'd recommend you try to re-dump the collection to a new dump file while checking the OS logs for possible disk-related errors. If no disk errors appear, the dump succeeds but the restore fails again, there may be a bug in mongodump/restore that would need investigation. | |
| Comment by Pavlo Grinchenko [ 03/Nov/14 ] | |
I will upgrade to the latest one and try again |