[SERVER-27765] MongoDB, python and pymongo: BSONObj size is invalid Created: 19/Jan/17 Updated: 24/Jan/17 Resolved: 21/Jan/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 2.6.12 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Antani | Assignee: | Kelsey Schubert |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Participants: |
| Description |
|
I am running mongodb 2.6.12 on ubuntu 64bits. We have an instance with more than 10TB of scientific research data, a python app is using the db via pymongo. Suddenly, after a db compact the database I was/am unable to run a mongodump because of errors like this:
I also tried to run a db repair, but I got this:
So now we are unable to fix the db, someone can help? Unfortunately, we don't have many foundings, so we were unable to pay a backup server for so much data. |
| Comments |
| Comment by Kelsey Schubert [ 24/Jan/17 ] | |
|
Hi antan23, Please continue to run the mongodump operation until it completes. In some situations, the estimate provided by the progress bar may not be accurate. Thank you, | |
| Comment by Antani [ 21/Jan/17 ] | |
|
Thanks for your reply. I already stopped the mongo deamon and tried a: mongodump --repair --dbpath /path/to/db --db dbname --forceTableScan But after reaching the 100% it was going on and I stopped it at 114% (at that point I thought it was not working. I am not sure if mongod should be stopped running this operation or why this is going over 100%, any hint? Thanks so much. | |
| Comment by Kelsey Schubert [ 20/Jan/17 ] | |
|
Hi antan23, Unfortunately, this error indicates that the data became corrupted in some way. It's not clear whether this due to a faulty storage layer or some other issue. In cases like this identifying the root cause can be very challenging without a reproduction. Please note that MongoDB 2.6.12 is no longer supported, and we recommend upgrading to a newer version. I see that the repairDatabase command was unsuccessful at recovering your data. Therefore, my advice would be to create a copy of your current $dbpath, and execute the following command as a last effort.
mongodump in MongoDB 2.6 utilizes a different algorithm to repair corruption, which may be able to recover some of your data. For more details about executing this command, please review our documentation. Please be aware that some manual intervention may be required to deduplicate documents after this process completes and there is no guarantee that this operation will recover all or most of your data depending on the extent of corruption. I understand that there are costs associated with the backup methods we recommend. By upgrading to MongoDB 3.0 or later, you would be able to take advantage of WiredTiger's Compression to reduce the data size of your files on disk, and, consequently, the associated costs to store backups. Kind regards, |