[SERVER-9182] mongorestore won't restore manually edited .metadata.json file if user editor adds trailing newline Created: 29/Mar/13  Updated: 11/Jul/16  Resolved: 29/Mar/13

Status: Closed
Project: Core Server
Component/s: Tools
Affects Version/s: 2.2.3, 2.4.1
Fix Version/s: 2.4.2, 2.5.0

Type: Bug Priority: Critical - P2
Reporter: J Rassi Assignee: J Rassi
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
Related
is related to SERVER-6947 db.createCollection creates undefined... Closed
Operating System: ALL
Participants:

 Description   

SERVER-6947 describes an issue where collections created with createCollection are dumped incorrectly with all mongodump 2.2.x/2.4.x versions currently released. The incorrect dumps are characterized by invalid options present in the JSON metadata file for these collections.

One known workaround for this issue is to manually edit the JSON metadata files to correct the invalid options (note: another known workaround is to restore with mongorestore >=2.4). However, this workaround has a flaw. The JSON metadata files are created without trailing newlines; however, when users attempt to edit these files to implement the workaround, their editors will often add trailing newlines.

Restoring with these edited metadata files trips a fatal assertion in mongo::Restore::parseMetadataFile, which checks that there are no bytes in the file past the end of the first encountered JSON object. This check fails because of the trailing newline. The assertion generates the unhelpful message "JSON object size didn't match file size".

Reproduce with:

rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $ ./mongo --eval 'db.dropDatabase()'
MongoDB shell version: 2.4.1
connecting to: test
[object Object]
rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $ ./mongo --eval 'db.foo.insert({})'
MongoDB shell version: 2.4.1
connecting to: test
rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $ ./mongodump -d test
connected to: 127.0.0.1
Fri Mar 29 16:09:32.127 DATABASE: test	 to 	dump/test
Fri Mar 29 16:09:32.129 	test.system.indexes to dump/test/system.indexes.bson
Fri Mar 29 16:09:32.130 		 1 objects
Fri Mar 29 16:09:32.131 	test.foo to dump/test/foo.bson
Fri Mar 29 16:09:32.132 		 1 objects
Fri Mar 29 16:09:32.133 	Metadata for test.foo to dump/test/foo.metadata.json
rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $ echo >> dump/test/foo.metadata.json
rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $ ./mongo --eval 'db.dropDatabase()'
MongoDB shell version: 2.4.1
connecting to: test
[object Object]
rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $ ./mongorestore
connected to: 127.0.0.1
Fri Mar 29 16:09:54.016 dump/test/foo.bson
Fri Mar 29 16:09:54.017 	going into namespace [test.foo]
assertion: 15934 JSON object size didn't match file size
rassi@laptop:~/dist/mongodb-osx-x86_64-2.4.1/bin $



 Comments   
Comment by auto [ 29/Mar/13 ]

Author:

{u'date': u'2013-03-29T22:40:30Z', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-9182 Allow restores of dumps containing edited .metadata.json files
Branch: v2.4
https://github.com/mongodb/mongo/commit/3bf409d4e597a0afd48361245e53e38b2e2b6e21

Comment by auto [ 29/Mar/13 ]

Author:

{u'date': u'2013-03-29T22:40:30Z', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-9182 Allow restores of dumps containing edited .metadata.json files
Branch: master
https://github.com/mongodb/mongo/commit/afb0f9b2434d53642df88376a30ee519c125d99d

Generated at Thu Feb 08 03:19:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.