[SERVER-12092] Modifying collection options can cause restores of collection to fail Created: 13/Dec/13  Updated: 11/Jul/16  Resolved: 17/Dec/13

Status: Closed
Project: Core Server
Component/s: Internal Code, Tools
Affects Version/s: 2.4.8
Fix Version/s: 2.4.9, 2.5.5

Type: Bug Priority: Major - P3
Reporter: Shaun Verch Assignee: J Rassi
Resolution: Done Votes: 0
Labels: 26qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Operating System: ALL
Participants:
Linked BF Score: 0

 Description   
Issue Status as of December 30th, 2013

ISSUE SUMMARY
With mongorestore v2.4.8 and earlier, mongorestore is unable to restore some collections that meet both of the following conditions:

  • the create command was used to create the collection
  • the collection's options were modified (through the collMod command, or through creation of a TTL or text index)

The process of updating the collection options was found in these cases to sometimes corrupt the collection metadata (as stored in the system collection system.namespaces). Other than affecting the ability for the collection to be restored with mongorestore, the corruption is benign.

USER IMPACT
When mongorestore encounters a collection affected by this issue, it will log the error message "no such cmd" and exit with a non-zero exit status. Data will only be partially restored.

SOLUTION
mongorestore was fixed to understand collection metadata that was corrupted in this way.

WORKAROUNDS
It is recommended that mongorestore v2.4.9 or greater be used to complete the restore if this issue is encountered (even if the server being restored to is using a lesser version), but the following workaround may be employed instead if a fixed version is not available:

  • With a text editor, open the <collection>.metadata.json file in the dump directory for the collection that failed to be restored.
  • Remove the {create: <collection>} field from the "options" subdocument.
  • Repeat the restore process.

PATCHES
Production release v2.4.9 contains the fix for this issue, and production release v2.6.0 will contain the fix as well. Use mongorestore v2.4.9 or later.

Previous Description

If the "create" command is used to create a collection, the field "create" is included as the first collection option in the collection's system.namespaces entry. If the collection options are later modified, the internal update to the system.namespaces collection can re-order the "create" field such that it no longer appears first. Dumping/restoring this collection will subsequently not succeed: mongorestore will send a garbled command to the server upon attempting to re-create the collection, as it assumes that "create" is the first element in the options document (if it appears at all).

Reproduce with:

> db.createCollection("foo", {"capped" : true, "size" : 10000})
{ "ok" : 1 }
> db.system.namespaces.find()
{ "name" : "test.system.indexes" }
{ "name" : "test.foo.$_id_" }
{ "name" : "test.foo", "options" : { "create" : "foo", "capped" : true, "size" : 10000 } }
> db.foo.runCommand("collMod",{usePowerOf2Sizes:true})
{ "usePowerOf2Sizes_old" : false, "usePowerOf2Sizes_new" : true, "ok" : 1 }
> db.system.namespaces.find()
{ "name" : "test.system.indexes" }
{ "name" : "test.foo.$_id_" }
{ "name" : "test.foo", "options" : { "capped" : true, "create" : "foo", "flags" : 1, "size" : 10000 } }
>

Dumping/restoring this collection fails with "no such command: capped".

This does not affect mongod 2.5.x, since the update refactor causes updates to no longer re-order fields.

Original ticket description:

Run against a 2.4 mongod with textSearchEnabled=true
 
> db.dropDatabase()
> db.createCollection("test", { "capped" : true, "size" : 10000 })
> db.test.ensureIndex({ "a" : "text" })
> exit
 
$ ./mongodump
connected to: 127.0.0.1
2013-12-13T16:20:59.110-0500 all dbs
2013-12-13T16:20:59.111-0500 DATABASE: test	 to 	dump/test
2013-12-13T16:20:59.111-0500 	test.system.indexes to dump/test/system.indexes.bson
2013-12-13T16:20:59.111-0500 		 2 objects
2013-12-13T16:20:59.111-0500 	test.test to dump/test/test.bson
2013-12-13T16:20:59.112-0500 		 0 objects
2013-12-13T16:20:59.112-0500 	Metadata for test.test to dump/test/test.metadata.json
 
> db.dropDatabase()
 
$ ./mongorestore
connected to: 127.0.0.1
2013-12-13T16:21:09.000-0500 dump/test/collection.bson
2013-12-13T16:21:09.000-0500 	going into namespace [test.collection]
2013-12-13T16:21:09.245-0500 	Created collection test.collection with options: { "create" : "collection", "max" : 2, "capped" : true, "size" : 100000 }
file dump/test/collection.bson empty, skipping
2013-12-13T16:21:09.245-0500 	Creating index: { key: { _id: 1 }, ns: "test.collection", name: "_id_" }
2013-12-13T16:21:09.246-0500 dump/test/test.bson
2013-12-13T16:21:09.246-0500 	going into namespace [test.test]
2013-12-13T16:21:09.246-0500 dev: lastError==0 won't report:Creating collection test.test failed. Errmsg: no such cmd: capped
assertion: 15936 Creating collection test.test failed. Errmsg: no such cmd: capped
 
I don't know the root cause of this issue.  The question is whether this is something we need to backport (it works in 2.5.4).



 Comments   
Comment by Githook User [ 12/Feb/14 ]

Author:

{u'username': u'Zarkantho', u'name': u'Shaun Verch', u'email': u'shaun.verch@10gen.com'}

Message: SERVER-12092 Uncommented no longer failing test
Branch: master
https://github.com/mongodb/mongo/commit/e42c285567ffae3d562b9743bf073cf25624b279

Comment by Githook User [ 20/Dec/13 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-12092 Fix mongorestore warning that collection options changed

Variable "obj" being shadowed by declaration in containing scope.

Conflicts:

src/mongo/tools/restore.cpp
Branch: v2.4
https://github.com/mongodb/mongo/commit/dc1e3b1a0c16b08aa887ec60a3031ec05ad8431f

Comment by Githook User [ 20/Dec/13 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-12092 mongorestore should generate valid "create" command obj

Fixes an issue where fields in the "create" command object generated
by mongorestore could be out of order, such that an invalid command
would be run instead of "create".

Conflicts:

src/mongo/tools/restore.cpp
Branch: v2.4
https://github.com/mongodb/mongo/commit/815bb1efe0f5c3f8ae6c5f0eccf6cf96e4936ab8

Comment by Githook User [ 20/Dec/13 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-12092 Fix mongorestore warning that collection options changed

Variable "obj" being shadowed by declaration in containing scope.
Branch: master
https://github.com/mongodb/mongo/commit/3bb6baf7ac2c8e8a90f09fcba40aa436c6195285

Comment by Githook User [ 18/Dec/13 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-12092 Fix test that erroneously expects non-null coll. options
Branch: master
https://github.com/mongodb/mongo/commit/0155cd5bc6fb62300ec26130c0f8c16a8a669c82

Comment by Githook User [ 17/Dec/13 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-12092 mongorestore should generate valid "create" command obj

Fixes an issue where fields in the "create" command object generated
by mongorestore could be out of order, such that an invalid command
would be run instead of "create".
Branch: master
https://github.com/mongodb/mongo/commit/2bff949748dc4b9eb63225abbe08907d1fe1d444

Comment by Githook User [ 17/Dec/13 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-12092 "create" command shouldn't set "create" collection option
Branch: master
https://github.com/mongodb/mongo/commit/35bf3e7660a1ff6ad7d3fa6024604f01029b401b

Comment by Daniel Pasette (Inactive) [ 16/Dec/13 ]

should fix mongorestore.

Comment by J Rassi [ 14/Dec/13 ]

Fix plan:

  • mongorestore (2.6.0?) should re-order options to generate a valid "create" command object. This should be backported to 2.4.
  • mongod (2.6.0?) should no longer put "create" in options when running the create command (it is an optional field anyway). It doesn't seem as urgent to me to backport this.
Generated at Thu Feb 08 03:27:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.