[SERVER-13927] Copydb breaks on (some) capped collections Created: 13/May/14  Updated: 10/Dec/14  Resolved: 20/May/14

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 2.6.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Nick Stott Assignee: J Rassi
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-13976 Cloner needs to detect failure to cre... Closed
Related
related to SERVER-13968 Report invalid collection options in ... Closed
is related to SERVER-13737 CollectionOptions parser should skip ... Closed
Operating System: ALL
Steps To Reproduce:

1. in 2.4.x, create a capped collection as above
2. upgrade to 2.6
3. run a copydatabase onto a new 2.6 instance

Participants:

 Description   

capped collections in earlier versions of mongodb (2.4.x) can sometimes look like this:
{ "name" : "test-1.two", "options" :

{ "create" : "two", "size" : 1000000, "max" : "", "capped" : true, "autoIndexId" : true }

},
when a database with a capped collection like that is upgraded to 2.6.1, and then a copydatabase is run against it, the copy database succeeds, but the capped collections are not copied properly



 Comments   
Comment by Daniel Pasette (Inactive) [ 20/May/14 ]

Closing as duplicate of SERVER-13737 which will be fixed in server version 2.6.2. Thanks nstott for your help tracking down this issue.

Comment by J Rassi [ 17/May/14 ]

nstott: thanks again for reporting this. We've opened SERVER-13968 and SERVER-13976 as a result.

Could you let us know how these collections were created (perhaps the db.createCollection() shell helper, or the db.runCommand(...) shell helper, or a "create collection" helper from an official driver)? This will help us assess how widespread the issue is of having a collection with an invalid value of "max" in the collection options.

It's (unfortunately) not possible to manipulate the collection options directly to remove the invalid collection option. If you are able to take this node offline for a maintenance window, you can use dump/restore to remove the invalid collection option with the following:

  • Dump the collection contents: mongodump -d test-1 -c two -o dump/
  • Drop the collection (assuming the dump was successful): mongo --eval 'db.getSiblingDB("test-1").two.drop()'
  • Create the collection with the correct options: mongo --eval 'db.getSiblingDB("test-1").runCommand({create: "two", size: 1000000, capped: true})'
  • Restore the collection contents: mongorestore dump/

You can also use the convertToCapped command to restore cappedness to any copies already made of this collection.

Comment by Nick Stott [ 14/May/14 ]

yes, it loses that meta data on a resync as well
on a capped collection as shown above brought in from an earlier version, stopping the secondary, removing the local.*, starting the secondary will reproduce this behaviour.

on the primary I have:
{ "name" : "test-a.capped", "options" :

{ "create" : "capped", "size" : 100000, "max" : "", "capped" : true, "autoIndexId" : true }

}
{ "name" : "test-a.uncapped", "options" :

{ "create" : "uncapped", "size" : "", "max" : "", "capped" : false }

}

and on the secondary, I just have

{ "name" : "test-a.capped" } { "name" : "test-a.uncapped" }
Comment by Nick Stott [ 14/May/14 ]

I agree, it's the max:"" that seems to break things,
I wonder if that will break on a resync as well, that would be annoying.. I'll have a look at that

Comment by Thomas Rueckstiess [ 13/May/14 ]

The problem seems to be the non-numeric value in the "max" field. This wasn't validated in createCollection in 2.4.x (it is now, in 2.6.x). The capped-ness of the collection is not preserved in that case.

rassi@10gen.com, can you have a look at this?

Generated at Thu Feb 08 03:33:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.