[SERVER-1311] mongodump/mongorestore loses information such as capped params Created: 28/Jun/10 Updated: 12/Jul/16 Resolved: 12/Dec/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Tools |
| Affects Version/s: | 1.4.3 |
| Fix Version/s: | 2.1.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Roger Binns | Assignee: | Spencer Brody (Inactive) |
| Resolution: | Done | Votes: | 13 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Operating System: | ALL | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
Doing a dump, then a drop then a restore does not give you back what you had before. In particular meta-information about a collection such as it being capped and the capped size is not restored. I believe that paddingFactor is also lost although that won't be a functional difference like a collection being capped or not. |
| Comments |
| Comment by auto [ 12/Dec/11 ] |
|
Author: {u'login': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@10gen.com'}Message: Log a warning when restoring with different options than existing collection. |
| Comment by auto [ 07/Dec/11 ] |
|
Author: {u'login': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@10gen.com'}Message: Add test for dumping and restoring indexes and collection metadata. |
| Comment by auto [ 07/Dec/11 ] |
|
Author: {u'login': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@10gen.com'}Message: Make mongorestore restore collection options and indexes from the metadata file. |
| Comment by auto [ 07/Dec/11 ] |
|
Author: {u'login': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@10gen.com'}Message: Make mongodump create a metadata file with collection options and indexes to be used by mongorestore. |
| Comment by Spencer Brody (Inactive) [ 03/Aug/11 ] |
|
If not using --drop check the system.namespaces collection and output a warning if the options don't match the options that were dumped. |
| Comment by Mathias Stearn [ 03/Aug/11 ] |
|
Decided solution: Create a collection_name.metadata.json file that has the following format: options is optional and will only be there if the options field exists in db.system.namespaces.findOne( {name:'db.collection'}) indexes is a list of the objects in db.system.indexes.find( {ns: 'db.collection'}). On restore, after inserting the data for the collection, update the ns field and reinsert all the index objects. Make sure mongorestore works even without a metadata.json file |
| Comment by Eliot Horowitz (Inactive) [ 01/Jun/11 ] |
|
This might be a dupe |
| Comment by Roger Binns [ 28/Jun/10 ] |
|
Actually it looks like I can do your solution instead, since I really don't want to hard code what attributes are normal/default for a collection. I was expecting system.namespaces to have a lot more crud in it. New algorithm:
|
| Comment by Roger Binns [ 28/Jun/10 ] |
|
Thanks. What I am going to implement is this:
That should tide me over until Mongo has the official fix in, working with current and future versions. BTW I would assume that having system.namespaces.bson would be really bad since older versions of mongorestore would try to restore it unilaterally. Using .json would avoid that issue. |
| Comment by Eliot Horowitz (Inactive) [ 28/Jun/10 ] |
|
I guess we could make it system.namespaces.json instead of system.namespaces.bson |
| Comment by Eliot Horowitz (Inactive) [ 28/Jun/10 ] |
|
I think we'll probably do something different. Most likely we'll just write a system.namespaces.bson since that's what the name already is. Then we'll just be smart about applying it. |
| Comment by Roger Binns [ 28/Jun/10 ] |
|
Is my proposed solution acceptable? I need to implement something today, ideally the same or compatible with the eventual official fix. |
| Comment by Roger Binns [ 28/Jun/10 ] |
|
I'd like to propose the following solution:
This will be forwards and backwards compatible with existing dumps and in the vast majority of cases the dumps will be identical. Reasoning:
|