[SERVER-3560] CopyDB doesn't seem to work when copying from sharded db. Created: 09/Aug/11  Updated: 29/Feb/12  Resolved: 29/Sep/11

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 1.8.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Greg Studer Assignee: Greg Studer
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

at least ubuntu linux 10.10


Issue Links:
Duplicate
duplicates SERVER-1440 copyDatabase doesn't work in sharded ... Closed
Operating System: ALL
Participants:

 Description   

See http://www.google.com/url?sa=t&source=web&cd=1&ved=0CBwQFjAA&url=http%3A%2F%2Fgroups.google.com%2Fgroup%2Fmongodb-user%2Fbrowse_thread%2Fthread%2F3a831d2675b0ec12&rct=j&q=google%20groups%20copydatabase%20failed&ei=mltBTvzrF4Hq0gHO-Om2CQ&usg=AFQjCNEAdHW-X8VFfuZ52talNbQ6GIyp7g&cad=rja

Copying is incomplete or does not happen at all.

Workaround is to script something like:

var mongoTo = mongod; var mongoFrom = mongos;
var collections = mongoFrom.getDB( "test" ).getCollectionNames()
var dbTo = "test2"; var dbFrom = "test"
var collections = mongoFrom.getDB( dbFrom ).getCollectionNames()
for( var i = 0; i < collections.length; i++ ){
var cursor = mongoFrom.getCollection( dbFrom + "." + collections[i] ).find()
while( cursor.hasNext() )

{ mongoTo.getCollection( dbTo + "." + collections[i] ).insert( cursor.next() ) }

}



 Comments   
Comment by Scott Hernandez (Inactive) [ 03/Feb/12 ]

This is not a good idea to do in javascript in 1.8.x as there are lots of data types which won't be kept through the transformation. You will not be able to keep full data fidelity. A good example of a missing type, or one that will change is an integer.

It is much better to use a programming language with full data type support like java/.net/c/c++/python/ruby/etc.

Comment by Dominik Herbst [X] [ 03/Feb/12 ]

I got the same problem with mongos and mongo 2.0.2

I tried to copy a database from mongos on another machine.
Both have mongodb 2.0.2 and Debian GNU/Linux 6.0 in 64bit

db.copyDatabase('mydb', 'mydb', 'mongos.server')

Command hang on first collection after 81 copied objects.

I modified the script a bit to work and to give me progress of everything and it works. Just takes long to copy hundreds of GB and millions of objects:

var mongoTo = new Mongo; var mongoFrom = new Mongo('mongos.server');
var dbTo = "mydb"; var dbFrom = "mydb";

var collections = mongoFrom.getDB( dbFrom ).getCollectionNames();
printjson(collections);

var steps;
for( var i = 0; i < collections.length; i++ ){
var cursor = mongoFrom.getCollection( dbFrom + "." + collections[i] ).find();
var cnt=0;
var total=cursor.count();
steps=total>100?Math.round(total/100):100;
printjson(

{collection: collections[i], total:total, steps:steps}

);
while( cursor.hasNext() ) {
cnt++;
mongoTo.getCollection( dbTo + "." + collections[i] ).insert( cursor.next() );
if(cnt%steps==0) printjson(

{collection: collections[i], curr: cnt, total:total, perc:(cnt/total*100).toFixed(1)}

);
}
}

Generated at Thu Feb 08 03:03:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.