Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-3082

mongorestore fails to copy/clone an existing database to a new one

    • Type: Icon: Question Question
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: mongorestore
    • Environment:
    • Needed
    • Hide
      The documentation on mongodump and mongorestore should clearly specify that at the exact place how quotes should be used (or should they not be used at all) while passing the values of --nsFrom and --nsTo arguments in the mongorestore command. While using single quotes (as written in the document) executes mongorestore properly in a Mac computer, but it fails in doing so in a Windows PC. To run the same in a Windows PC successfully, the values for --nsFrom and --nsTo need to be passed with either double quotes or no quotes at all. The correct procedure should be updated in the documentation properly.
      Show
      The documentation on mongodump and mongorestore should clearly specify that at the exact place how quotes should be used (or should they not be used at all) while passing the values of --nsFrom and --nsTo arguments in the mongorestore command. While using single quotes (as written in the document) executes mongorestore properly in a Mac computer, but it fails in doing so in a Windows PC. To run the same in a Windows PC successfully, the values for --nsFrom and --nsTo need to be passed with either double quotes or no quotes at all. The correct procedure should be updated in the documentation properly.
    • v5.0

      Problem Statement/Rationale

      In my Windows 10 PC using MongoDB 5.0.7 Community edition, while trying to copy an existing database to a new one using mongodump and mongorestore commands as mentioned in the documentation of MongoDB here, although mongodump is executing and creating the dump file properly, the mongorestore command fails every time generating a duplicate key error.

       

      Surprisingly, it seems to restore the collection Products in mydb database instead of newdb. As mydb already contains the products collection having rows of documents with unique ids, it’s creating a clash generating the duplicate key error. Somehow, mongo seems to be ignoring the new database argument altogether.

      Steps to Reproduce

      1. Create a database named mydb and create a collection called Products containing three documents in it.
        use mydb;
        
        db.createCollection("Products");
        
        db.Products.insert( { "name" : "Cake", "price" : 35 } );
        db.Products.insert( { "name" : "Cheese Pizza", "price" : 105 } );
        db.Products.insert( { "name" : "Capsicum Pizza", "price" : 99 } );


        # Dump the mydb database to an archive in the PC.
        mongodump --archive="mongodump-mydb" --db=mydb 


        # Now restore the mydb database from the dump to a new database named newdb.
        mongorestore --archive="mongodump-mydb" --nsFrom='mydb.*' --nsTo='newdb.*' 


        h3. Expected Results

        As the mongodump command is executing and creating the dump file properly on the disc, the mongorestore command should have copied the contents of the mydb database into a new database named newdb.

        Actual Results

      Instead of copying the contents of mydb database to a new one named newdb, MongoDB is generating a duplicate key error.

      2022-04-17T19:09:04.136+0530    preparing collections to restore from
      2022-04-17T19:09:04.146+0530    reading metadata for mydb.products from archive 'mongodump-mydb'
      2022-04-17T19:09:04.150+0530    restoring to existing collection mydb.products without dropping
      2022-04-17T19:09:04.157+0530    restoring mydb.products from archive 'mongodump-mydb'
      2022-04-17T19:09:04.209+0530    continuing through error: E11000 duplicate key error collection: mydb.products index: _id_ dup key: { _id: ObjectId('625974fe025923931b6e5e7f') }
      2022-04-17T19:09:04.211+0530    continuing through error: E11000 duplicate key error collection: mydb.products index: _id_ dup key: { _id: ObjectId('62597508025923931b6e5e80') }
      2022-04-17T19:09:04.211+0530    continuing through error: E11000 duplicate key error collection: mydb.products index: _id_ dup key: { _id: ObjectId('62597508025923931b6e5e81') }
      2022-04-17T19:09:04.211+0530    finished restoring mydb.products (0 documents, 3 failures)
      2022-04-17T19:09:04.211+0530    no indexes to restore for collection mydb.products
      2022-04-17T19:09:04.211+0530    0 document(s) restored successfully. 3 document(s) failed to restore. 

       

      Additional Notes

      While delving deep into the problem, I found the reason. The problem lies in using single quotes while passing the values of -nsFrom and -nsTo arguments of the mongorestore command. While doing so, in some OS environments (especially Windows), mongo keeps on restoring the dump to the existing database instead of the new one, thus creating a clash between the existing data and the data being restored generating the duplicate key error. Instead, either using double quotes or not using any quote at all to pass the values of --nsFrom and --nsTo arguments solves the problem, thus executing mongorestore properly and copying the contents of mydb into newdb successfully.

       

      Surprisingly, the mongorestore command works perfectly with single quotes while passing the values of --nsFrom and --nsTo arguments in a Mac computer, making it even more confusing for Windows users.

       

      This seems to be a bug in MongoDB Server system or its syntax and needs to be rectified in the next update. The same needs to be updated in the documentation also. If there's some OS-specific syntax issue involved, the documentation on mongodump and mongorestore should clearly specify that at the exact place here []|https://www.mongodb.com/docs/database-tools/mongodump/?&_ga=2.234397854.1469458796.1649914316-1657763159.1648793391#copy-clone-a-database].]

      Otherwise, it ends up being extremely puzzling for readers, especially beginners, who would definitely try to implement the code in the documentation for testing and development.

            Assignee:
            Unassigned Unassigned
            Reporter:
            priyamtheone@hotmail.com Priyam Ganguly
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: