Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2625

Catch multiple collections that differ only by capitalization

XMLWordPrintableJSON

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • 101.0.0
    • Affects Version/s: None
    • Component/s: mongodump
    • None
    • 1,642

      For MacOS and Windows that have case-insensitive filenames, mongodump will corrupt a BSON file when there is both an abc.MyColl and abc.mycoll namespace. It seems to write both into the same file on disk.

      Example

      mongodump --ssl --username myuser --password mypass --authenticationDatabase admin --db MYDB --out d:/Backups/MYDB_20200622_Backup.dmp
2020-06-22T07:59:46.658+0100 writing MYDB.assetItem to
2020-06-22T07:59:46.661+0100 writing MYDB.assetItemChangeLog to
2020-06-22T07:59:46.663+0100 writing MYDB.asset to
2020-06-22T07:59:46.664+0100 writing MYDB.assetitem to
2020-06-22T07:59:46.849+0100 done dumping MYDB.assetitem (723 documents)
2020-06-22T07:59:46.851+0100 writing MYDB.assetChangeLog to
2020-06-22T07:59:46.885+0100 done dumping MYDB.assetChangeLog (23 documents)
2020-06-22T07:59:46.886+0100 writing MYDB.Environment to
2020-06-22T07:59:46.918+0100 done dumping MYDB.Environment (1 document)
2020-06-22T07:59:46.920+0100 writing MYDB.sequence to
2020-06-22T07:59:46.953+0100 done dumping MYDB.sequence (1 document)
2020-06-22T07:59:49.200+0100 done dumping MYDB.asset (12741 documents)
2020-06-22T07:59:49.517+0100 [########................] MYDB.assetItemChangeLog 7371/21379 (34.5%)
2020-06-22T07:59:49.520+0100 [#####...................] MYDB.assetItem 61836/266883 (23.2%)
2020-06-22T07:59:49.522+0100
2020-06-22T07:59:52.399+0100 [#################.......] MYDB.assetItemChangeLog 15463/21379 (72.3%)
2020-06-22T07:59:52.400+0100 [#########...............] MYDB.assetItem 105952/266883 (39.7%)
2020-06-22T07:59:52.408+0100
2020-06-22T07:59:55.380+0100 [########################] MYDB.assetItemChangeLog 21379/21379 (100.0%)
2020-06-22T07:59:55.383+0100 [##############..........] MYDB.assetItem 157672/266883 (59.1%)
2020-06-22T07:59:55.399+0100 done dumping MYDB.assetItemChangeLog (21379 documents)
2020-06-22T07:59:58.357+0100 [##################......] MYDB.assetItem 208478/266883 (78.1%)
2020-06-22T08:00:00.945+0100 [########################] MYDB.assetItem 266883/266883 (100.0%)
2020-06-22T08:00:01.189+0100 done dumping MYDB.assetItem (266883 documents)

mongorestore --username myuser --password mypass --authenticationDatabase admin d:/Backups/MYDB_20200622_Backup.dmp
2020-06-22T08:04:04.186+0100 preparing collections to restore from
2020-06-22T08:04:04.227+0100 restoring MYDB.assetItem from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\assetItem.bson
2020-06-22T08:04:04.232+0100 reading metadata for MYDB.assetItemChangeLog from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\assetItemChangeLog.metadata.json
2020-06-22T08:04:04.249+0100 reading metadata for MYDB.assetChangeLog from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\assetChangeLog.metadata.json
2020-06-22T08:04:04.265+0100 reading metadata for MYDB.asset from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\asset.metadata.json
2020-06-22T08:04:04.337+0100 restoring MYDB.assetItemChangeLog from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\assetItemChangeLog.bson
2020-06-22T08:04:04.387+0100 restoring MYDB.assetChangeLog from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\assetChangeLog.bson
2020-06-22T08:04:04.433+0100 restoring MYDB.asset from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\asset.bson
2020-06-22T08:04:04.462+0100 no indexes to restore
2020-06-22T08:04:04.470+0100 finished restoring MYDB.assetChangeLog (23 documents)
2020-06-22T08:04:04.550+0100 reading metadata for MYDB.Environment from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\Environment.metadata.json
2020-06-22T08:04:04.612+0100 restoring MYDB.Environment from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\Environment.bson
2020-06-22T08:04:04.669+0100 no indexes to restore
2020-06-22T08:04:04.699+0100 finished restoring MYDB.Environment (1 document)
2020-06-22T08:04:04.701+0100 reading metadata for MYDB.sequence from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\sequence.metadata.json
2020-06-22T08:04:04.757+0100 restoring MYDB.sequence from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\sequence.bson
2020-06-22T08:04:04.803+0100 error: Client Error: bad object in message: invalid bson type in object with unknown _id
2020-06-22T08:04:04.814+0100 Failed: MYDB.assetItem: error restoring from d:\Backups\MYDB_20200622_Backup.dmp\MYDB\assetItem.bson: reading bson input: invalid BSONSize: 1700003840 bytes

      Possible solutions

      1. Detect that it's dumping to a case-insensitive FS and use a different filename for the duplicate collection, e.g. mycoll.bson and mycoll-2.bson

      2. Detect the duplicate name and simply refuse to dump it, and output an error message.

            Assignee:
            Unassigned Unassigned
            Reporter:
            nicholas.cottrell@mongodb.com Nic Cottrell
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: