Uploaded image for project: 'MongoDB ETL Tools'
  1. MongoDB ETL Tools
  2. TOOLS-54

mongoimport doesn't show error messages for duplicate records

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: Legacy C++ Implementation
    • Fix Version/s: 2.8.0-rc1
    • Component/s: mongoimport
    • Labels:
      None
    • Environment:
      Linux 2.6.35-23-generic #37-Ubuntu SMP Fri Nov 5 19:16:29 UTC 2010 x86_64 GNU/Linux

      Description

      I found strange behavior. I found "duplicate record" errors from mongoimport are printed to the output correctly when I use --dbpath , but if I connect to the database via mongo daemon then I don't see those errors. Is it possible to have get those errors in same way as it appears with --dbpath . Actually, I think behavior should be the same for both ways, no matter how mongoimport connects to mongodb.

      saulius@saulius:~/temp/mong$ cat data.csv
      attribute1, attribute2
      a,a
      a,a
      b,b
      saulius@saulius:~/temp/mong$ mongod --version
      db version v1.6.4, pdfile version 4.5
      Wed Nov 3 19:06:55 git version:
      4f5c02f8d92ff213b71b88f5eb643b7f62b50abc
      saulius@saulius:~/temp/mong$ mongo
      MongoDB shell version: 1.6.4
      connecting to: test
      > use things;

      switched to db things
      > db.things.remove();
      > db.things.ensureIndex(

      {attribute1: 1, attribute2: 1}

      ,

      {unique: true}

      );
      > exit

      bye
      saulius@saulius:~/temp/mong$ sudo mongoimport --file data.csv --db
      things --collection things --type csv --headerline
      connected to: 127.0.0.1
      imported 4 objects

      No errors!!! Importer just silently skipped duplicate records:

      saulius@saulius:~/temp/mong$ mongo
      MongoDB shell version: 1.6.4
      connecting to: test
      > use things;

      switched to db things
      > db.things.find();

      { "_id" : ObjectId("4cd19739cec6b527a76e5c0e"), "attribute1" : "a", "attribute2" : "a" } { "_id" : ObjectId("4cd19739cec6b527a76e5c10"), "attribute1" : "b", "attribute2" : "b" }

      BUT, when I use direct import to mongodb files:

      saulius@saulius:~/temp/mong$ mongoimport --file data.csv --db things
      --collection things --type csv --headerline -dbpath /var/lib/mongodb/
      Wed Nov 3 19:14:26 [tools] insert things.things exception 11000
      E11000 duplicate key error index: things.things.
      $attribute1_1_attribute2_1 dup key: { : "a", : "a" } 0ms
      imported 4 objects
      Wed Nov 3 19:14:26 dbexit:

      Wed Nov 3 19:14:26 [tools] shutdown: going to close listening
      sockets...
      Wed Nov 3 19:14:26 [tools] shutdown: going to flush oplog...
      Wed Nov 3 19:14:26 [tools] shutdown: going to close sockets...
      Wed Nov 3 19:14:26 [tools] shutdown: waiting for fs preallocator...
      Wed Nov 3 19:14:26 [tools] shutdown: closing all files...
      Wed Nov 3 19:14:26 closeAllFiles() finished

      Wed Nov 3 19:14:26 [tools] shutdown: removing fs lock...
      Wed Nov 3 19:14:26 dbexit: really exiting now

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                adinoyi.omuya Adinoyi Omuya
                Reporter:
                ian.whalen Ian Whalen
              • Votes:
                3 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: