Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2159

mongoimport --upsertFields does not accept nested keys by dot notation from CSV

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.1.7
    • Affects Version/s: 4.0.4
    • Component/s: mongoimport
    • Labels:
    • Environment:
      Fedora 29, MongoDB Community Edition v4.0.4
    • 2

      I try to use mongoimport upsert mode to import a csv. Below is the content of file records.csv to be imported:



      The mongoimport --upsertFields docs says "To ensure adequate performance, indexes should exist for this field or field". So I created a unique index for the field level1.level2.key1:

      db.records.createIndex({ level1.level2.key1: 1 }, { unique: true })


      Next, I executed mongoimport with upsert mode:

      mongoimport --db test --collection records --type csv --headerline --columnsHaveTypes --mode upsert --upsertFields level1.level2.key1 --file records.csv


      Suppose the records collection is empty in the beginning. Running the above mongoimport command for the first time successfully inserted a document. But when the same command is run for the second time, surprisingly an insertion error was returned:

      error inserting documents: E11000 duplicate key error collection: test.records index: level1.level2.key1_1 dup key: { : "34532" }


      Looking at this error, it seems that the "level1.level2.key1" field specified by --upsertFields was not used to match against existing documents and thus the upsert mode degenerated to an insert mode (i.e. the document in the csv file should update the existing document in the DB, but it was treated as a new document to be inserted), which certainly caused "error inserting documents: E11000 duplicate key error collection."


      I started to doubt that this could be due to the matching key is nested. So I changed level1.level2.key1 to simply just key1, then everything went well.

            david.golden@mongodb.com David Golden
            wen7qi Jacky Lee
            0 Vote for this issue
            4 Start watching this issue