Uploaded image for project: 'MongoDB ETL Tools'
  1. MongoDB ETL Tools
  2. TOOLS-894

mongoimport --upsert --type json with _id being an object does not work most of the times

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.5
    • Fix Version/s: 3.1.8, 3.0.7
    • Component/s: None
    • Labels:
      None
    • Environment:
      OS X 10.10.5 | mongodb 3.0.5
    • Backport Requested:
      v3.0
    • Sprint:
      Kernel Tools 9 09/18/15

      Description

      I have problems importing data into mongodb with. I have a `test.json` file like this:

          {"_id":{"s":{"$numberLong":"38851448"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38853194"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38760498"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"39099662"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38855558"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38760487"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38760488"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"39099663"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38851450"},"a":5},"someKey":"someValue"}
          {"_id":{"s":{"$numberLong":"38853546"},"a":5},"someKey":"someValue"}
      

      I try to import it with following command:

      mongoimport --type json --db test --collection coll --file test.json --upsert

      The import fails almost always with the same error message:

          2015-08-27T17:02:15.510+0200    error inserting documents: The _id field cannot be changed from {_id: { s: 38851448, a: 5 }} to {_id: { a: 5, s: 38851448 }}.
          2015-08-27T17:02:15.511+0200    error inserting documents: The _id field cannot be changed from {_id: { a: 5, s: 38760487 }} to {_id: { s: 38760487, a: 5 }}.
      

      The importer seems to change the order of the properties for `_id` for some of the documents.

      I event tried to change the order of `s` and `a` in `_id`, but the problem stays the same.

      If I trie to run the exact same import command with exactly the same datafile multiple times the documents with errors change and one time it imported all rows as expected, but just one time.

      For completeness: I'm using mongo 3.0.5 on Mac OS X 10.10.5 installed by homebrew

        Attachments

        1. OGh7u.png
          OGh7u.png
          530 kB
        2. test.json
          0.7 kB

          Issue Links

            Activity

              People

              • Assignee:
                kyle.erf Kyle Erf
                Reporter:
                jfahnenbruck Jens Fahnenbruck
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: