Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-2519

Add the ability to force maintainInsertionOrder to false when mode is merge or upsert



    • Task
    • Status: Accepted
    • Major - P3
    • Resolution: Unresolved
    • None
    • None
    • mongoimport
    • None


      If the insertion mode is upsert or merge, the order is important. If you want to apply a series of changes to some documents using mongoimport, if you don't use ordered: true, you could end up applying the later changes first and the earlier changes last. This could result in stale or inconsistent data. This is why we currently force maintainInsertionOrder to be true when using upsert or merge.

      However it would be safe if a user can absolutely guarantee one of the following:

      1. Each file they import impacts documents only once. 
      2. They use merge mode and each source document has different fields (except the upsertFields) to every other source document that has matching upsertFields.

      A power user of mongoimport has case 1 and would like to force maintainInsertionOrder to be false so that they can reap the performance benefits.

      This is an inherently dangerous feature to expose. I trust a sophisticated user to know what they are doing, but there is a danger that more casual users could accidentally shoot themselves in the foot.

      I think there are two possible interfaces for this:

      1. Explicitly specifying --maintainInsertionOrder=false overrides the default behavior. This would require us fixing TOOLS-2504 so that we can tell the difference between a false zero value and a false being manually set.
      2. Add a hidden option such as --forceConcurrent that can override the default behavior. If the option is hidden, it could insulate more casual users from a potentially dangerous options, while still allowing power users to take advantage of it. 

      If we decide to move forward, we should probably do TOOLS-2510 at the same time since it's related.


        Issue Links



              Unassigned Unassigned
              tim.fogarty@mongodb.com Tim Fogarty
              1 Vote for this issue
              6 Start watching this issue