Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-14322

Retry on predicate unique index violations of update + upsert -> insert when possible

XMLWordPrintableJSON

    • Type: Icon: Improvement Improvement
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Write Ops
    • Labels:
    • Query

      Issue Status as of Jan 29, 2019

      ISSUE DESCRIPTION AND IMPACT

      During an update with upsert:true option, two (or more) threads may attempt an upsert operation using the same query predicate and, upon not finding a match, the threads will attempt to insert a new document. Both inserts will (and should) succeed, unless the second causes a unique constraint violation. This MongoDB behavior is documented here.

      It is expected that the client will take appropriate action upon detection of such constraint violation. Appropriate action may vary depending on individual application requirements.

      IMPROVEMENT DESCRIPTION

      As part of SERVER-37124, fixed in versions 4.1.6 and later, the server will automatically retry performing the upsert in cases where it is detectably safe to do so. It is safe for the server to retry when the following conditions hold:

      • The operation has the upsert:true flag.
      • The operation is not a multi-update.
      • The upsert failed with the DuplicateKey error code due to a unique index with key pattern kp.
      • The match condition is either a single equality predicate, or a logical AND of equality predicates.
      • The set of fields over which there is an equality predicate equals the set of fields in the index key pattern kp.
      • The fields in the match condition are not mutated by the update modifiers.

      The following table contains examples of upsert operations that the server can automatically retry on DuplicateKey error, and examples where the server cannot automatically retry:

         kp                                                              update                     server will retry
      {_id: 1} db.coll.update({_id:5}, {$set:{a:5}}, {upsert:true})     yes
      {a:1} db.coll.update({a:{$in:[5]}}, {$set:{b:5}}, {upsert:true})       yes 
      {a:1,b:1} db.coll.update({a:5,b:7},{$set:{c:5}}, {upsert:true})       yes
      {a:1,b:1} db.coll.update({a:5,b:7},{$set:{a:7,c:5}},{upsert:true})       no, since a is in kp and is mutated 
      {a:1} db.coll.update({_id:5},{$set:{b:6}}, {upsert:true})       no, since the predicate does not contain a 
      {_id:1} db.coll.update({_id:5,v:6},{$set:{a:5,b:6}}, {upsert:true})       no, since v is not in kp 
      {a:1} db.coll.update({a:{$lt:5}},{$set:{b:5}}, {upsert:true})      no, since the predicate on a is not an equality 

      This ticket will now be resolved as duplicate of SERVER-37124 since we don't believe there is further work that we can do on the server and the application must correctly determine whether the operation that failed can be safely retried again or not.

      VERSIONS
      This improvement in SERVER-37124 will be included in the 4.2 production release

      Original description

      It is possible that two updates come in with upsert:true, resulting in neither finding a document and both inserting new documents which conflict on unique index violations of the query predicate. In this case it is possible for the server to retry any failed updates.

      Currently clients can retry the update to get the same behavior which is expected from the server.

      This affects both updates and findAndModify. See SERVER-10350 for slightly more background.

            Votes:
            117 Vote for this issue
            Watchers:
            144 Start watching this issue

              Created:
              Updated:
              Resolved: