[SERVER-59119] Intermittent Duplicate key exception on upserting an existing document Created: 05/Aug/21  Updated: 18/Oct/21  Resolved: 18/Oct/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Rahul Jangra Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

I am facing a duplicate key exception intermittently when trying to upsert an existing document.

  • Exception

 

E11000 duplicate key error collection: userprofiledb.userProfile index: _id_ dup key: { : "9441959569" }; nested exception is com.mongodb.MongoWriteException: E11000 duplicate key error collection: userprofiledb.userProfile index: _id_ dup key: { : "9441959569" }

 

  • Indexes on the document:

 

[
    {
        "v" : 1,
        "key" : {
            "_id" : 1
        },
        "name" : "_id_",
        "ns" : "userprofiledb.userProfile"
    }
]

 

 

  • Query is like so: (Its an upsert query with replace document, the "u" part of the query doesnt contain _id key)

 

{
   "update":"userProfile",
   "ordered":true,
   "updates":[
      {
         "q":{
            "_id":"9711665996"
         },
         "u":{
            "key1":"value1",
            "key2":"value2",
            "key3":[
               "value3a",
               "value3b",
               "value3c"
            ]
         },
         "upsert":true
      }
   ]
}

 

 

Is there any other possible sequence of events where we can get duplicate key exception.

 

 

 



 Comments   
Comment by Eric Sedor [ 18/Oct/21 ]

rahul.jangra@airtel.com I'm going to close this ticket, but again, if you see issues occurring on a currently supported version (4.0+ at this time), we would be happy to reopen the ticket or address a new ticket.

Comment by Eric Sedor [ 24/Sep/21 ]

Hi rahul.jangra@airtel.com,

I apologize for the delay. MongoDB 3.6 reached end of life in April of 2021. Would you be able to move to a currently supported version (MongoDB 4.0+), and see if this issue reoccurs? If so, that would allow us to investigate it here.

Gratefully,
Eric

Comment by Rahul Jangra [ 11/Aug/21 ]

Hi Eric,

I am using mongodb 3.6

There is a small difference between the scenario mentioned in the linked tickets and the scenario that I am asking above.
Upsert can do both insert and update a document. The scenarios mentioned in linked ticket happens when upsert tries to "insert" the document, But in my scenario, the document already exists and upsert is trying to "update" the document.

So my question is ->

  1. Is it possible to get duplicateKeyException when upsert is trying to "update" an existing document ?
  2. Is it possible that mongo treats an "update" operation as an "insert" operation for an existing document if mongo is not able to find an existing document in the query phase(maybe because the search query was very slow around 1000ms)

 

 

Comment by Eric Sedor [ 10/Aug/21 ]

Hi rahul.jangra@airtel.com, can you clarify what version of MongoDB you are running? I ask because SERVER-14322/SERVER-37124 (against which the ticket you mentioned, SERVER-21006, were deduplicated) seems to cover this exact case and was addressed in MongoDB 4.2.

Comment by Rahul Jangra [ 06/Aug/21 ]

UPDATE: seeing a lot of slow queries on this collection recently, do you think "upsert" will try to "insert" if the query fails or query timesout due to a slow query ?

Generated at Thu Feb 08 05:46:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.