[SERVER-7170] Upserts using unidexed query can result in duplicate documents Created: 26/Sep/12  Updated: 09/Jul/16  Resolved: 27/Dec/12

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 2.2.0, 2.3.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Randolph Tan Assignee: Unassigned
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

storm 0.8


Attachments: File dup.js    
Issue Links:
Duplicate
is duplicated by SERVER-9423 Make findAndModify/update + upsert ==... Closed
Related
related to DOCS-861 Clarify behavior of findAndModify wit... Closed
is related to SERVER-4639 update yielding with upsert can dupli... Closed
is related to DOCS-1429 Clarify ramification of upsert in upd... Closed
Operating System: ALL
Participants:

 Description   

using strom, the threads(also in different process) run code:

myDB("mytable").update(DBObject("key1" -> new ObjectId(myKey), "key2" -> coKey), myUpdates, true, false, WriteConcern.SAFE)

and my storm log as following (node1 and node2 is different machine, the last two columns is using :System.currentTimeMillis(),Thread.currentThread() ):

on node1: storm-0.8.1/logs/worker-6711.log:2012-09-14 11:18:11 STDIO [INFO] 1347592691476 Thread[Thread-16,5,main]
on node2: storm-0.8.1/logs/worker-6705.log:2012-09-14 11:18:11 STDIO [INFO] 1347592691456 Thread[Thread-24,5,main]

there is no record in mytable meets the query

DBObject("key1" -> new ObjectId(myKey), "key2" -> coKey)

before they run, and after they run, there will exist exactly two same records, which I think is quite the opposite of what I know of the meaning of upsert = true.

from the logs:

Fri Sep 21 13:18:57 [initandlisten] MongoDB starting : pid=32671 port=27017 dbpath=/home/myuser/mydb 64-bit host=mymachine
Fri Sep 21 13:18:57 [initandlisten] db version v2.2.0, pdfile version 4.5
 
....
 Fri Sep 21 13:35:29 [conn196] update mydb.mytable query: { key1: "k12306", key2: "k86" } update: { field1: "f1", field2: "f2" } nscanned:3341 nupdated:1 upsert:1 keyUpdates:0 numYields: 26 locks(micros) w:7033 333ms
...
 Fri Sep 21 13:35:29 [conn192] update mydb.mytable query: { key1: "k12306", key2: "k86" } update: { field1: "f1", field2: "f2" } nscanned:3343 nupdated:1 upsert:1 keyUpdates:0 numYields: 26 locks(micros) w:6906 335ms
....

we can see that the commands which cause duplicated records is shown above.

other conditions is:
a) when mytable is with index on db.mytable.ensureIndex(

{"key1":1, "key2":1}

), it works ok, no duplicated record generated.
b) when mytable is without any index, which will cause every mongo command cost much longer time, bang~, the duplicated records appeared..

original thread:
https://groups.google.com/forum/?fromgroups=#!topic/mongodb-user/MBafvLXkTYw



 Comments   
Comment by Daniel Pasette (Inactive) [ 27/Dec/12 ]

This is known behavior. upserts are not isolated without using the $atomic operator.

Comment by Randolph Tan [ 14/Oct/12 ]

Fully reproducible in 2.3 (git version 352bd601c0fdaa24edf0357975f20abb0dddfb8c). Attaching test code - dup.js (might need to adjust numbers accordingly depending on what kind of machine it was run, see comments in test for more details).

Here's the log from the failed case:

Sun Oct 14 19:28:53 [conn10] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31251 locks(micros) w:17062089 65716ms
Sun Oct 14 19:28:53 [conn8] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31249 locks(micros) w:20103778 65716ms
Sun Oct 14 19:28:53 [conn9] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31253 locks(micros) w:17682175 65720ms
Sun Oct 14 19:28:53 [conn11] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31245 locks(micros) w:19936652 65727ms
Sun Oct 14 19:28:53 [conn6] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31249 locks(micros) w:17218732 65724ms
Sun Oct 14 19:28:53 [conn5] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31253 locks(micros) w:16424249 65723ms
Sun Oct 14 19:28:53 [conn7] update dup.upsert query: { x: 1.0, y: 2.0 } update: { x: 1.0, y: 2.0, f: "this is it" } nscanned:4000000 nupdated:1 upsert:1 keyUpdates:0 numYields: 31257 locks(micros) w:16068438 65728ms

Generated at Thu Feb 08 03:13:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.