[SERVER-10350] Atomicity of Upserts on multi field unique index Created: 26/Jul/13 Updated: 14/Sep/15 Resolved: 20/Jun/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Concurrency, Write Ops |
| Affects Version/s: | 2.4.4 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Tom Coupland | Assignee: | Unassigned |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Red Hat Enterprise Linux Server release 6.3 (Santiago) |
||
| Issue Links: |
|
||||||||||||||||||||||||
| Operating System: | Linux | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Description |
|
We're getting a handful of live exceptions resulting from concurrent upserts, with the same query, complaining about unique index violations. It was my understanding that the upsert process would be atomic, with respect to the decision to update or insert, and therefore the 'winner' of the obvious race condition would perform an insert and second place would update the new document. An example of the exception: com.mongodb.MongoException$DuplicateKey: { " , , The unique index in this case is a composite of four fields, one guid, the others are shorter strings all contained within a sub-document. The query element of the upsert consists solely of the unique index fields, the update is a document containing a single field $inc and a $set for half a dozen other fields. I would post an example, but i've been unable to recreate the issue with test code. The database is not under significant load (~100 updates per second, ~4% locked, rarely a single item in either write or read queues), the concurrent queries would be coming from separate clients, so not the same connection pool. The upserts are operating within an "ensureConnection" block (java driver), so we can check a variety of write concerns. It's my understanding this shouldn't be happening. Is there any extra information you need to help confirm this issue and track it down? |
| Comments |
| Comment by Daniel Pasette (Inactive) [ 30/Jul/13 ] |
|
I've added a documentation request to clarify the behavior here: |
| Comment by Tom Coupland [ 29/Jul/13 ] |
|
So in summery: concurrent upserts are not safe, you need a try/catch block around them to retry on a DuplicateKey exception. I think that discussion needs to be referenced here: docs.mongodb.org/manual/core/create/#crud-create-update. To be honest i think this issue should be explained in a red border so it's clear that this is not an atomic operation. Thanks for the fast feedback Andy. |
| Comment by Andy Schwerin [ 26/Jul/13 ] |
|
There are cases when the server could declare a winner, as the reporter suggests. In essence, those are the cases in which the server can describe a serial order over updates to the index region / index / index entry being modified. Determining when such a serialization actually exists or could exist requires further study. |
| Comment by Andy Schwerin [ 26/Jul/13 ] |
|
http://docs.mongodb.org/manual/reference/command/findAndModify/ The discussion near the bottom of this document starting with "When the findAndModify command includes the upsert: true" is relevant to update as well as findAndModify. A workaround for this behavior is described. |