[SERVER-21434] FindAndModify query unexpectedly returns null when using multiple readers Created: 12/Nov/15  Updated: 19/Jan/16  Resolved: 13/Nov/15

Status: Closed
Project: Core Server
Component/s: Querying, Write Ops
Affects Version/s: 3.2.0-rc2
Fix Version/s: 3.2.0-rc3

Type: Bug Priority: Major - P3
Reporter: Nick Judson Assignee: Charlie Swanson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File Threading_Test.7z    
Issue Links:
Duplicate
Related
related to SERVER-22002 Do not retry findAndModify operations... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Please use the attached sample

Sprint: QuInt C (11/23/15)
Participants:

 Description   

If a findAndModify request specifies a sort, we use a limit of 1 to reduce the memory requirements and use a top-k sort. With such a request, if a WriteConflictException is encountered while trying to update or delete a matching document, the operation will return no matches. This happens even if there are other documents that match the query. We should retry the entire operation in this scenario to avoid not returning a document when there were results that matched the query.

Original Description

Using MongoDb with WT to create a simple message queue, I used one writer to add some documents with a state of 'Q'. I then use 50 readers (threads) to consume these documents, with each thread using a FindAndModify query to atomically update the state of a document to 'in process' and return it for processing. At the end of processing the document state is updated to 'processed'.

The query (using the c# driver) looks as follows:

var message = await m_QueueMessages.FindOneAndUpdateAsync(Builders<MongoMessage>.Filter.Where(m => m.QueueType == QueueType.Queued),
                                                                Builders<MongoMessage>.Update.Set(m => m.QueueType, QueueType.InProcess),
                                                                new FindOneAndUpdateOptions<MongoMessage> { Sort = Builders<MongoMessage>.Sort.Ascending(m => m.ObjectId), IsUpsert = false, MaxTime = TimeSpan.FromSeconds(5) },
                                                                CancellationToken.None);

This query often returns null, even though there are documents matching the filter. Removal of the sort apparently fixes this. I have confirmed with Craig Wilson (MongoDb c# driver) that this is not a driver issue.



 Comments   
Comment by Githook User [ 13/Nov/15 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'charlie.swanson@mongodb.com'}

Message: SERVER-21434 Retry a findAndModify if it conflicts on an update or delete
Branch: master
https://github.com/mongodb/mongo/commit/cd766108b26e7e272f3c367f9de8767399feeadd

Comment by Charlie Swanson [ 12/Nov/15 ]

Updated the description to match our understanding of the problem.

Comment by Nick Judson [ 12/Nov/15 ]

Great - thanks.

Comment by David Storch [ 12/Nov/15 ]

nick@innsenroute.com, thanks for reporting this. I think we understand the issue from the details you already provided. We are working on a repro script and a fix, so please continue to watch this ticket for updates.

Comment by Nick Judson [ 12/Nov/15 ]

This may be a duplicate of SERVER-21227.

Generated at Thu Feb 08 03:57:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.