[SERVER-62890] replace_one with upsert=True incorrectly matches documents Created: 22/Jan/22  Updated: 22/Jun/22  Resolved: 12/May/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.3, 5.0.5
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Currn Hyde Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: Bug, filter, replace_one
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

I'm using mongo in docker and tested with mongo:5.0.5 and mongo:5.0.3 both have this issue.

Using pymongo version 4.0

Python 3.10.1

Download the github repo I've created to demonstrate this problem:

https://github.com/M15terHyde/mongodb_replace_one_issue 

It contains two docker containers. One for mongo, the other for the (trimmed down) api which uses replace_one to do Create/Update with non-duplication. It also includes a (trimmed down) test script for that api endpoint. Only the functions necessary to demonstrate this issue have been left in these files.

Participants:

 Description   

Using replace_one with upsert=True to insert new docs or update existing without duplication incorrectly matches existing docs and replaces rather than upserting new doc.

This is my second time encountering this false positive/false-negative filter bug in replace_one. This first time is documented here: https://stackoverflow.com/questions/70560121/mongodb-replace-one-with-upsert-true-upserts-when-it-should-replace-how-to-fix

After creating a collection for products and creating/ensuring a geoindex for it to search on during startup, the program then exposes several api endpoints one of these is for adding documents. We don't want duplicate products. Three fields are used to define a unique/distinct product: The foreignAPIAuthority, locationID, and productID. If a new product matches all these when added then it is considered a duplicate and should only update the existing one (replace), not be added as its own product (upsert).

I've previously encountered a false negative filter matching issue with mongo but it mysteriously went away. Now it's come back as a false positive filter matching issue. A product defined by filter: {foreignAPIAuthority;'TEST', locationID:'1', productID:'1234'}

is being matched by and replaced by a product filter which is different and ought to be upserted: {foreignAPIAuthority;'TEST', locationID:'1', productID:'1235'}



 Comments   
Comment by Eric Sedor [ 08/Mar/22 ]

Hi curranhydespam@gmail.com and thank you so much for your patience. We're grateful for the pared down application you provided but are not able to reproduce this using internal methods.

I would like to ask you to discuss your issue with our community by posting on the MongoDB Developer Community Forums. But I have a quick initial idea: I am suspicious of the output that asserts product4 inserted objectID: 61ec70e2e0ea2044bc4bd7cf (Incorrectly updates product3 (aka product1) instead of product2) because that assertion seems to rely on this query initiated by the endpoint. I think this query should include productID, but an even stronger reproduction of an issue would verify the incorrect contents of documents, rather than checking that a document exists for the given filter criteria.

Hopefully this makes sense. Our community on the forums can help you troubleshoot application behavior and, if there is an issue here, work towards an even more minimal reproduction.

Thank you!

Generated at Thu Feb 08 05:56:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.