Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-62890

replace_one with upsert=True incorrectly matches documents

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: 5.0.3, 5.0.5
    • Component/s: None
    • ALL
    • Hide

      I'm using mongo in docker and tested with mongo:5.0.5 and mongo:5.0.3 both have this issue.

      Using pymongo version 4.0

      Python 3.10.1

      Download the github repo I've created to demonstrate this problem:

      https://github.com/M15terHyde/mongodb_replace_one_issue 

      It contains two docker containers. One for mongo, the other for the (trimmed down) api which uses replace_one to do Create/Update with non-duplication. It also includes a (trimmed down) test script for that api endpoint. Only the functions necessary to demonstrate this issue have been left in these files.

      Show
      I'm using mongo in docker and tested with mongo:5.0.5 and mongo:5.0.3 both have this issue. Using pymongo version 4.0 Python 3.10.1 Download the github repo I've created to demonstrate this problem: https://github.com/M15terHyde/mongodb_replace_one_issue   It contains two docker containers. One for mongo, the other for the (trimmed down) api which uses replace_one to do Create/Update with non-duplication. It also includes a (trimmed down) test script for that api endpoint. Only the functions necessary to demonstrate this issue have been left in these files.

      Using replace_one with upsert=True to insert new docs or update existing without duplication incorrectly matches existing docs and replaces rather than upserting new doc.

      This is my second time encountering this false positive/false-negative filter bug in replace_one. This first time is documented here: https://stackoverflow.com/questions/70560121/mongodb-replace-one-with-upsert-true-upserts-when-it-should-replace-how-to-fix

      After creating a collection for products and creating/ensuring a geoindex for it to search on during startup, the program then exposes several api endpoints one of these is for adding documents. We don't want duplicate products. Three fields are used to define a unique/distinct product: The foreignAPIAuthority, locationID, and productID. If a new product matches all these when added then it is considered a duplicate and should only update the existing one (replace), not be added as its own product (upsert).

      I've previously encountered a false negative filter matching issue with mongo but it mysteriously went away. Now it's come back as a false positive filter matching issue. A product defined by filter: {foreignAPIAuthority;'TEST', locationID:'1', productID:'1234'}

      is being matched by and replaced by a product filter which is different and ought to be upserted: {foreignAPIAuthority;'TEST', locationID:'1', productID:'1235'}

            Assignee:
            eric.sedor@mongodb.com Eric Sedor
            Reporter:
            curranhydespam@gmail.com Currn Hyde
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: