[SERVER-55554] Fix out of order array element updates for operators like $min Created: 26/Mar/21  Updated: 19/May/21  Resolved: 19/May/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.4.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Arun Banala Assignee: James Wahlin
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-50778 "Cannot apply $bit" when $bit update ... Closed
Related
is related to SERVER-50778 "Cannot apply $bit" when $bit update ... Closed
Operating System: ALL
Sprint: Query Optimization 2021-05-31
Participants:

 Description   

This issue was fixed on master by SERVER-50778. In 4.4 and order versions, the array updates were done lexicographically on the index keys. So an array value at index 10 would have been updated ahead of say index 3. If there was an array no value at index 10, we would have create array entries with 'null' values upto index 9. Because of the missing field semantics, we would just update the value in $min for index 10. When we try to update index 3 next, there will be a 'null' value. Since the 'null' value is considered less than any other BSON element, we would not update the value for index 3. If the update system had started with index 3 instead, we would have updated both index value 3 and 10.



 Comments   
Comment by James Wahlin [ 19/May/21 ]

Closing as a duplicate of SERVER-50778

Comment by James Wahlin [ 19/May/21 ]

While a backport to 4.4 of SERVER-50778 would address this issue, we are hesitant to move forward with it as it is a behavior change which there has not been significant user demand for. For the time being we will explore addressing failures in our continuous integration environment via fuzzer changes. If this becomes a larger issue for users we can reconsider a backport.

Comment by Ana Meza [ 08/Apr/21 ]

Moving this to QO, we think that the fix for this is to Back Port SERVER-50778 which was implemented by Ted

Comment by Sam Mercier [ 31/Mar/21 ]

Ohhhhh we do on 4.4. Okay gotcha. Sorry that was the bit I was missing. Thanks for being patient with me arun.banala. Sorry I didn't have more to contribute.

Comment by Arun Banala [ 31/Mar/21 ]

samuel.mercier For the case on $min, we don't actually error. We end up doing wrong update operation, which is a bigger problem than just failing, since it's misleading the users. I think we should fix the problem or at least convert this to an error on 4.4.

Comment by Sam Mercier [ 31/Mar/21 ]

Sorry for butting in y'all. But I think we already have precedent for ignoring such errors. (At least the errors that BF-20201 tracks). Let me know if I'm entirely misunderstanding the situation. 

See:
https://github.com/10gen/jstestfuzz/commit/7a61a8f19d76a42cb74fecfff708283837d094dd (authored by ted.tuckman)
https://github.com/10gen/jstestfuzz/commit/48aef3d0a9abf84878f1792abdd8bb9a54eb869d (authored by me) with associated issue TIG-2919.

Comment by Kyle Suarez [ 26/Mar/21 ]

In general, I think the newer behavior post-SERVER-50778 is preferable to the old 4.4 behavior. We should also consider ignoring these differences in the fuzzer.

Comment by Arun Banala [ 26/Mar/21 ]

Backporting SERVER-50778 to 4.4 would fix this, but since the change impacts all the update operators, I don't think we can release this in a minor release.

Repro.

// On 4.4.
> db.arrays.drop(); db.arrays.insert({arr: []})
WriteResult({ "nInserted" : 1 })
> db.arrays.update({}, {$min: {'arr.3': 1, 'arr.10': 1}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : ObjectId("605e048a2b8d17f10d09d5bd"), "arr" : [ null, null, null, null, null, null, null, null, null, null, 1 ] }
 
 
> db.arrays.drop(); db.arrays.insert({arr: []})
WriteResult({ "nInserted" : 1 })
> db.arrays.update({}, {$min: {'arr.3': 1, 'arr.9': 1}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.arrays.find()
{ "_id" : ObjectId("605e04a62b8d17f10d09d5be"), "arr" : [ null, null, null, 1, null, null, null, null, null, 1 ] }

Generated at Thu Feb 08 05:36:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.