[SERVER-80922] Object field order is undefined when using $setWindowField Created: 08/Sep/23  Updated: 08/Dec/23  Resolved: 13/Oct/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 7.0.0, 7.0.1, 6.3.2
Fix Version/s: 7.2.0-rc0, 7.0.5, 7.1.2

Type: Task Priority: Major - P3
Reporter: Colin Stolley Assignee: Drew Paroski
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
Assigned Teams:
Query Execution
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.1, v7.0
Sprint: QE 2023-10-02, QE 2023-10-16
Participants:
Linked BF Score: 105

 Description   

We store $setWindowFields output functions in a StringMap, which has an undefined iteration order. This can cause the field order to vary from one execution to the next. This can lead to surprising results, as illustrated in the following reproduction script:

foo = db.getSiblingDB("repro");
foo.bar.drop();
foo.bar.insert({ "obj" : { "obj" : {  } } });
foo.bar.insert({});for (let i=0; i<100; i++) {
        res = foo.bar.aggregate([
        {
          "$setWindowFields" : {
            "output" : {
              "obj.num" : {
                "$first" : 1
              },
              "obj.obj" : {
                "$first" : 2
              }
            }
          }
        },
        {"$sortByCount" : "$obj" }
        ]);
        printjson(res.toArray());
}

The output from the above script oscillates between:

[ { "_id" : { "obj" : 2, "num" : 1 }, "count" : 2 } ] 

and

[{"_id": {"obj":2,"num":1},"count":1},{"_id":{"num":1,"obj":2},"count":1}]

depending on how the field order comes out.

Converting the above StringMap to a std::map fixes the oscillation issue, but it's unclear if we should do more to enforce a specific field order.

This was first spotted by the fuzzer in BF-29818.



 Comments   
Comment by Githook User [ 07/Dec/23 ]

Author:

{'name': 'Matthew Boros', 'email': 'matt.boros@mongodb.com', 'username': 'mattBoros'}

Message: SERVER-80922: Fix inconsistent output field ordering in $setWindowField

GitOrigin-RevId: d37a2b7583d2f2b6a6c6f65e5a5b270f1d9488bc
Branch: v7.0
https://github.com/mongodb/mongo/commit/76c5a062ff2546b80dc02526d63d2e48c5a4ee0f

Comment by Githook User [ 07/Dec/23 ]

Author:

{'name': 'Matthew Boros', 'email': 'matt.boros@mongodb.com', 'username': 'mattBoros'}

Message: SERVER-80922: Fix inconsistent output field ordering in $setWindowField
Branch: v7.1
https://github.com/mongodb/mongo/commit/06b0374ac16b84b9a71a7fbc03fe209965719079

Comment by Matt Boros [ 01/Dec/23 ]

projjal.chanda@mongodb.com did we consider backporting this? I'm seeing a multiversion fuzzer failure where the root cause is this issue. (BF-30994)

Comment by Githook User [ 12/Oct/23 ]

Author:

{'name': 'Projjal Chanda', 'email': 'projjal.chanda@mongodb.com', 'username': 'projjal'}

Message: SERVER-80922: Fix inconsistent output field ordering in $setWindowField
Branch: master
https://github.com/mongodb/mongo/commit/0d9f182c6fe4a071b9e40bbefa98f7de4ad923ac

Comment by Kevin Cherkauer [ 08/Sep/23 ]

This same kind of thing is occurring in BF-29907 for $project pushed down to SBE (with some as-yet unexplained dependency on a $sort having been done first). Concise repro and differences are documented in this comment in the BF.

Generated at Thu Feb 08 06:44:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.