Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-96476

Adding 'missing' field to MutableDocument may change the order of subsequent inserts

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 7.3.0-rc7, 8.1.0-rc0
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • 200

      This can happen e.g. when $project is applied to non-existent field in the document. the projection_node will insert `Value()` object into MutableDocument which is invisible when the document is materialized into BSON. If latter stage attempts to insert the field with the same key it is not inserted at the end of the document. Instead, the 'missing' value is replaced by the new one in place. 

      This matters because one could apply projection in this way to subdocuments and later use the same subdocument as a key for e.g. $group and that will result in documents wrongly ending up in a different group. 

      let test = {
          "pipe": [
              {
                  "$project": {
                      "obj.obj.obj.obj.obj.date": 1,
                      "obj.obj.date": ISODate("0001-01-01T00:00:00Z"),
                  }
              },
              {
                  "$setWindowFields": {
                      "sortBy": {"obj.obj.obj.obj.num": 1},
                      "output": {
                          "obj.obj.obj.obj.obj": {
                              "$first": {"$unsetField": {"field": "obj.obj.obj", "input": {"num": 43785}}}
                          }
                      }
                  }        }
          ],
          "docs": [
              {"_id": 1, "obj": {"obj": {"obj": {"obj": {}}}}},
              {"_id": 2},
          ],
      };
      db.test.drop();
      db.test.insertMany(test.docs);
      db.test.aggregate(test.pipe);
      

      When only running the first stage all seems fine

      > db.test.aggregate(test.pipe.slice(0,1));
      { "_id" : 1, "obj" : { "obj" : { "obj" : { "obj" : {  } }, "date" : ISODate("0001-01-01T00:00:00Z") } } }
      { "_id" : 2, "obj" : { "obj" : { "date" : ISODate("0001-01-01T00:00:00Z") } } }

      Running the whole pipeline inserts `num` before the `date` for the document with {_id: 2} 

       > db.test.aggregate(test.pipe);
      { "_id" : 1, "obj" : { "obj" : { "obj" : { "obj" : { "obj" : { "num" : 43785 } } }, "date" : ISODate("0001-01-01T00:00:00Z") } } }
      { "_id" : 2, "obj" : { "obj" : { "obj" : { "obj" : { "obj" : { "num" : 43785 } } }, "date" : ISODate("0001-01-01T00:00:00Z") } } }

       

      There is a unit test in attachment for the MutableDocument that might help.

      Another way to verify aggregation result is correct would be to convert Document to BSON and back in DocumentSource::GetNext() to force materialization of the documents between stages.

            Assignee:
            Unassigned Unassigned
            Reporter:
            serhii.lysenko@mongodb.com Serhii Lysenko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: