[SERVER-61982] $merge changing structure of fields containing '.' in output collection Created: 10/Dec/21  Updated: 02/Nov/23

Status: Backlog
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Michael Gargiulo Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: query-product-1-3-3
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-30575 Please add escaping convention for do... Backlog
Assigned Teams:
Query Optimization
Operating System: ALL
Steps To Reproduce:
  1. db.createCollection('mergeTest', {})
  2. db.'mergeTest'.insertOne( { "ts" : ISODate("2021-11-19T09:27:44.001Z"), "sensorId" : "abc2c", "quantity.goods" : 2.0, "price" : 10.0, "_id" : ObjectId("619c63912dfeb5ace44e3c32") })
  3. db.getCollection('mergeTest').aggregate([\{$limit : 1}, \\{$merge : "testing1"}])
  4. db.getCollection('testing1').findOne({})

Output Results: 

{
"_id": ObjectId("619c63912dfeb5ace44e3c32"),
"price": 10.0,
"quantity":

{ "goods": 2.0 }

,
"sensorId": "abc2c",
"ts": ISODate("2021-11-19T09:27:44.001Z")
}

Participants:

 Description   

When running an aggregation thats ends in $merge against any collection type the structure stored in the output collection for any field names containing a '.' is transformed to a nested objected resulting in a different format from the base collection. 



 Comments   
Comment by Katya Kamenieva [ 14/Jan/22 ]

I think the result of the merge should have dotted field 'quantity.goods' as it was in the original document

Comment by Kyle Suarez [ 14/Jan/22 ]

kateryna.kamenieva, you were involved during the Dots and Dollars project – do you have an opinion on what the right behavior should be here?

Comment by Michael Gargiulo [ 14/Dec/21 ]

An acceptable answer could also be to do nothing, but clearly document the behavior to not potentially break user expected behavior and/or scripts. I'll leave it up to christopher.harris & kateryna.kamenieva to make determinations on the expected behavior and what next steps should be 

Comment by Asya Kamsky [ 10/Dec/21 ]

This only happens when merging with existing documents, if (non-default) whenMatched:"replace" is specified, the problem does not manifest since replace uses insert semantics.

Comment by Asya Kamsky [ 10/Dec/21 ]

Note that this is not related to agg validating (or not) dotted fields - it's passing dotted field to update, but since there is no way to escape the . it's resulting in "incorrect" document modification (from user POV).

 

Comment by Asya Kamsky [ 10/Dec/21 ]

This isn't really agg $merge as much as the fact that update command cannot $set dotted field names.

 

Since $merge uses update it causes setting of new field quantity with subfield goods instead of dotted field quantity.goods.

Note that $out does not have the same problem because it uses insert and not update under the covers.

Generated at Thu Feb 08 05:53:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.