[SERVER-40557] Enable querying of dotted updatedFields in Change Stream pipeline Created: 10/Apr/19  Updated: 25/Apr/19  Resolved: 22/Apr/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Querying
Affects Version/s: 4.0.8
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Simon Yarde Assignee: Eric Sedor
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-21889 Oplog format for $set operations on s... Backlog
Participants:

 Description   

In a Change Stream pipeline, we would like to refer to updated fields in
embedded documents, e.g. updateDescription: { updatedFields: { "a.b.c": "Foo" } }.

But field path expressions cannot refer to such fields because they contain
dots, and field path expressions lack an escape-character; for example like
"$updateDescription.updatedFields.a..b..c".

A workaround for some scenarios is to use the full-document option to have MongoDB
lookup the document and include it in the result, but this incurs a performance-cost
and may return the document in a different state; both of which make the workaround unsuitable for our use-case.

{
    "_id": { "_data": /**/ },
    "operationType": "update",
    "ns": { "db": "test", "coll": "test" },
    "documentKey": { "_id": /**/ },
    "updateDescription": {
       "updatedFields": { "a.b.c": "Foo" },
       "removedFields": []
    }
}

 

changeStream = db.test.watch(
    [
        {
            $project: {
                c: "$updateDescription.updatedFields.a.b.c"
            }
        }
    ]
);



 Comments   
Comment by Simon Yarde [ 25/Apr/19 ]

Thanks Eric and Asya!

I didn't find that ticket when searching and the workarounds are very helpful.

Comment by Eric Sedor [ 22/Apr/19 ]

simony we are going to close this ticket as a duplicate of SERVER-21889. Please watch that ticket for updates.

Comment by Asya Kamsky [ 22/Apr/19 ]

I think this is another request that's duplicate of SERVER-21889 same as SERVER-35279 and SERVER-32782.

Comment by Eric Sedor [ 15/Apr/19 ]

simony, I am passing this to an appropriate team to consider this request against our planned work.

Another workaround that may help is to use $objectToArray. While $unwind is not permitted in changeStream pipelines, the following at least allows you to match on events that update specific dotted fields:

[{$project:{"updatedDescription.updatedFields":{$objectToArray:"$updateDescription.updatedFields"}}},
{$match:{"updatedDescription.updatedFields.k":"a.b.c"}}]

Note that any information you can provide about your use-case and underlying motivations for this request will help us reason about it.

Comment by Eric Sedor [ 11/Apr/19 ]

Thanks for your report! We are taking a look.

Generated at Thu Feb 08 04:55:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.