[SERVER-14466] Support projecting the $id or $ref for a DBRef() field Created: 04/Jul/14 Updated: 27/Mar/20 Resolved: 22/Nov/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | 2.6.0 |
| Fix Version/s: | 4.3.3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Stennie Steneker (Inactive) | Assignee: | Ian Boros |
| Resolution: | Done | Votes: | 20 |
| Labels: | QFB, asya, eng-m, expression, query-44-grooming, read-only-views, usability | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||||||||||||||||||||||
| Sprint: | Query 2019-11-18, Query 2019-12-02 | ||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||
| Linked BF Score: | 39 | ||||||||||||||||||||||||||||||||||||||||
| Description |
|
DBRef fields are only supported as passthrough values. It would be useful to be able to extract the $id and $ref values for manipulation in a pipeline. *edit* it's also possible to use full DBRef values for comparisons. |
| Comments |
| Comment by Roberto Como [ 27/Mar/20 ] | |||||||||||||||
|
Are there any plans to merge this fix in a production (4.2.X ) build ? | |||||||||||||||
| Comment by Githook User [ 21/Nov/19 ] | |||||||||||||||
|
Author: {'email': 'ian.boros@mongodb.com', 'name': 'Ian Boros', 'username': 'puppyofkosh'}Message: | |||||||||||||||
| Comment by Githook User [ 21/Nov/19 ] | |||||||||||||||
|
Author: {'name': 'Mathew Robinson', 'username': 'chasinglogic', 'email': 'mathew.robinson@mongodb.com'}Message: Revert " | |||||||||||||||
| Comment by Githook User [ 20/Nov/19 ] | |||||||||||||||
|
Author: {'name': 'Ian Boros', 'username': 'puppyofkosh', 'email': 'ian.boros@mongodb.com'}Message: | |||||||||||||||
| Comment by Asya Kamsky [ 17/Nov/19 ] | |||||||||||||||
|
You cannot use a field added in a stage till the following stage - in other words new field is added to documents as a result of your $addFields, therefore you cannot refer to it in the same stage that's defining it. These types of questions are better resolved in one of the support forums. | |||||||||||||||
| Comment by naresh buruzula [ 13/Nov/18 ] | |||||||||||||||
|
I am trying to use new filed (created using $addFields) inside path. $addFields: { "identifier": {$ifNull: ["$requests.identifiers.id", ""]}, } I want to use the identifier field from fist step in the path of the second step (while creating the key field). but I don't get $identifier evaluated and get the below err. err: FieldPath field names may not start with '$'. Is this allowed in mongodb 3.6? | |||||||||||||||
| Comment by Asya Kamsky [ 13/Feb/18 ] | |||||||||||||||
|
Note that while projection of individual sub-components of DBRef doesn't currently work in aggregation (other than via workaround mentioned above) it is possible to use full DBRef in comparison operations as I show here | |||||||||||||||
| Comment by Josef Sábl [ 13/Dec/17 ] | |||||||||||||||
|
I've hit this wall due to this bug. | |||||||||||||||
| Comment by Asya Kamsky [ 15/Jun/17 ] | |||||||||||||||
|
For a more generalized conversion of an object "foo" into an object that has all the leading "$" stripped from the key names, this expression should do it:
This takes field "$foo" that's an object and makes "newfoo" the same object but with key names "$name" stipped of first character. So
becomes
| |||||||||||||||
| Comment by Andreas B. [ 14/Jun/17 ] | |||||||||||||||
|
Thanks charlie.swanson! That looks hacky, but I guess it'll have to do until the issue is resolved. | |||||||||||||||
| Comment by Charlie Swanson [ 14/Jun/17 ] | |||||||||||||||
|
Hi a.braun, If you're just looking to get the values out, this should work:
It uses the new $objectToArray expression, introduced in | |||||||||||||||
| Comment by Andreas B. [ 14/Jun/17 ] | |||||||||||||||
For the uninitiated, could you please post the workaround? I'm quite interested in knowing how to do this. | |||||||||||||||
| Comment by Siyuan Zhou [ 13/Jun/17 ] | |||||||||||||||
|
david.storch, thanks for asking! charlie.swanson showed me how to use $objectToArray + $filter + $map + $arrayElemAt to work around this issue, so we are not blocked by this issue. | |||||||||||||||
| Comment by Asya Kamsky [ 08/Jun/17 ] | |||||||||||||||
|
When change notification is first stage in aggregation any data manipulation that's done in that pipeline may need to project or check/use the value of "o.$__" field ($set, $unset, etc). | |||||||||||||||
| Comment by David Storch [ 08/Jun/17 ] | |||||||||||||||
|
siyuan.zhou, is this required for the $changeNotification work, or do you have a workaround? I'd like to know if you need work from the query team so that we plan our time accordingly. | |||||||||||||||
| Comment by Siyuan Zhou [ 07/Jun/17 ] | |||||||||||||||
|
Aggregation over local.oplog.rs also will benefit from this when referring o.$set to get the changed fields of updates. | |||||||||||||||
| Comment by Asya Kamsky [ 19/Apr/17 ] | |||||||||||||||
|
Same for distinct. db.coll.distinct("foo.$ref") | |||||||||||||||
| Comment by Asya Kamsky [ 19/Apr/17 ] | |||||||||||||||
|
Also can cause an error when doing projection on DBRef in a view.
| |||||||||||||||
| Comment by Asya Kamsky [ 21/Jul/16 ] | |||||||||||||||
|
Also affects $graphLookup the same way. | |||||||||||||||
| Comment by Igor Dianov [X] [ 14/Feb/16 ] | |||||||||||||||
|
This also affects the $lookup stage introduced in 3.2 which was designed to do joins to other documents which are quite frequently referenced by DBRef objects. For example: db.getCollection('Positions').aggregate([{$lookup: {from: "Employees", localField: "employeeRef.$id", foreignField: "_id", as: "employee" }}]) assert: command failed: { "ok" : 0, "errmsg" : "FieldPath field names may not start with '$'.", "code" : 16410 } : aggregate failed | |||||||||||||||
| Comment by Asya Kamsky [ 04/Feb/16 ] | |||||||||||||||
|
From description of duplicate ticket: While the $match stage does allow references to DBRef fields (i.e. $id, $ref, $db), users cannot refer to those fields in $project and $group stages (I haven't tested others). I don't see a reason why this would conflict with field path and system variables, since the "$" in question is not a prefix. Could the assertions be relaxed to allow these DBRef fields to be used in pipelines? I'll note that while storing a referenced document's ID in a basic field (or array for reference-many relationships) may be the general advice for users, we've never formally deprecated use of DBRef (not to be confused with DBPointer, which was deprecated in BSON). The DBRef convention is still widely used in ODM libraries. |