[SERVER-4589] aggregation: need an array indexing operator Created: 30/Dec/11 Updated: 05/Feb/16 Resolved: 18/Jun/15 |
|
Status: | Closed |
Project: | Core Server |
Component/s: | Aggregation Framework |
Affects Version/s: | None |
Fix Version/s: | 3.1.5 |
Type: | New Feature | Priority: | Major - P3 |
Reporter: | Dan Pasette | Assignee: | Charlie Swanson |
Resolution: | Fixed | Votes: | 75 |
Labels: | None |
Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||
Backwards Compatibility: | Minor Change | ||||||||||||||||||||||||||||||||||||||||
Sprint: | Quint Iteration 4, Quint Iteration 5 | ||||||||||||||||||||||||||||||||||||||||
Participants: |
Description |
At present, there's no way to extract a specific element from an array-valued field. An operator should be added that could be used in a projection, e.g.: db.runCommand({aggregate:"c", pipeline:[ }} The effect of this would be put add the computed field (virtual field) foo to the projection, and give it the value of arrayField[5]. |
Comments |
Comment by Scott Hernandez (Inactive) [ 30/Dec/11 ] | ||||||
Should this support something like $slice as well? | ||||||
Comment by Chris Westin [ 13/Mar/12 ] | ||||||
$slice ounds like a good idea. Going after a single element is a special case of $slice, so it seems like we should just implement that, and support functional use ($slice) as well as array element syntax to call the same thing. | ||||||
Comment by Gavin Hogan [ 21/Aug/12 ] | ||||||
This would be very helpful in combination with $unwind. | ||||||
Comment by Christian Csar [ 30/Nov/12 ] | ||||||
I think it's important to note that for $match this is not a new feature, rather it appears to be a bug in a currently existing feature (and arguably for $project as well). See the documented syntax at http://www.mongodb.org/display/DOCS/Dot+Notation+%28Reaching+into+Objects%29#DotNotation%28ReachingintoObjects%29-ArrayElementbyPosition and the language for $match which says "The syntax passed to the $match is identical to the query syntax." See https://groups.google.com/d/topic/mongodb-user/7o5frOLbPQ8/discussion for examples showing limitations of $match vs query in regard to this. At present this makes it seemingly impossible to make use of the individual values in geospatial fields. I'd appreciate it if this was reclassified as a bug. | ||||||
Comment by Asya Kamsky [ 15/Nov/13 ] | ||||||
Note: $match does allow referencing positions of array, at least in the latest master:
This allows matching only documents where a is array or a is non-array, but the same doesn't work in {{ $project {$cond} }} for example. | ||||||
Comment by Nathan Tate [ 12/Aug/14 ] | ||||||
I'm running into a similar issue where I want to pluck the nth item from an array in each matching document:
I can unwind the values, then extract the first or the last, but getting one of the values at a specific index seems impossible. | ||||||
Comment by Christian Csar [ 29/Oct/14 ] | ||||||
Having been reminded of this once more again in regards to Geo, it appears that one of the links above is now dead. With regards to $project, http://docs.mongodb.org/manual/reference/operator/aggregation/project/#include-specific-fields-from-embedded-documents specifies that Dot Notation can be used, and http://docs.mongodb.org/manual/core/document/#dot-notation does still specify that it applies to specific elements of arrays. Again I have not confirmed whether this limitation still exists, but I would have expected some additional comment on this. | ||||||
Comment by Roland Bouman [ 29/Dec/14 ] | ||||||
I completely agree with mr Christian Csar. This is a bug. The dot notation does work for subdocuments in the $project stage. Why would it conceptually be different for arrays? That simply doesn't make sense. Mean while, this is preventing me from flattening specific array elements to properties. I'm arriving at the conclusion that this is currently simply not possible in mongodb - (at least if we discount map/reduce). | ||||||
Comment by Roland Bouman [ 29/Dec/14 ] | ||||||
I'd like to add that I do not see any reason for introducing another operator or to be forced to use $slice to do this. It would be nice if $slice was supported, but this is another matter - this has simply to do with the dot notation not working as advertised. Besides, $slice still returns an array - The problem here is that a dot notation expression should simply extract the value from the array (having whichever type that value has). Consider the following query: db.zips.aggregate([{$match: {"loc.0": -72.188455}}, {$project: {_id:1, loc: 1, lon: {$let: { vars: {lon: "$loc.0"}, in: "$$lon"}} }}]); this returns: { "_id" : "01010", "loc" : [ -72.188455, 42.116543 ], "lon" : [ ] }This is wrong. Expected was: { "_id" : "01010", "loc" : [ -72.188455, 42.116543 ], "lon" : -72.188455 }Notice that The simpler query without $let, should, as far as I can see also work: db.zips.aggregate([{$match: {"loc.0": -72.188455}}, {$project: {_id:1, loc: 1, lon: "$loc.0" }}]); ...but yields the same result - empty array. This already indicates that mongodb knows it is accessing an array so at least it partially understands that "loc" is being accessed (it is the only array in the document) | ||||||
Comment by Asya Kamsky [ 08/Apr/15 ] | ||||||
This would return a scalar (or value of type of array element, rather than array of a single element which is what slice would return. | ||||||
Comment by Githook User [ 18/Jun/15 ] | ||||||
Author: {u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'charlie.swanson@mongodb.com'}Message: | ||||||
Comment by Charlie Swanson [ 18/Jun/15 ] | ||||||
The new expression is called $arrayElemAt. It takes two arguments, an array and an index. As is hopefully self explanatory, it returns the element at the given index in the array. Negative indices are accepted as indexes from the back of the array. If the index is out of bounds, it returns the missing value, meaning the field will not exist in the output. If either argument is null, it returns null, otherwise it errors if the first argument is not an array, or the second argument cannot be represented as an integer value (2, NuberLong(2), and 2.0 work, 2.5 and NumberLong(2^40) do not). For all the edge case details, see the javascript test which was part of the commit. | ||||||
Comment by Oleg Rekutin [ 24/Aug/15 ] | ||||||
For anyone interested, backports of this useful feature: 3.0: https://github.com/evergage/mongo/commit/cde793ed9c85b9356e3c20a6bc09f536cc35f245 |