[SERVER-991] $push with $slice + $sort Created: 09/Apr/10 Updated: 06/Apr/23 Resolved: 13/Dec/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Write Ops |
| Affects Version/s: | None |
| Fix Version/s: | 2.3.2 |
| Type: | New Feature | Priority: | Minor - P4 |
| Reporter: | Ian White | Assignee: | Alberto Lerner |
| Resolution: | Done | Votes: | 141 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Participants: |
Alberto Lerner, auto, Auzzie Booth, bishwa shrestha, Colin Mollenhour, Dave Edelhart, Eliot Horowitz, Flavien, Geoffrey Gallaway, Ian White, Javier Ferrero, Jon Hoffman, Juliusz Gonera, Karoly Negyesi, Keith Branton, Kevin Ryan, Martin Lazarov, Paulo Lomanto, Pierre Dane, Richard Oliver, Scott Lowe, Stephan B, Suhail, Thomas, Victor Boivie, Xiang.Song
|
||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
You can now limit the resulting elements in the array after you push. The new syntax is :
The initial implementation (see Sorting is by embedded document fields and doesn't work any type which is not an embedded docs (in the array). |
| Comments |
| Comment by Flavien [ 19/Jul/13 ] | ||||||||
|
Is there a chance to have this work with the aggregation framework in a $group operator, so that we can find the top K items in each group? | ||||||||
| Comment by auto [ 03/Jan/13 ] | ||||||||
|
Author: {u'date': u'2013-01-03T21:23:09Z', u'name': u'Alberto Lerner', u'email': u'alerner@10gen.com'}Message: | ||||||||
| Comment by auto [ 03/Jan/13 ] | ||||||||
|
Author: {u'date': u'2013-01-03T17:19:13Z', u'name': u'Alberto Lerner', u'email': u'alerner@10gen.com'}Message: | ||||||||
| Comment by auto [ 13/Dec/12 ] | ||||||||
|
Author: {u'date': u'2012-12-13T18:48:38Z', u'email': u'alerner@10gen.com', u'name': u'Alberto Lerner'}Message: | ||||||||
| Comment by auto [ 12/Dec/12 ] | ||||||||
|
Author: {u'date': u'2012-12-12T21:21:43Z', u'email': u'alerner@10gen.com', u'name': u'Alberto Lerner'}Message: | ||||||||
| Comment by Geoffrey Gallaway [ 12/Dec/12 ] | ||||||||
|
Hmm, just tested this and it works great but I had to specifically cast the value for $trimTo to a NumberInt or NumberLong in the mongo shell. { $push : { 'v' : { $each : [ 19 ], $trimTo : 2 }}} gave me "$trim value must be a numeric integer". Was that intended? also, one thought.. Maybe the $trimTo values could be used as hints for padding factors? I would assume the use case for this involves updates to the same collections always having the same $trimTo values and starting off with only one value and growing until $trimTo when the values get popped off. This is probably over-optimization. | ||||||||
| Comment by Alberto Lerner [ 12/Dec/12 ] | ||||||||
|
Added a new variation of $push that trims the resulting array to a given size. Sam and Alberto discussed already a bit about it. | ||||||||
| Comment by Alberto Lerner [ 12/Dec/12 ] | ||||||||
|
Author: {u'date': u'2012-12-12T15:01:10Z', u'email': u'alerner@10gen.com', u'name': u'Alberto Lerner'}Message: | ||||||||
| Comment by auto [ 12/Dec/12 ] | ||||||||
|
Author: {u'date': u'2012-12-12T14:56:18Z', u'email': u'alerner@10gen.com', u'name': u'Alberto Lerner'}Message: | ||||||||
| Comment by Scott Lowe [ 05/Dec/12 ] | ||||||||
|
Definitely a much required basic feature - IMO this is more useful even than capped collections. I can't imagine it's that difficult to implement - is it? | ||||||||
| Comment by Thomas [ 01/Aug/12 ] | ||||||||
|
This feature needs to exist. I'd almost consider just running a patched/forked version of mongodb just for this feature. | ||||||||
| Comment by bishwa shrestha [ 29/May/12 ] | ||||||||
|
@Stephan B : +1 to that | ||||||||
| Comment by Juliusz Gonera [ 29/May/12 ] | ||||||||
|
Why exactly has this been postponed again? What is the problem and how could the community help in resolving it? | ||||||||
| Comment by Stephan B [ 29/May/12 ] | ||||||||
|
With $42M of new investment maybe this tiny 1-day-patch could be considered to be included in the 2.1/2.2 release? :-/ | ||||||||
| Comment by Jon Hoffman [ 24/May/12 ] | ||||||||
|
I implemented this with slightly different syntax than what's proposed. Haven't really tested out in production though: Usage: { $pushMax : { a : {max: 5, data: 4 }} } Also includes a new $hash (should probably be renamed $md5) query operation so that replication works correctly. | ||||||||
| Comment by Richard Oliver [ 23/May/12 ] | ||||||||
|
@Stephan B absolutely, I've been waiting for nearly two years for this feature. It's really frustrating watching it slip further and further back. | ||||||||
| Comment by Stephan B [ 23/May/12 ] | ||||||||
|
Is there ANY chance this might be re-considered for 2.2? I mean it seems trivial to implement and is quite inefficient to solve in application code. I'd just like to understand why a small, but very popular feature like this can't make it into 2.1/2.2 | ||||||||
| Comment by Pierre Dane [ 01/May/12 ] | ||||||||
|
This feature is also really important for us. Would be great if this could get pushed up so it gets into 2.2. Thanks | ||||||||
| Comment by Stephan B [ 17/Apr/12 ] | ||||||||
|
No! Why in 2.3? It says "Estimate: Small ( < 1 day)" - so why not in 2.2? PS: According to "Popular Issues" this ranks #9. | ||||||||
| Comment by Javier Ferrero [ 29/Mar/12 ] | ||||||||
|
+1 for this useful feature. Hope it gets resolved soon. | ||||||||
| Comment by Eliot Horowitz (Inactive) [ 26/Mar/12 ] | ||||||||
|
Yes. | ||||||||
| Comment by Colin Mollenhour [ 26/Mar/12 ] | ||||||||
|
Makes sense. However, that second to last push example is not valid JSON.. Did you mean this?
| ||||||||
| Comment by Eliot Horowitz (Inactive) [ 26/Mar/12 ] | ||||||||
|
We'd like to move more towards the way $addToSet works with $each Current syntax:
newer
given that, perhaps
| ||||||||
| Comment by Victor Boivie [ 25/Mar/12 ] | ||||||||
|
@Colin: How will you handle a $pushAllTrim-syntax?. The argument for $pushAll is an array which would conflict with this proposal. | ||||||||
| Comment by Colin Mollenhour [ 25/Mar/12 ] | ||||||||
|
In light of the discussion on github here is a new syntax proposal:
These operators take a 2-tuple as an argument for each key where the first value is the value to $push or $addToSet and the second is the length to trim to after the push. This holds with the "one operation per key" requirement while allowing different keys to have different trim lengths in one update (which the syntax in the description of this ticket does not allow). | ||||||||
| Comment by Victor Boivie [ 25/Feb/12 ] | ||||||||
|
@Colin: I like that idea even though it breaks the rule that the same field can not be referenced by more than one modifier ( I took a stab at it (called it $trim however) and implemented support for it at https://github.com/mongodb/mongo/pull/181 Please review and/or comment it. | ||||||||
| Comment by Colin Mollenhour [ 14/Feb/12 ] | ||||||||
|
Alternative syntax proposal which would not change any existing modifiers and would work with multiple capped arrays while allowing different sizes for each in one query and also allow for easy "ad-hoc" capping:
MongoDb would have to make the assumption that $prune always is applied after all other operators. | ||||||||
| Comment by Auzzie Booth [ 31/Aug/11 ] | ||||||||
|
Please move this to 1.8.4 if possible. This feature is no too big but save a lot of queries and client-side processing in many scenarios | ||||||||
| Comment by Martin Lazarov [ 03/Aug/11 ] | ||||||||
|
It's really useful and I hope it would be done in version 1.8.4 | ||||||||
| Comment by Paulo Lomanto [ 20/Mar/11 ] | ||||||||
|
+1 for this as this features brings a lot of new possibilities. We have a similar situation as reported by Keith and it is very expensive to deal with a virtual capped array inside our application. | ||||||||
| Comment by Keith Branton [ 18/Nov/10 ] | ||||||||
|
Dave, it is unclear if you are suggesting a mongo stored function (these tend to be a bit slow iir) or application server code. I'm trying to track the last 100 (unique) members that view each profile in a social application. Collection is document per profile, and there is an array in that document called recentViewers. Remember if you are going to do this in the application servers, i.e. load the profile document, change the array and store it again, with the few very popular pages the profiles have probably been updated several times since you loaded the document. Not so easy, unless I'm missing something obvious (which is possible My biggest concern about this proposal is that the size is repeated at each point of use - not very dry. If updates are performed from more than one place in the code (maybe even implemented in different languages) then there is increased risk that the limits will not always be kept in sync, leading to unexpected results. I would much rather see a limit be stored in metadata in the database. Perhaps a better way of implementing this functionality would be a new bson type; capped_array ::= int32 document, supporting all the usual array operations while imposing the limit | ||||||||
| Comment by Dave Edelhart [ 18/Nov/10 ] | ||||||||
|
This may be a useful action but it doesn't follow the behavior of push as it is currently understood and implemented in most languages. Because it implies a limit parameter, if implemented, it should be distinct and named_thus (push_stack(target, [elements], limit)? stack_add(target, [elements], limit)? Again - this feels like a creeping attempt to force business logic into the model. This can just as easily be accomplished with a few lines of javaScript or server side code. | ||||||||
| Comment by Xiang.Song [ 08/Nov/10 ] | ||||||||
|
It's really useful. Hope it would be done by version 1.7 stable | ||||||||
| Comment by Kevin Ryan (Inactive) [ 10/Oct/10 ] | ||||||||
|
any update or eta on this? | ||||||||
| Comment by Suhail [ 06/Jun/10 ] | ||||||||
|
And $addToSet please =) | ||||||||
| Comment by Karoly Negyesi [ 06/Jun/10 ] | ||||||||
|
Like a virtual capped collection, hm? |