[SERVER-32739] allow limit in $filter expression Created: 17/Jan/18  Updated: 30/Oct/23  Resolved: 26/Oct/21

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 5.2.0

Type: Improvement Priority: Major - P3
Reporter: Asya Kamsky Assignee: Maddie Zechar
Resolution: Fixed Votes: 0
Labels: neweng, storch
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-59303 Support limit expression for $filter ... Closed
Documented
is documented by DOCS-14896 Investigate changes in SERVER-32739: ... Closed
Related
is related to SERVER-25957 Optimize $filter + $arrayElemAt to av... Backlog
Backwards Compatibility: Fully Compatible
Sprint: QO 2021-09-06, Query Optimization 2021-08-09, QO 2021-09-20, QO 2021-10-04, QO 2021-10-18, QO 2021-11-01, QO 2021-08-23
Participants:
Case:

 Description   

Currently $filter expression will return all matching array elements of its input.

This is a request to add a "limit" option which caps the number of matches returned.

{$filter:{
    input:[1,2,3,10,11,12,15],
    cond:{$lt:["$$this", 13]},
    limit: 2
}}
// result
[1,2]



 Comments   
Comment by Githook User [ 05/Nov/21 ]

Author:

{'name': 'Maddie Zechar', 'email': 'mez2113@columbia.edu', 'username': 'madelinezec'}

Message: SERVER-32739: Allow limit in $filter expression
Branch: master
https://github.com/mongodb/mongo/commit/893b336353cbb74437d73763bbf0c1986d2d2e78

Comment by Githook User [ 27/Aug/21 ]

Author:

{'name': 'Maddie Zechar', 'email': 'mez2113@columbia.edu', 'username': 'madelinezec'}

Message: Revert "SERVER-32739: allow limit in $filter expression"
Branch: master
https://github.com/mongodb/mongo/commit/5f9352c058d14b28c4ab32046f47d8169dff0d1d

Comment by Maddie Zechar [ 27/Aug/21 ]

Please ignore the commit message above - I accidentally pasted in the wrong ticket and commit message (but the correct code changes) when enqueuing my patch to the merge queue. I'm working on reverting and recommitting with correct message now.

Comment by Githook User [ 27/Aug/21 ]

Author:

{'name': 'Maddie Zechar', 'email': 'mez2113@columbia.edu', 'username': 'madelinezec'}

Message: SERVER-32739: allow limit in $filter expression
Branch: master
https://github.com/mongodb/mongo/commit/cd10b5e5b12a927455c33d494ad41bc91b9b33cb

Comment by Asya Kamsky [ 18/Jan/18 ]

Since filter preserves order of original array (i.e. $filter with true condition will just return the input) I think limit should be first n (i.e. equivalent to $slice from start of result).

The user can currently use $slice but without SERVER-25957 we will perform $filter on entire array before taking $slice where $limit inside $filter would allow us to short circuit the evaluation after n elements matched.

Comment by Kyle Suarez [ 17/Jan/18 ]

Recalling an in-person conversation from long ago, if there is a limit of n, what if users then want ordering guarantee? For example,

  • the first n
  • the last n
  • the first n after skipping the first m
  • ...
Generated at Thu Feb 08 04:31:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.