[SERVER-4796] Add limit/max to remove command Created: 27/Jan/12  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Ryan Nitz Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 15
Labels: asya-close, query, remove
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to SERVER-1599 support ability to pass hint to update Closed
Assigned Teams:
Query Optimization
Participants:

 Description   

It would be nice to be able remove/delete data in blocks.

To support this, an optional max/limit parameter is required in the remove function.



 Comments   
Comment by Nitin Katkam [ 15/Oct/20 ]

We can possibly take the approach of allowing the limit on deleteMany/updateMany for non-sharded collections, and return an error for sharded collections similar to the restriction on updateOne without a shardKey... but, yes, someone might want to be able to do this for sharded collections .

The closest solution for updateMany is with the aggregation pipeline [ $match, $limit, $set, $merge ] in MongoDB 4.4.
To do a similar deleteMany, we can update a field that is used for a TTL index but the delete by a TTL index isn't immediate - the TTL check runs at an interval.

Comment by Asya Kamsky [ 02/Feb/18 ]

I don't see how this can be honored in a sharded system (when not deleting based on shard key value and targeting a single shard).

Maybe a "soft limit" that's upper max but no guarantees that that many documents will be removed, only max-that-many?

(achieved by sending limit divided by number-of-shards-targeted to each targeted shard?)

Comment by Oleg Rekutin [ 07/Aug/15 ]

The primary use case for us is to throttle deletes. Sometimes under heavy load mass deletes can significantly affect db performance. A limit on remove operations would allow rerunning of delete periodically until data is gone--in this use case, it doesn't matter the order in which data is removed.

Comment by Scott Hernandez (Inactive) [ 29/Jul/12 ]

Sorting would need to be allowed as well, or else you could get things removed in different orders depending on the index(es) selected, or query execution.

Generated at Thu Feb 08 03:07:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.