[SERVER-8468] Feature request: after {criteria}, from {criteria} to {criteria} support Created: 07/Feb/13  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: Jonathan Richard Ong Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Optimization
Participants:

 Description   

Many sites have "infinite scroll" where pagination doesn't make sense with just "skips". Instead, people usually have "after ID" or "from ID1 to ID2". This report is for this feature, but I am pushing more for the "after" feature.

Currently, this can all be done very easily on the client, but it costs extra network bandwidth and CPU. It would thus be much better if done on the server.

The database should check to make sure the `after`, `from`, and `to` parameters are valid (ie those documents exist in the collection). If they don't, it is an invalid query. I'm not sure whether it should return "no documents found" or an actual error. Since most likely these parameters are just `_id` parameters, it should not be expensive. This is a check that is relatively more costly on the client.

There should be a "seek limit" option, meaning if you have both an `after` and `seek` parameter, if `seek` documents have been checked and skipped for one that matches `after`, give up. This is so the query doesn't check the entire database, specifically when the `sort` parameters are not indexed.

Thus, a query might look like so:

 
db.things.find({
  deleted: false,
  type: 'great'
}).sort({
  'date.created': -1
}).after({
  _id: ObjectId("123412341234123412341234")
}).seek(1000).limit(15)
 

Where `.after` could just take `ObjectId()` as well.
In fact, an `ObjectId`-only restriction is welcome.

Thus, for the aforementioned query, all "things" will be sorted by newest, and the next (at most) 15 documents created after (but not including) ObjectId("123412341234123412341234") will be returned.

Hopefully I covered the specifications pretty well. Let me know if you want clarification.



 Comments   
Comment by Jonathan Richard Ong [ 08/Feb/13 ]

Also require that a `sort` parameter is present.

Comment by Jonathan Richard Ong [ 07/Feb/13 ]

These parameters can also be considered as implicit sort range values.

Comment by Jonathan Richard Ong [ 07/Feb/13 ]

Okay, just realized that the `seek` option is completely unnecessary on the database, but required if done on the client (since you don't want to return half the collection unnecessarily). However, there should be an extra restriction that `after`, `from`, and `to` parameters are unique.

Generated at Thu Feb 08 03:17:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.