[SERVER-74453] Implement support for getMore against a mongot cursor created with a user-defined limit Created: 28/Feb/23  Updated: 29/Oct/23  Resolved: 24/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 6.0.7, 7.0.0-rc3, 5.0.20

Type: Task Priority: Major - P3
Reporter: Nicholas Zolnierz Assignee: Militsa Sotirova
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-76404 Call the no-prefetch TaskExecutorCurs... Closed
Documented
is documented by DOCS-16160 Investigate changes in SERVER-74453: ... Closed
Assigned Teams:
Query Optimization
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.0, v6.0, v5.0, v4.4
Sprint: QO 2023-05-15, QO 2023-05-29
Participants:

 Description   

The first batch from mongot may not contain the full set of results for the following cases:

  1. A document may be filtered out by the $idLookup stage
  2. 16MB batch size limit (more likely with storedSource, since normally mongot only returns _id and a score)

For these cases, mongod should not need to request the full limit for the subsequent getMore. Instead, we could build a method for only requesting (limit - N) results, where N is the number of documents which made it past the $idLookup stage. The actual approach will be left up to the implementation.

Note that for storedSource queries, there is no $idLookup stage so the solution should not be tied to the presence of it. Also there is no guarantee that one of the cases above does not happen again for the subsequent getMore, so the solution should be generic and handle this. 



 Comments   
Comment by Githook User [ 10/Jul/23 ]

Author:

{'name': 'Militsa Sotirova', 'email': 'militsa.sotirova@mongodb.com', 'username': 'militsasotirova'}

Message: SERVER-74453 Implement support for getMore against mongot with user-defined limit
Branch: v5.0
https://github.com/mongodb/mongo/commit/564b3b0528f2ebc6cc6255f3684bfe8e27fb8fce

Comment by Githook User [ 02/Jun/23 ]

Author:

{'name': 'Militsa Sotirova', 'email': 'militsa.sotirova@mongodb.com', 'username': 'militsasotirova'}

Message: SERVER-74453 Implement support for getMore against mongot with user-defined limit
Branch: v6.0
https://github.com/mongodb/mongo/commit/3b9ef88e5a613a7f5c62fb2d0c847482c6a5d85a

Comment by Githook User [ 01/Jun/23 ]

Author:

{'name': 'Militsa Sotirova', 'email': 'militsa.sotirova@mongodb.com', 'username': 'militsasotirova'}

Message: SERVER-74453 Implement support for getMore against mongot with user-defined limit
Branch: v7.0
https://github.com/mongodb/mongo/commit/6cf5cdd46120d20e5db538b3c5ffe60784c9b98e

Comment by Githook User [ 24/May/23 ]

Author:

{'name': 'Militsa Sotirova', 'email': 'militsa.sotirova@mongodb.com', 'username': 'militsasotirova'}

Message: SERVER-74453 Implement support for getMore against mongot with user-defined limit
Branch: master
https://github.com/mongodb/mongo/commit/a47114a67f6bf4810fac6d730f7e6f35f9f31b22

Generated at Thu Feb 08 06:27:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.