[SERVER-57469] Return partial results from subset of shards when `maxTimeMS` and `allowPartialResults` are set Created: 04/Jun/21  Updated: 29/Oct/23  Resolved: 27/Oct/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.0.24, 4.2.14, 4.4.6, 5.0.0-rc0
Fix Version/s: 6.2.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Blake Oler Assignee: Steve Tarzia
Resolution: Fixed Votes: 0
Labels: query-director-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
causes SERVER-71241 allow_partial_results_with_maxTimeMS.... Closed
causes SERVER-71372 getMore fails to return partial resul... Closed
causes SERVER-71440 Remove OpCtx::setIgnoreInterruptsExce... Closed
Related
Backwards Compatibility: Fully Compatible
Sprint: QE 2021-08-09, QE 2021-08-23, QE 2021-09-06, QE 2021-09-20, QE 2021-10-04, QE 2021-10-18, QE 2021-11-01, QE 2021-11-15, QE 2021-11-29, QE 2021-12-13, QE 2021-12-27, QE 2022-01-10, QE 2022-02-07, QE 2022-01-24
Participants:
Case:
Linked BF Score: 154

 Description   

For all MongoDB versions:

Scenario

  • Multi-shard query with maxTimeMS and allowPartialResults targeting shards A and B
  • Mongos is holding in-memory preliminary results from shard A
  • maxTimeMS is reached before getting results from shard B
  • The mongos throws a maxTimeMS error instead of returning partial results.

Question

Should the shard return partial results?



 Comments   
Comment by Githook User [ 27/Oct/22 ]

Author:

{'name': 'Steve Tarzia', 'email': 'steve.tarzia@mongodb.com', 'username': 'starzia'}

Message: SERVER-57469 return partial results when a shard times out
Branch: master
https://github.com/mongodb/mongo/commit/e3794c64dfa909a99655c34f5ca591ae56061377

Comment by Ethan Zhang (Inactive) [ 16/Jul/21 ]

kody.jones linda.qin I am gonna schedule this for the next iteration.

Comment by Andy Schwerin [ 22/Jun/21 ]

I’m not sure when the drivers give up waiting for a response. After maxTimeMS expires, they may have given up. I’ve asked some drivers writers for feedback.

Comment by Ethan Zhang (Inactive) [ 22/Jun/21 ]

So we should consider returning a partial result when either `maxTimeMS` or kDefaultFindHostTimeout is reached, whichever is earlier.

Comment by Kyle Suarez [ 16/Jun/21 ]

It sounds to me like the next steps for this ticket are:

  • Investigate if not setting maxTimeMS would have allowed the partial result set to be returned, or if the current behavior is maintained;
  • If we just don't return partial results at all, consider converting this ticket to an improvement request to more aggressively return partial results like schwerin has mentioned

Sending this back to the triage queue for reconsideration. We may not have free engineers to look at this until after the 5.0 release has passed, so it would be good to get a sense of the priority of this from the Support Team and other interested parties.

Comment by Andy Schwerin [ 15/Jun/21 ]

I think generally we should be more aggressive about sending results back to the client when allowPartialResults is set and the query doesn't require waiting for all shards to respond. If a client hadn't set maxTimeMS in this scenario, when would the partial results be returned instead? Assuming that shardB stalls indefinitely returning results?

Comment by Blake Oler [ 15/Jun/21 ]

linda.qin is this conclusion acceptable?

Comment by Ethan Zhang (Inactive) [ 15/Jun/21 ]

We discussed this and the current semantics seems correct to us, blake.oler do you have a use case which we should handle it differently?

Comment by Blake Oler [ 09/Jun/21 ]

The query will time out with a maxTimeMS error and will not return partial results.

Comment by Kyle Suarez [ 08/Jun/21 ]

blake.oler, do you know what is the current behavior of the server?

Generated at Thu Feb 08 05:41:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.