[SERVER-31484] Operation deadline and awaitData timeout should be separate Created: 10/Oct/17  Updated: 30/Oct/23  Resolved: 31/Jan/18

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 3.6.3, 3.7.2

Type: Improvement Priority: Major - P3
Reporter: Matthew Russotto Assignee: Martin Neupauer
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.6
Sprint: Query 2017-11-13, Query 2017-12-04, Query 2017-12-18, Query 2018-01-01, Query 2018-01-15, Query 2018-01-29, Query 2018-02-12
Participants:
Linked BF Score: 0

 Description   

Currently we use the same timeout for two conceptually different things

1) How long should an operation take, overall?

2) With an awaitData cursor, how long should we wait if there is no data.

It would be useful if these were separate things; we may wish to allow for slow systems which might take several seconds to return data from a potentially large and complex query, but still return quickly (perhaps immediately) if no data is available.

This would consist of at least

1) Adding a new field to the getMore command for the no-data wait (maxTimeMS is currently consistently used for operation deadlines)

2) Keeping track of this new data on the OperationContext

3) Making expiration of the no-data wait time an interrupt, but not a fatal one.

4) Handling old queries with only maxTimeMS the same way they are now.



 Comments   
Comment by Githook User [ 05/Feb/18 ]

Author:

{'email': 'martin.neupauer@mongodb.com', 'name': 'Martin Neupauer', 'username': 'MartinNeupauer'}

Message: SERVER-31484 separate the operation deadline from awaitData deadline in sharded queries.

The deadline has been been already separated for non-sharded queries.

(cherry picked from commit 15a7ac9ca54f2d580e2b1d1ab01fe095be1233db)
Branch: v3.6
https://github.com/mongodb/mongo/commit/6ea8d653429b0a75d87847c2fb4c0802fce727d9

Comment by Githook User [ 31/Jan/18 ]

Author:

{'email': 'martin.neupauer@mongodb.com', 'name': 'Martin Neupauer', 'username': 'MartinNeupauer'}

Message: SERVER-31484 separate the operation deadline from awaitData deadline in sharded queries.
The deadline has been been already separated for non-sharded queries.
Branch: master
https://github.com/mongodb/mongo/commit/15a7ac9ca54f2d580e2b1d1ab01fe095be1233db

Comment by Charlie Swanson [ 08/Jan/18 ]

One known call site that will need to change is here. That call site will use the deadline on the operation as the timeout for waiting. Once we remove the timeout from the OperationContext, it should instead use the await data timeout.

Comment by Charlie Swanson [ 08/Jan/18 ]

Copying a comment from david.storch on the code review for SERVER-31684 that has some goals for this ticket:

As a followup (not part of this patch), I think we should extend this work to the mongos query path. That would allow us to delete the DeadlineStash and the special timeout-swallowing behavior in ExpressionContext::checkForInterrupt().

Comment by Charlie Swanson [ 17/Nov/17 ]

I actually think this ticket should remain open and SERVER-31684 should be closed as a duplicate of this, since this ticket is more representative of the engineering work required. I'm re-assigning this ticket to martin.neupauer, sorry for not noticing this was already slotted for this sprint james.wahlin!

Comment by James Wahlin [ 17/Nov/17 ]

martin.neupauer - Charlie thought that you are covering the work described by this ticket under another one. Can you confirm? I will close as a dupe if that is the case.

Comment by David Storch [ 10/Oct/17 ]

While I think this would be a great improvement to the implementation of awaitData cursors, I do not think we should change the interface exposed to clients. This has compatibility implications and would require work from the drivers team. Instead, we can internally separate our handling of maxTimeMS when it represents a deadline from our handling of maxTimeMS when it represents the wait time for an awaitData cursor.

Generated at Thu Feb 08 04:27:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.