Context

Currently for every operation timeout, the driver will close the database connection that was used for that operation. That can lead to connection storms in the case that a lot of client-side timeouts happen in a short period of time.

The CSOT spec describes adding maxTimeMS to commands to limit their runtime on the database. However, the current algorithm for calculating maxTimeMS uses the minimum observed round-trip time to predict the next round-trip time, which will likely be an underestimate, leading to the client-side deadline being reached before the database-side deadline. To minimize client-side timeouts, we need to use a maxTimeMS algorithm that maximizes the chances that the database-side deadline will be reached before the client-side deadline. We also need to avoid edge cases where setting a low maxTimeMS causes operations to time out on the database unexpectedly early.

Definition of done

• Update the maxTimeMS algorithm to minimize connection churn when timeouts happen.
• Reproduce a connection storm and show that the updated algorithm mitigates the connection storm.
• Consider adding a config for enabling the more aggressive maxTimeMS values so that users can opt-in to the behavior.
• Create a ticket for porting the feature to v2.0

Pitfalls

• maxTimeMS can't be used or doesn't work the same for all commands.
  • maxTimeMS for find commands that require iterating a cursor (i.e. not all results are included on the initial reply) doesn't work the same.
• If we set maxTimeMS too agressively (i.e. too low), users may get database-side timeouts that occur earlier than expected based on the provided client-side timeout.