[GODRIVER-2038] Use "ConnectionTimeout" for creating all new connections and background connection creation Created: 02/Jun/21  Updated: 28/Oct/23  Resolved: 20/Oct/21

Status: Closed
Project: Go Driver
Component/s: None
Affects Version/s: None
Fix Version/s: 1.8.0

Type: Improvement Priority: Unknown
Reporter: Matt Dale Assignee: Matt Dale
Resolution: Fixed Votes: 0
Labels: rp-toSched
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on GODRIVER-2078 Stress test Go driver connections dur... Closed
is depended on by GODRIVER-2138 Remove unnecessary operation Context ... Closed
Related
related to GODRIVER-1902 pool.Put() always releases the semaphore Closed
related to GODRIVER-1826 Rate limit new connection creations (... Closed
related to GODRIVER-1827 Implement and Test Connection Pool Pa... Closed
related to DRIVERS-2347 Prevent conflating operation timeout ... In Progress
related to GODRIVER-2024 Connection pool, long semaphore wait ... Closed
related to GODRIVER-2065 Error early if insufficient time rema... Closed
related to GODRIVER-2165 Make "pool.clear()" behavior match CM... Closed
is related to GODRIVER-1799 Avoiding connection storms Closed
Epic Link: Connection pool improvements
Case:

 Description   

Use separate “new connection” timeout

When a user creates a new client, an explicit connection timeout context is provided (and, optionally, a separate connection timeout setting). However, when a user runs an operation and the connection pool doesn’t have any available connections, the pool will try to create a new connection using the same context as the operation (see pool.go#L457). The result is that a completely different timeout is used when creating connections for a new client as when creating connections during an operation, likely violating the user’s expectation and possibly causing a scenario where new connections are never or rarely established successfully.

Instead, we should use a separate connection timeout when creating new connections, whether when creating a new client or when executing an operation.

Establish new connections in the background

When a user runs an operation, a connection must be selected from the connection pool. If one is available, it is removed from the pool, used, then returned to the pool. If one is not available, we either wait for an available connection to be returned to the pool or create one if the connection pool is below the max size. Currently, the connection creation is attempted and cancelled synchronously with the operation, meaning if the operation times out, the connection creation must stop to maintain the expected operation timeout behavior (see pool.go#L339). In some common circumstances, creating a connection may take much longer than an individual query (e.g. when using TLS-encrypted connections to Atlas clusters and querying small documents using an index), so it may be necessary to continue creating a new connection even after the operation times out.

Instead, create new connections in a separate goroutine when necessary and wait for that or another connection to be available when running an operation. If a new operation required creating a new connection, continue creating the connection up to the default or configured connection timeout.



 Comments   
Comment by Githook User [ 20/Oct/21 ]

Author:

{'name': 'Matt Dale', 'email': '9760375+matthewdale@users.noreply.github.com', 'username': 'matthewdale'}

Message: GODRIVER-2038 Make all connections in the background using connectTimeoutMS. (#716)
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/8539500195a2dd50e12e86f8dcf0502daa979cb6

Comment by Matt Dale [ 10/Sep/21 ]

PR: https://github.com/mongodb/mongo-go-driver/pull/716

Generated at Thu Feb 08 08:37:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.