[GODRIVER-2845] Support for the waitQueueTimeoutMS connection parameter Created: 12/May/23  Updated: 27/Oct/23  Resolved: 01/Jun/23

Status: Closed
Project: Go Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: D V Assignee: Preston Vasquez
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2023-05-12-12-56-09-888.png    
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

It appears that the `waitQueueTimeoutMS` connection string parameter isn't supported by the Go driver yet. I thought it was because it's listed as an option on this page: https://www.mongodb.com/docs/drivers/go/current/faq/#how-does-connection-pooling-work-in-the-go-driver-

I've tested this behaviour using 1.11.1 against MongoDB 5.0 by adding `waitQueueTimeoutMS=1&maxPoolSize=1` to my connection string and then adding lots of load on an endpoint that performs queries. What I saw was requests queueing up quickly and waiting for a long time until I stopped the test at which point they slowly recovered.

Here's a visual representation of the "sawtooth" pattern that I saw during the test:

This pattern is also what we're seeing in our production environment during traffic spikes. Requests are hanging for many seconds, sometimes minutes.

Would it be possible to implement the `waitQueueTimeoutMS`, so waiting on a connection pool acquisition could be aborted quickly? Ideally, this would generate a PoolEvent, so we could track occurrences of this situation in our metrics.

I found this old comment: https://jira.mongodb.org/browse/GODRIVER-211?focusedCommentId=1791246&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1791246 . In case it's still the reason why this connection string parameter isn't supported I'd like to offer a use case where that explanation isn't sufficient. In our case, we have some occasional long-running queries that are using the same client instance that's handling most of our traffic and making only quick queries. If we were to use a timeout on the context or on the client itself we would either cause the longer-running queries to fail unnecessarily (with a small timeout) or for goroutines to wait much longer than our users would when the connection pool is exhausted (with a long timeout). The `waitQueueTimeoutMS` option enables quick load shedding while allowing for longer timeouts under low-load conditions. In general, we would like queries to run for as long as they need to if there're enough connections available in the pool, but fail quickly if the pool is exhausted. Is there another way of accomplishing this without the `waitQueueTimeoutMS` option?



 Comments   
Comment by D V [ 30/May/23 ]

preston.vasquez@mongodb.com Thanks for the suggested workaround! I think this should work in our case. We'll try it out within the next month or so.

Comment by Preston Vasquez [ 25/May/23 ]

dv@glyphy.com , thank you for your detailed explanation! There is a drivers-wide initiative to deprecate waitQueueTimeoutMS in favor of the client-side operation timeout functionality provided by clientOptions.SetTimeout. The team would like to suggest a solution defining load-shedding logic on the user-side using the x/sync/semaphore package:

Gist: https://gist.github.com/prestonvasquez/0092f606f6fec51330a656a67257fc2e
Go Playground: https://go.dev/play/p/xGQyN6v9J6M

 

Comment by PM Bot [ 12/May/23 ]

Hi dv@glyphy.com, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Thu Feb 08 08:39:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.