[CSHARP-4850] MongoDB.Driver.MongoWaitQueueFullException Created: 20/Nov/23 Updated: 13/Dec/23 Resolved: 13/Dec/23 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | None |
| Affects Version/s: | 2.18.0 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Unknown |
| Reporter: | Mohamed BOUZIDI | Assignee: | Adelin Mbida Owona |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | external-user, performance | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Documentation Changes Summary: | 1. What would you like to communicate to the user about this feature? |
| Description |
|
When the number of threads that trie to execute mongodb queries using MongoDb.Driver 2.18, we start to have some threads that hangs for a while (some seconds to 1 minute), then a lot of errors are logged which all point aout the same issue:
I tried to set Min and Max connection pool size to different values 100 to 500 but didn't resolve the issue:
I tried also to profile the application to find any long queries but I could not find out any. All the times I did it, The first thing that take time based on DotTrace profiler is the Acquire method of the connection pool (I will add some pictures) |
| Comments |
| Comment by Mohamed BOUZIDI [ 13/Dec/23 ] |
|
The issue is fixed now after limiting the number of operations in //. Thank you for your help |
| Comment by Mohamed BOUZIDI [ 24/Nov/23 ] |
|
adelin.mbidaowona@mongodb.com Clear for me, thank you again |
| Comment by Adelin Mbida Owona [ 22/Nov/23 ] |
|
Hi mohamed.bouzidi.90@gmail.com, creating different instances of MongoClient per thread is not advisable. A MongoClient instance is thread safe and you will only need one instance of MongoClient even with multiple threads. When you instantiate a MongoClient, there are a variety of comparatively expensive operations that must be performed including DNS resolution (including TXT and SRV lookups if you are using mongodb+srv://), creating connection pools for each cluster member, establishing monitoring connections to each cluster member, etc. Much of this is done in the background, but it is the reason why we recommend creating your MongoClient instance during application bootstrapping and reusing that instance throughout the lifetime of the application. However, If you are connecting to multiple MongoDB clusters with different connection strings, you would have to create one MongoClient instance per cluster. |
| Comment by Mohamed BOUZIDI [ 22/Nov/23 ] |
|
adelin.mbidaowona@mongodb.com Thank you for the full explanation. I will try to limit the number of concurrent threads that needs to connect to mongodb and let you know if that fixes the issue for us. How about creating different instances of MongoClient without limiting the number of // threads, is it a bad thing to do ? |
| Comment by Adelin Mbida Owona [ 21/Nov/23 ] |
|
Hi mohamed.bouzidi.90@gmail.com, I understand that you are experiencing MongoWaitQueueFullExceptions. This exception is not necessarily a bug. It just indicates that too many threads are trying to use MongoDB at once. By default the connection pool has 100 connections. Threads pull connections from the connection pool and return them when they are done. Once all connections are in use, any further threads that want a connection must wait for some other thread to return a connection to the pool. The driver limits the number of threads that are allowed to wait for a connection at the same time. The default wait queue size is the WaitQueueMulitple (default 5) times the MaxPoolSize (default 100). Thus a default wait queue size of 500. Also to help prevent connection storms, the maxConnecting setting was introduced, which limits the number of concurrent connection establishment requests to a cluster node. The default value is 2. If you are not at maxPoolSize but are seeing MongoWaitQueueFullExceptions, you may be slow to establish new connections. The solution is to either decrease the number of threads trying to use MongoDB at the same time (decrease the degree of parallelism) or to configure the connection pool to have either more connections or a higher wait queue limit, or both. You can also try increasing maxConnecting either via the connection string or via MongoClientSettings.MaxConnecting. I would suggest trying 4 (e.g. double the default value) and gradually increasing from there to see if it resolves the issue. Hopefully these suggestions help solve your problem but if none of these work, this maybe a problem we would like to reproduce. |
| Comment by PM Bot [ 20/Nov/23 ] |
|
Hi mohamed.bouzidi.90@gmail.com, thank you for reporting this issue! The team will look into it and get back to you soon. |