[CSHARP-1180] The wait queue for acquiring a connection to server is full Created: 05/Feb/15  Updated: 07/Aug/17  Resolved: 05/Feb/15

Status: Closed
Project: C# Driver
Component/s: Performance
Affects Version/s: 2.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Casoa Makaso Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Win 8.1 x64, MongoDB 2.6.7


Issue Links:
Duplicate
duplicates CSHARP-1144 Investigate tests busting over the wa... Closed

 Description   

I wanted to try the new 2.0 async features and used the C# Driver Tutorial with the following snippet:

Parallel.For(0, 100000, async _ =>
{
var e = new Entity

{ Name = "Tom" }

;
await collection.InsertOneAsync(e);
});

But after a short period my program fails:

Unhandled Exception:
Unhandled Exception:
Unhandled Exception: MongoDB.Driver.MongoWaitQueueFullException: The wait queue
for acquiring a connection to server localhost:27017 is full.
at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.<AcquireConnec
tionAsync>d__0.MoveNext()
— End of stack trace from previous location where exception was thrown —

When I change the connectionstring to: "mongodb://localhost?maxpoolsize=20000 it seems to work, but hogs alot of memory (> 1GB Ram).

Btw. 1.1.0 works totally fine.



 Comments   
Comment by Jeremy Stafford [ 07/Aug/17 ]

I'm actually encountering an issue with this. The only instance of my trying to write to mongo is throttled with a semaphore. The semaphore is set to only allow 150 concurrent writes and the server is reporting back to me that 226 connections available (before I start) yet I'm still getting this error intermittently.

Comment by Robert Stam [ 11/Jun/16 ]

This exception is not necessarily a bug. It just indicates that too many threads are trying to use MongoDB at once. In the example provided in the description the code is trying to do 100,000 inserts in parallel.

By default the connection pool has 100 connections. Threads pull connections from the connection pool and return them when they are done. Once all connections are in use, any further threads that want a connection must wait for some other thread to return a connection to the pool. The driver limits the number of threads that are allowed to wait for a connection at the same time. The default limit is 500.

With 100,000 parallel inserts it's not unexpected that more than 600 threads might be actively trying to do an insert at the same time, which will result in this exception being thrown.

The solution is to either decrease the number of threads trying to use MongoDB at the same time (decrease the degree of parallelism) or to configure the connection pool to have either more connections or a higher wait queue limit, or both.

If you see this exception and you know there are fewer than 600 threads that would be something we would like to reproduce.

Comment by Baburaj James [ 11/Jun/16 ]

Is it still not fixed, I still get this error from the latest driver.

Comment by Craig Wilson [ 05/Feb/15 ]

Hi Casoa and Ben,

Thanks for the reports. We have seen this before, but thank you so much for the easy repro. While, it's somewhat evident what is going on and what is happening (limited resource connection pool and massive quantity of requests all in parallel), we are working through how to make this work. Please track the other ticket for the fix.

Craig

Comment by Ben Brockway [ 05/Feb/15 ]

I have just encountered the same issue, using collection.Find().ToListAsync instead of InsertAsync

Generated at Wed Feb 07 21:38:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.