[CSHARP-4661] The driver could possibly use less threads for background operations Created: 26/May/23  Updated: 27/Oct/23  Resolved: 16/Jun/23

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Aristarkh Zagorodnikov Assignee: Boris Dogadov
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

Hi!

Currently, MongoDB driver starts multiple separate threads for background activities:
connection pool maintenance, roundtrip monitor and heartbeat. When an application is using multiple instances of MongoDB, especially replica sets, the number of these threads grows quickly; in one of the recent dumps I've been looking at when debugging another issue, I encountered 162 threads dedicated to MongoDB (of ~300 threads in total for a loaded web app backend).

I understand why the driver does not use built-in thread pool – it can be starved by application tasks and the driver itself could starve it when there is a spike of new servers – having dedicated threads solves this issue. But the current numbers of threads that mostly sit there waiting for something, occupying stack space still looks suboptimal to me. I undestand that there is nothing wrong with threads from a functional standpoint, but still hope that you would consider reducing the number of threads used for background operations, by either combining multiple tasks into one thread, using some kind of task queue or any other option that you see fit.



 Comments   
Comment by Boris Dogadov [ 16/Jun/23 ]

Thank you onyxmaster for your question!

Comment by Aristarkh Zagorodnikov [ 14/Jun/23 ]

Boris, thank you for the response.
I understand that our use case significantly differs from the common ones, so the impact of optimizations in this area probably won't justify the effort required. Since it does not pose any real performance problems for us, we will just keep this in mind as "this is ok" when profiling our application.
Again, thank you for taking a look into this

Comment by Boris Dogadov [ 13/Jun/23 ]

Thanks for the additional info onyxmaster 

In this case the amount of threads in expected.
As you've mentioned, driver tries to balance between starvation (and overall performance) and resource consumption for the most common use cases.
For more rare cases as this one, the application architecture needs to account for resources consumed by the driver.
Unfortunately currently we don't have any optimizations planned in this area.

Comment by Aristarkh Zagorodnikov [ 06/Jun/23 ]

I investigated it a bit further. It's a large application that legitimately connects to multiple replica sets, using different combination of settings to some of them, which increases the amount of active pools. So yes, it's connected to 54-56 servers (different ServerIds).

Comment by Aristarkh Zagorodnikov [ 04/Jun/23 ]

Boris, thanks for looking into this. I hope to get back with more details next week.

Comment by Boris Dogadov [ 30/May/23 ]

Hi onyxmaster 

162 threads is not an expected behaviour. As you noticed, driver uses 3 threads for every server, for example replica set with 5 nodes will result in 15 internal threads.
Few possible reasons for observing such high amount of threads:

If the none of those points is the root-cause, it would be very helpful to receive a simple self-contain repo, that demonstrates the unbounded threads amount used by the driver.
Thanks.

Comment by PM Bot [ 26/May/23 ]

Hi onyxmaster, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Wed Feb 07 21:48:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.