[CSHARP-1544] System.TimeoutException: when using more than 10 tasks Created: 22/Jan/16  Updated: 05/Apr/19  Resolved: 26/Jan/16

Status: Closed
Project: C# Driver
Component/s: Connectivity, Error Handling, Performance
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Critical - P2
Reporter: Roman Assignee: Unassigned
Resolution: Done Votes: 0
Labels: question
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7, MongoDB v3.0.6


Attachments: File MongoDbAnalysis.7z    

 Description   

Hello, I am trying to insert the documents into the MongoDB using the Task.Run functionality. When I am creating 10 tasks everything works fine, but if there are for example 50 tasks created then I get the System.TimeoutException.
I attached the archive with solution that I created for testing purposes.
Could you please help me to resolve this issue?
I want to create 1000 tasks and each task inside should do 10000 inserts.



 Comments   
Comment by Roman [ 26/Jan/16 ]

Hi Craig,

Thank you for fast response.
I changed the method to return a Task and changed the driver method to InsertOneAsync. It seems that it resolved my issue with the TimeOutException.
After some tests I was able to rich the number of 500 Tasks, probably this is the normal number for my PC. Also I tested this solution in my real web application and it works fine.
You can close this task.

Thank you very much!

Comment by Craig Wilson [ 22/Jan/16 ]

First off, I could not replicate any issues. I even went up to 1000 and had no trouble. So, some statements:

I've noticed you increased the defaults this should help when under high loads. However, this is a console application. If you were to profile the speed of your inserts, you'd find that increasing the thread count to 1000 ultimately hurts performance. You are constrained by the number of cores you have on your machine. In addition, this seems to be a semi-concocted perf test with somewhat arbitrary input. It would be helpful to know what your actual workload is rather than inserting a bunch of instantiated error logs as fast as possible. Where are these error logs coming from? Are you likely to be constrained on ingestion speed as well?

Anyways...

1. What can help with this is to use asynchronous inserts. This is simple enough with the driver by changing your InsertObject method to return a task and internally calling InsertOneAsync. This will let your local computer utilize resources better while network IO is going on. Note that async tends to be a bit slower individually, but ultimately can yield a higher throughput.

2. Instead of picking something arbitrarily high (like 1000), start low, measure, and increase your thread count until you start to get a decline in throughput. That is your magic number.

3. Consider using Parallel.For instead of a for loop composed of Task.Run.

Generated at Wed Feb 07 21:39:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.