[CSHARP-1021] Memory usages on Client grows without limit Created: 31/Jul/14 Updated: 04/Apr/16 Resolved: 04/Apr/16 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | None |
| Affects Version/s: | 1.9.2 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Ennis Lynch | Assignee: | Unassigned |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows 7 Professional, 32GB ram, Visual Studio 2012 C# |
||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Description |
|
While using InsertBatch to insert 100,000 documents of between .5mb and 10mb in size the memory usage grows with out bound, reaching the system size of 32GB and then slowly stops writing to Mongo. When commenting out the specifc mongo code, memory usage stays within 500k for the entire set. the Specific mongos is a sharded cluster but the shards all reside on the same machine. When tracing through the Source Code from the Official C# API I was unable to locate any obvious issues. Although, there are several delayed LINQ casts and copying of memory that could lead to "confusion" in the Framework. It could possibly be an issue with LINQ and .NET but if that were the case then removing the LINQ and additional copying would likely resolve the issue as well. The Attached code file operates in two threads. The primary thread loads an XML files from the file system (300) of them and for each file changes the date, one for each day of 2014. This then queues the Documents to a Task which spins and calls Mongo InsertBatch every time the queue is larger than 100 items. The document size ranges from 3kb to 10KB on disk, so slightly smaller when converted to BSON. |
| Comments |
| Comment by Ennis Lynch [ 31/Jul/14 ] |
|
I searched leak but nothing showed up, I thought it might be already reported. We are porting the code I have now to Java to see if the same dataset will cause the same issue in Java. My speculation is that it revolves around the multiple array copies and casts that occur behind the scenes. documents.Cast<object>() from Insert batch is not evaluated until enumerated in according to the .NET documentation, however, if you trace through the code there is a nest of such operations, including lambda expressions. I am not able to provide you with the dataset because of information security, however, if you are unable to duplicate from the code and information I provided I can probably "fake" the data and provide a copy. |
| Comment by Craig Wilson [ 31/Jul/14 ] |
|
Thanks Ennis, I've linked a duplicate entry. I've attempted to reproduce and track this down before with no avail. Perhaps your additional information will provide some more information. Craig |