[CSHARP-1021] Memory usages on Client grows without limit Created: 31/Jul/14  Updated: 04/Apr/16  Resolved: 04/Apr/16

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: 1.9.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ennis Lynch Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 Professional, 32GB ram, Visual Studio 2012 C#


Attachments: Text File releventCode.cs    
Issue Links:
Duplicate
duplicates CSHARP-904 C# driver memory leak Closed

 Description   

While using InsertBatch to insert 100,000 documents of between .5mb and 10mb in size the memory usage grows with out bound, reaching the system size of 32GB and then slowly stops writing to Mongo. When commenting out the specifc mongo code, memory usage stays within 500k for the entire set. the Specific mongos is a sharded cluster but the shards all reside on the same machine.

When tracing through the Source Code from the Official C# API I was unable to locate any obvious issues. Although, there are several delayed LINQ casts and copying of memory that could lead to "confusion" in the Framework. It could possibly be an issue with LINQ and .NET but if that were the case then removing the LINQ and additional copying would likely resolve the issue as well.

The Attached code file operates in two threads. The primary thread loads an XML files from the file system (300) of them and for each file changes the date, one for each day of 2014. This then queues the Documents to a Task which spins and calls Mongo InsertBatch every time the queue is larger than 100 items. The document size ranges from 3kb to 10KB on disk, so slightly smaller when converted to BSON.



 Comments   
Comment by Ennis Lynch [ 31/Jul/14 ]

I searched leak but nothing showed up, I thought it might be already reported. We are porting the code I have now to Java to see if the same dataset will cause the same issue in Java. My speculation is that it revolves around the multiple array copies and casts that occur behind the scenes. documents.Cast<object>() from Insert batch is not evaluated until enumerated in according to the .NET documentation, however, if you trace through the code there is a nest of such operations, including lambda expressions.

I am not able to provide you with the dataset because of information security, however, if you are unable to duplicate from the code and information I provided I can probably "fake" the data and provide a copy.

Comment by Craig Wilson [ 31/Jul/14 ]

Thanks Ennis,

I've linked a duplicate entry. I've attempted to reproduce and track this down before with no avail. Perhaps your additional information will provide some more information.

Craig

Generated at Wed Feb 07 21:38:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.