[CSHARP-3978] High Memory Usage on InsertMany and InsertManyAsync Created: 27/Nov/21  Updated: 28/Oct/23  Resolved: 02/Dec/21

Status: Closed
Project: C# Driver
Component/s: Operations
Affects Version/s: None
Fix Version/s: 2.14.1

Type: Bug Priority: Critical - P2
Reporter: Metin N/A Assignee: Boris Dogadov
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2021-11-27-23-16-55-836.png     PNG File image-2021-11-27-23-19-21-371.png     PNG File image-2021-11-28-09-36-21-561.png     PNG File image-2021-11-28-09-37-06-162.png     PNG File image-2021-11-28-10-13-04-151.png     PNG File image-2021-11-28-10-20-35-963.png     PNG File image-2021-11-28-10-24-15-578.png     PNG File image-2021-11-28-10-31-13-576.png    

 Description   

I have a basic .NET 6 Application which generates GUID and insert into MongoDB which is running on my local Docker container.

The problem I am having is memory usage of C# application is increasing enormously. 

Here is the insert logic:

I have reviewed memory usage on Visual Studio 2022 Community edition. 

Here is the memory usage and CancellationTokenSource+CallbackNode is equals to the number of documents inserted to mongodb.



 Comments   
Comment by Githook User [ 02/Dec/21 ]

Author:

{'name': 'Boris', 'email': 'boris.dogadov@mongodb.com', 'username': 'BorisDog'}

Message: CSHARP-3978: CancellationTokenSource dispose in SemaphoreSlimSignalable and MaintenanceHelper
Branch: v2.14.x
https://github.com/mongodb/mongo-csharp-driver/commit/bd1e73e24a5f7c6993a796021abc692be7ff5163

Comment by Githook User [ 02/Dec/21 ]

Author:

{'name': 'Boris', 'email': 'boris.dogadov@mongodb.com', 'username': 'BorisDog'}

Message: CSHARP-3978: CancellationTokenSource dispose in SemaphoreSlimSignalable and MaintenanceHelper
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/f4fd11d2c3efea1844ce0979b97381749173b73e

Comment by James Kovacs [ 30/Nov/21 ]

Hi, dikbas.metin@gmail.com,

Thank you for reporting this bug. We have reproduced it in 2.14.0 and are investigating the root cause. We will provide updates and information on a fix in this ticket.

Sincerely,
James

Comment by Metin N/A [ 28/Nov/21 ]

And 3rd test with last previous version of the MondoDB driver which is v2.13.3. Memory consumption is stable.

 

Mongo create script: 

Same as before

Nuget packages:

<ItemGroup>
    <PackageReference Include="MongoDB.Driver" Version="2.13.3" />
  </ItemGroup>

Application code:

Same as before

 

 

Comment by Metin N/A [ 28/Nov/21 ]

I have done 2nd test with previous version of the driver and there is no problem with high memory usage like in the previous test with MongoDB.Driver v2.14.0.

Mongo create script: 

docker run --name some-mongo -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=Aa123456. -e MONGO_INIT_DATABASE=keysdb -p "27000-27002:27017-27019" mongo

 

Nuget packages:

  <ItemGroup>
    <PackageReference Include="MongoDB.Driver" Version="2.12.5" />
  </ItemGroup>

Code:

Same with the one in the previous comment.

 

Comment by Metin N/A [ 28/Nov/21 ]

Hello, sorry I didn't create any repository online.

This problem occurs for both InsertOne and InsertMany. 

Here is the more details:

Mongo create script: 

docker run --name some-mongo -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=Aa123456. -e MONGO_INIT_DATABASE=keysdb -p "27000-27002:27017-27019" mongo

 

.NET 6 Console Application:

Csproj file

<Project Sdk="Microsoft.NET.Sdk">  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>  <ItemGroup>
    <PackageReference Include="MongoDB.Driver" Version="2.14.0" />
  </ItemGroup></Project>

Program.cs file

 

// See https://aka.ms/new-console-template for more information
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
using System.Diagnostics;Console.WriteLine("Hello, World!");
 
var dbClient = new MongoClient("mongodb://mongoadmin:Aa123456.@127.0.0.1:27000");
var db = dbClient.GetDatabase("keysdb");
var collection = db.GetCollection<Doc>("Docs");
 
var keys = Builders<Doc>.IndexKeys.Ascending("Key");
var indexOptions = new CreateIndexOptions { Unique = true };
var model = new CreateIndexModel<Doc>(keys, indexOptions);
collection.Indexes.CreateOne(model);
 
var watch = new Stopwatch();
watch.Start();
long i = 0;
 
while (i < 10000000) // 10 million
{
    var guid = Guid.NewGuid();
    collection.InsertOne(new Doc { Key = guid.ToString() });
    
    if (i % 5000 == 0)
    {
        var mem = GC.GetTotalMemory(true);
        var elapsedSecs = watch.ElapsedMilliseconds / 1000;
        Console.WriteLine($"Time: {DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss")}\t" +
            $"Elapsed secs: {elapsedSecs} sec\t" +
            $"Inserted record: {collection.CountDocuments(x => true)}\t" +
            $"Memory Usage: {mem / (1024 * 1024)} MB");
    }
    i++;
}
 
internal class Doc
{
    [BsonId]
    [BsonRepresentation(BsonType.ObjectId)]
    public string Id { get; set; }
    
    [BsonElement("Key")]
    public string Key { get; set; }
}

MongoDB connection string:

 

mongodb://mongoadmin:Aa123456.@127.0.0.1:27000

 

  

Comment by Dmitry Lukyanov (Inactive) [ 28/Nov/21 ]

Hey dikbas.metin@gmail.com ,

thank you for your report, please provide the following details:

  1. Connection string (with hidden credentials).
  2. MongoClientSettings if it's configured. How do you configure MongoClient (as singleton or no)?
  3. Does this happen all the time? If no, how often, under what conditions?
  4. Is it reproducible without Docker? Also it would be good to get a repro in a small console application.
  5. Maybe you launched this code with previous driver versions, if so, did you see this issue there?
Comment by Metin N/A [ 28/Nov/21 ]

Framework: .NET 6.0

Dependencies:

MongoDB.Driver v2.14.0

Generated at Wed Feb 07 21:46:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.