[CSHARP-1959] Potential Deadlock? Created: 10/Apr/17  Updated: 07/Apr/23  Resolved: 03/Dec/20

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Sebastian Stehle Assignee: Unassigned
Resolution: Done Votes: 0
Labels: question
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hi,

I have no more ideas what to look after therefore I am writing this question here. I hope it is okay:

I am running a batch process to import 1.4Mio Hotels into our system. I use the Task Parallel Library for it, there is no .Wait() or .Result

My code looks like this:

public sealed class BatchWriter : IBatchWriter<Hotel>
        {
            private readonly IMongoCollection<MongoTransformedHotel> collection;
            private readonly List<MongoTransformedHotel> transformedHotels = new List<MongoTransformedHotel>();
 
            public BatchWriter(IMongoCollection<MongoTransformedHotel> collection)
            {
                this.collection = collection;
            }
 
            public void Dispose()
            {
            }
 
            public Task WriteBatch(IEnumerable<Hotel> items)
            {
                foreach (var hotel in items)
                {
                    ....
 
                    this.transformedHotels.Add(new MongoTransformedHotel { Id = factsheetId.FactsheetId, HotelCode = hotelCode.Code, Hotel = hotel });
                }
 
                return TaskHelper.Done;
            }
 
            public Task Complete()
            {
                var models = 
                    this.transformedHotels.Select(x => 
                        new ReplaceOneModel<MongoTransformedHotel>(Builders<MongoTransformedHotel>.Filter.Eq(y => y.Id, x.Id), x)
                        {
                            IsUpsert =  true
                        });
 
                return this.collection.BulkWriteAsync(models);
            }
        }

My Batch Size is 1000 Hotels, the documents are complex.

After usually around 400.000 Hotels the process hangs. When I Break All, I see no threads waiting for a lock or so, but there is one task and debugger points to "await batch.Complete()" in the following code. The status of the task is awaiting. I have absolutely no idea what is going on there. Could it be running out of connections or so?

new ActionBlock<T[]>(async b =>
                {
                    using (var batch = writer.StartWriteBatch())
                    {
                        await batch.WriteBatch(b);
                        await batch.Complete();
                    }
                }, new ExecutionDataflowBlockOptions {BoundedCapacity = 2});



 Comments   
Comment by Robert Stam [ 13/Apr/17 ]

The code in that ActionBlock is not part of the driver. So I'm not sure what to do with that information yet.

Generated at Wed Feb 07 21:41:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.