-
Type: Bug
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Write Operations
-
None
-
Environment:MongoDB 3.6.2, Windows 10
By inserting data (~50000 documents at once) using the collection.InsertMany(documents) method I get the following exception:
*MongoDB.Driver.MongoCommandException: Command insert failed: BSONObj size: 16793676 (0x100404C) is invalid. Size must be between 0 and 16793600(16MB)
*
This occures in different collections independent of document structure.
I think that is related to the maxWriteBatchSize value, which was changed from 1000 to 100,000 operations because I am able to workaround this issue downgrading MongoDB to 3.4. So I am not sure if this bug hits the C# driver, but to split the operations is the driver thing, so it is related?
Another workaround I tried, was chunking to 500 documents at once, but unfortunately that did not help.
Full stack:
MongoDB.Driver.MongoCommandException: Command insert failed: BSONObj size: 16793676 (0x100404C) is invalid. Size must be between 0 and 16793600(16MB) First element: insert: "SuppliersArticleFilter".
at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol`1.ProcessReply(ConnectionId connectionId, ReplyMessage`1 reply)
at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol`1.Execute(IConnection connection, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Servers.Server.ServerChannel.ExecuteProtocol[TResult](IWireProtocol`1 protocol, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Servers.Server.ServerChannel.Command[TResult](ICoreSession session, ReadPreference readPreference, DatabaseNamespace databaseNamespace, BsonDocument command, IElementNameValidator commandValidator, BsonDocument additionalOptions, Func`1 responseHandling, Boolean slaveOk, IBsonSerializer`1 resultSerializer, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.RetryableWriteCommandOperationBase.ExecuteAttempt(RetryableWriteContext context, Int32 attempt, Nullable`1 transactionNumber, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.RetryableWriteOperationExecutor.Execute[TResult](IRetryableWriteOperation`1 operation, RetryableWriteContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkUnmixedWriteOperationBase`1.ExecuteBatches(RetryableWriteContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkUnmixedWriteOperationBase`1.Execute(RetryableWriteContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation.ExecuteBatch(RetryableWriteContext context, Batch batch, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation.Execute(IWriteBinding binding, CancellationToken cancellationToken)
at MongoDB.Driver.OperationExecutor.ExecuteWriteOperation[TResult](IWriteBinding binding, IWriteOperation`1 operation, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl`1.ExecuteWriteOperation[TResult](IClientSessionHandle session, IWriteOperation`1 operation, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl`1.BulkWrite(IClientSessionHandle session, IEnumerable`1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl`1.<>c_DisplayClass23_0.<BulkWrite>b_0(IClientSessionHandle session)
at MongoDB.Driver.MongoCollectionImpl`1.UsingImplicitSession[TResult](Func`2 func, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl`1.BulkWrite(IEnumerable`1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionBase`1.InsertMany(IEnumerable`1 documents, InsertManyOptions options, CancellationToken cancellationToken)
at NewCatalogue.DataAccess.Articles.ArticlesDataAdapter.InsertEntities[T](IEnumerable`1 entities, String collectionName) in D:\VSTS\Workspaces\NewCatalog\DevMain\Sources\NewCatalogue.DataAccess.Articles\ArticlesDataAdapter.cs:line 274
at ArticlesData.Import.ArticleFilters.MongoDbArticleFiltersImporter.ImportArticleFiltersForCar(Int32 carId, String collectionName, Int32 linkingTargetType) in D:\VSTS\Workspaces\NewCatalog\DevMain\Sources\ArticlesData.Import\ArticleFilters\MongoDbArticleFiltersImporter.cs:line 93
at ArticlesData.Import.ArticleFilters.MongoDbArticleFiltersImporter.<>c_DisplayClass4_0.<ImportArticleFiltersInternal>b_0(Int32 carId) in D:\VSTS\Workspaces\NewCatalog\DevMain\Sources\ArticlesData.Import\ArticleFilters\MongoDbArticleFiltersImporter.cs:line 71
at System.Threading.Tasks.Parallel.<>c_DisplayClass17_0`1.<ForWorker>b_1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c_DisplayClass176_0.<ExecuteSelfReplicating>b_0(Object )<---
MongoDB logfile:
2018-03-18T13:50:00.904+0100 I CONTROL [main] Trying to start Windows service 'MongoDB'
2018-03-18T13:50:00.906+0100 I CONTROL [initandlisten] MongoDB starting : pid=1544 port=27017 dbpath=C:\MongoData\db 64-bit host=xxxxxx
2018-03-18T13:50:00.906+0100 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2018-03-18T13:50:00.906+0100 I CONTROL [initandlisten] db version v3.6.2
2018-03-18T13:50:00.907+0100 I CONTROL [initandlisten] git version: 489d177dbd0f0420a8ca04d39fd78d0a2c539420
2018-03-18T13:50:00.907+0100 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1u-fips 22 Sep 2016
2018-03-18T13:50:00.907+0100 I CONTROL [initandlisten] allocator: tcmalloc
2018-03-18T13:50:00.916+0100 I CONTROL [initandlisten] modules: none
2018-03-18T13:50:00.916+0100 I CONTROL [initandlisten] build environment:
2018-03-18T13:50:00.916+0100 I CONTROL [initandlisten] distmod: 2008plus-ssl
2018-03-18T13:50:00.916+0100 I CONTROL [initandlisten] distarch: x86_64
2018-03-18T13:50:00.916+0100 I CONTROL [initandlisten] target_arch: x86_64
2018-03-18T13:50:00.916+0100 I CONTROL [initandlisten] options: { config: "C:\MongoData\mongod.cfg", net:
, service: true, setParameter:
{ maxIndexBuildMemoryUsageMegabytes: "3000" }, storage:
{ dbPath: "C:\MongoData\db", directoryPerDB: true }, systemLog:
{ destination: "file", path: "C:\MongoData\log\log.log" } }
2018-03-18T13:50:00.917+0100 I - [initandlisten] Detected data files in C:\MongoData\db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
- duplicates
-
CSHARP-2207 Large bulk writes not always splitting correctly
- Closed