[SERVER-38940] Batch update fails with BSON error Created: 10/Jan/19  Updated: 16/Nov/21  Resolved: 16/Jan/19

Status: Closed
Project: Core Server
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: lukas strassel Assignee: Danny Hatcher (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
is caused by NODE-1778 BulkWrite: Incorrect batch-size calcu... Closed
Operating System: ALL
Steps To Reproduce:

Not sure how to create a proper reproduction as I only face the issue at one very specific collection. I think it might be similar to https://jira.mongodb.org/browse/SERVER-32372 though.

Participants:

 Description   

When executing a batch update for one of our collections(5806 docs) we suddenly get a:

error: MongoError: BSONObj size: 16797454 (0x1004F0E) is invalid. Size must be between 0 and 16793600(16MB) First element: update: "collection_name"

using the node native driver at 3.1.10 on latest mongo 4.0 series @atlas

const bulk = db.collection(coll).initializeUnorderedBulkOp()
docs.forEach((\{query, update}) => bulk.find(query).updateOne(update)
bulk.execute()

While I managed to work around this issue by using manually decreasing the batch size via

let bulk = db.collection(coll).initializeUnorderedBulkOp();
let counter = 0;
for (const \{query, update} of docs) {
  bulk.find(query).updateOne(update);
  counter++;
  if ( counter % 500 == 0 ) {
    bulk.execute();
    counter = 0;
    bulk = db.collection(coll).initializeUnorderedBulkOp();
  }
}

as i get the docs this should be handled internally.



 Comments   
Comment by Danny Hatcher (Inactive) [ 16/Jan/19 ]

Hello Lukas,

I'm glad to hear it!

Danny

Comment by lukas strassel [ 16/Jan/19 ]

Hello Daniel,

sorry for the late response - I just tried with the 3.1.11 driver and the issue seems to be resolved! :+1:

Thanks for your work,
Lukas

Comment by Danny Hatcher (Inactive) [ 11/Jan/19 ]

Hello Lukas,

You may be experiencing NODE-1778 which was fixed in the next version of the driver (3.1.11). Could you upgrade to the latest version and see if you still experience the issue?

Thank you,

Danny

Generated at Thu Feb 08 04:50:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.