Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38940

Batch update fails with BSON error

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: JavaScript
    • Labels:
      None
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      Not sure how to create a proper reproduction as I only face the issue at one very specific collection. I think it might be similar to https://jira.mongodb.org/browse/SERVER-32372 though.

      Show
      Not sure how to create a proper reproduction as I only face the issue at one very specific collection. I think it might be similar to https://jira.mongodb.org/browse/SERVER-32372  though.

      Description

      When executing a batch update for one of our collections(5806 docs) we suddenly get a:

      error: MongoError: BSONObj size: 16797454 (0x1004F0E) is invalid. Size must be between 0 and 16793600(16MB) First element: update: "collection_name"
      

      using the node native driver at 3.1.10 on latest mongo 4.0 series @atlas

      const bulk = db.collection(coll).initializeUnorderedBulkOp()
      docs.forEach((\{query, update}) => bulk.find(query).updateOne(update)
      bulk.execute()
      

      While I managed to work around this issue by using manually decreasing the batch size via

      let bulk = db.collection(coll).initializeUnorderedBulkOp();
      let counter = 0;
      for (const \{query, update} of docs) {
        bulk.find(query).updateOne(update);
        counter++;
        if ( counter % 500 == 0 ) {
          bulk.execute();
          counter = 0;
          bulk = db.collection(coll).initializeUnorderedBulkOp();
        }
      }
      

      as i get the docs this should be handled internally.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: