Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38940

Batch update fails with BSON error

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: JavaScript
    • Labels:
      None
    • ALL
    • Hide

      Not sure how to create a proper reproduction as I only face the issue at one very specific collection. I think it might be similar to https://jira.mongodb.org/browse/SERVER-32372 though.

      Show
      Not sure how to create a proper reproduction as I only face the issue at one very specific collection. I think it might be similar to https://jira.mongodb.org/browse/SERVER-32372  though.

      When executing a batch update for one of our collections(5806 docs) we suddenly get a:

      error: MongoError: BSONObj size: 16797454 (0x1004F0E) is invalid. Size must be between 0 and 16793600(16MB) First element: update: "collection_name"
      

      using the node native driver at 3.1.10 on latest mongo 4.0 series @atlas

      const bulk = db.collection(coll).initializeUnorderedBulkOp()
      docs.forEach((\{query, update}) => bulk.find(query).updateOne(update)
      bulk.execute()
      

      While I managed to work around this issue by using manually decreasing the batch size via

      let bulk = db.collection(coll).initializeUnorderedBulkOp();
      let counter = 0;
      for (const \{query, update} of docs) {
        bulk.find(query).updateOne(update);
        counter++;
        if ( counter % 500 == 0 ) {
          bulk.execute();
          counter = 0;
          bulk = db.collection(coll).initializeUnorderedBulkOp();
        }
      }
      

      as i get the docs this should be handled internally.

            Assignee:
            daniel.hatcher@mongodb.com Danny Hatcher (Inactive)
            Reporter:
            lukasstrassel@googlemail.com lukas strassel
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: