Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39833

BSONObjectTooLarge when updating 250k documents with an unordered bulk write

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Incomplete
    • Icon: Minor - P4 Minor - P4
    • None
    • 4.0.2
    • None
    • None
    • ALL
    • Hide

      Again, sorry, deadlines to meet so I can't produce a minimal example right now.

      Show
      Again, sorry, deadlines to meet so I can't produce a minimal example right now.

    Description

      Sorry, this will be a mostly useless bug report, because I don't have time to put together a minimal reproducable example. I'm only reporting it because I figure it might help gives some clues is someone else comes across a similar issue.

      I was pushing about 250,000 updates to the server at once via an unordered bulk write, and was getting this message:

      Assertion: BSONObjectTooLarge: BSONObj size: 17390105 (0x1095A19) is invalid. Size must be between 0 and 16793600(16MB) First element: update: "my-collection-name" src/mongo/bson/bsonobj.cpp 101
       
      D COMMAND  [conn14143] assertion while parsing command: BSONObjectTooLarge: BSONObj size: 17390105 (0x1095A19) is invalid. Size must be between 0 and 16793600(16MB) First element: update: "my-collection-name"
       
      I COMMAND  [conn14143] query  numYields:0 ok:0 errMsg:"BSONObj size: 17390105 (0x1095A19) is invalid. Size must be between 0 and 16793600(16MB) First element: update: \"my-collection-name\"" errName:BSONObjectTooLarge errCode:10334 reslen:245 locks:{} 0ms

      I solved it by breaking the writes into groups of 1000.

      I confirmed that each of the update objects in the ops array was only a few hundred bytes at the absolute most. Each update object was similar to `{$set:{prop1:

      {a:1,b:2,c:3}

      }}`

      Here are my mongo details:

      > mongo --version
      MongoDB shell version v4.0.2
      git version: fc1573ba18aee42f97a3bb13b67af7d837826b47
      OpenSSL version: OpenSSL 1.1.0g  2 Nov 2017
      allocator: tcmalloc
      modules: none
      build environment:
          distmod: ubuntu1804
          distarch: x86_64
          target_arch: x86_64

      Here are my driver details:

      I NETWORK  [conn14177] received client metadata from 3.81.81.59:59234 conn14177: { driver: { name: "nodejs", version: "3.1.13" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.4.0-1066-aws" }, platform: "Node.js v10.15.0, LE, mongodb-core: 3.1.11" }

      If this is expected behaviour (e.g. because bulkWrite can't handle 250k at a time), then it would be neat if the error message were a bit more clear.

      Attachments

        Activity

          People

            eric.sedor@mongodb.com Eric Sedor
            anon2313 Anon
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: