Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-23107

Make mongo shell split bulk ops when size exceeds 16MB

    • Server Tooling & Methods

      According to this driver spec "[m]ore than 16MB worth of inserts are split into multiple messages, and error indexes are rewritten." This is in addition to splitting bulk ops that contain > 1000 operations into multiple ops.

      The mongo shell does split bulk ops that contain > 1000 operations into 1000-op batches. But when the bulk op exceeds 16MB, it issues an error:

      2016-03-14T11:09:57.077+0200 E QUERY    [thread1] Error: Converting from JavaScript to BSON failed: Object size 16795903 exceeds limit of 16793600 bytes. :
      DBQuery.prototype._exec@src/mongo/shell/query.js:112:28
      DBQuery.prototype.next@src/mongo/shell/query.js:283:5
      Bulk/executeBatch@src/mongo/shell/bulk_api.js:853:16
      Bulk/this.execute@src/mongo/shell/bulk_api.js:1139:11
      @bulk.js:31:1
      

      The python and C drivers, on the other hand, work as expected. They split bulk ops that either contain > 1000 ops or are > 16 MB into smaller batches.

        1. bulk.py
          18 kB
        2. bulk.js
          17 kB

            Assignee:
            backlog-server-stm Backlog - Server Tooling and Methods (STM) (Inactive)
            Reporter:
            eric.sommer@mongodb.com Eric Sommer
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: