Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-23107

Make mongo shell split bulk ops when size exceeds 16MB

    XMLWordPrintable

    Details

      Description

      According to this driver spec "[m]ore than 16MB worth of inserts are split into multiple messages, and error indexes are rewritten." This is in addition to splitting bulk ops that contain > 1000 operations into multiple ops.

      The mongo shell does split bulk ops that contain > 1000 operations into 1000-op batches. But when the bulk op exceeds 16MB, it issues an error:

      2016-03-14T11:09:57.077+0200 E QUERY    [thread1] Error: Converting from JavaScript to BSON failed: Object size 16795903 exceeds limit of 16793600 bytes. :
      DBQuery.prototype._exec@src/mongo/shell/query.js:112:28
      DBQuery.prototype.next@src/mongo/shell/query.js:283:5
      Bulk/executeBatch@src/mongo/shell/bulk_api.js:853:16
      Bulk/this.execute@src/mongo/shell/bulk_api.js:1139:11
      @bulk.js:31:1
      

      The python and C drivers, on the other hand, work as expected. They split bulk ops that either contain > 1000 ops or are > 16 MB into smaller batches.

        Attachments

        1. bulk.js
          17 kB
        2. bulk.py
          18 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated: