Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-63380

JSTest bulk api can create oversized batches

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.0.0-rc0, 5.3.0-rc1
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Fully Compatible
    • ALL
    • v5.3
    • Hide

      To reproduce the error, apply this patch onto master at commit c99e4833db76bba9842cf9ee9add83aaba0c340a. Run the test jstests/sharding/defragment_large_collection.js in the sharding suite.

      repro.patch

      Show
      To reproduce the error, apply this patch onto master at commit c99e4833db76bba9842cf9ee9add83aaba0c340a. Run the test jstests/sharding/defragment_large_collection.js in the sharding suite. repro.patch
    • Sharding EMEA 2022-02-21
    • 62

      The bulk api checks that a document will not make a batch too large at the point of inserting the document into the batch. At the point of building and executing the command, though, any inserts without an id get this field added to them. The resulting document is never checked for size overflow, and can result in the following error.

      [js_test:defragment_large_collection] Error: Converting from JavaScript to BSON failed: Object size 16793919 exceeds limit of 16793600 bytes. :
      [js_test:defragment_large_collection] Mongo.prototype.runCommand@src/mongo/shell/mongo.js:167:12
      [js_test:defragment_large_collection] runCommandWithRetries/<@jstests/libs/override_methods/implicitly_retry_on_background_op_in_progress.js:58:19
      [js_test:defragment_large_collection] assert.soon@src/mongo/shell/assert.js:366:21
      [js_test:defragment_large_collection] runCommandWithRetries@jstests/libs/override_methods/implicitly_retry_on_background_op_in_progress.js:54:5
      [js_test:defragment_large_collection] overrideRunCommand/Mongo.prototype.runCommand@jstests/libs/override_methods/override_helpers.js:80:20
      [js_test:defragment_large_collection] runClientFunctionWithRetries@src/mongo/shell/session.js:368:27
      [js_test:defragment_large_collection] runCommand@src/mongo/shell/session.js:468:25
      [js_test:defragment_large_collection] DB.prototype._runCommandImpl@src/mongo/shell/db.js:167:12
      [js_test:defragment_large_collection] DB.prototype.runCommand@src/mongo/shell/db.js:182:16
      [js_test:defragment_large_collection] DBCollection.prototype._dbCommand@src/mongo/shell/collection.js:174:16
      [js_test:defragment_large_collection] executeBatch@src/mongo/shell/bulk_api.js:904:22
      [js_test:defragment_large_collection] Bulk/this.execute@src/mongo/shell/bulk_api.js:935:17
      [js_test:defragment_large_collection] fillChunksToRandomSize@jstests/sharding/libs/defragmentation_util.js:75:30
      [js_test:defragment_large_collection] createFragmentedCollection@jstests/sharding/libs/defragmentation_util.js:10:9
      [js_test:defragment_large_collection] @jstests/sharding/defragment_large_collection.js:43:1
      [js_test:defragment_large_collection] @jstests/sharding/defragment_large_collection.js:10:2
      [js_test:defragment_large_collection] failed to load: jstests/sharding/defragment_large_collection.js

            Assignee:
            allison.easton@mongodb.com Allison Easton
            Reporter:
            allison.easton@mongodb.com Allison Easton
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: