[SERVER-63380] JSTest bulk api can create oversized batches Created: 07/Feb/22  Updated: 29/Oct/23  Resolved: 17/Feb/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0.0-rc0, 5.3.0-rc1

Type: Bug Priority: Major - P3
Reporter: Allison Easton Assignee: Allison Easton
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File repro.patch    
Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.3
Steps To Reproduce:

To reproduce the error, apply this patch onto master at commit c99e4833db76bba9842cf9ee9add83aaba0c340a. Run the test jstests/sharding/defragment_large_collection.js in the sharding suite.

repro.patch

Sprint: Sharding EMEA 2022-02-21
Participants:
Linked BF Score: 62

 Description   

The bulk api checks that a document will not make a batch too large at the point of inserting the document into the batch. At the point of building and executing the command, though, any inserts without an id get this field added to them. The resulting document is never checked for size overflow, and can result in the following error.

[js_test:defragment_large_collection] Error: Converting from JavaScript to BSON failed: Object size 16793919 exceeds limit of 16793600 bytes. :
[js_test:defragment_large_collection] Mongo.prototype.runCommand@src/mongo/shell/mongo.js:167:12
[js_test:defragment_large_collection] runCommandWithRetries/<@jstests/libs/override_methods/implicitly_retry_on_background_op_in_progress.js:58:19
[js_test:defragment_large_collection] assert.soon@src/mongo/shell/assert.js:366:21
[js_test:defragment_large_collection] runCommandWithRetries@jstests/libs/override_methods/implicitly_retry_on_background_op_in_progress.js:54:5
[js_test:defragment_large_collection] overrideRunCommand/Mongo.prototype.runCommand@jstests/libs/override_methods/override_helpers.js:80:20
[js_test:defragment_large_collection] runClientFunctionWithRetries@src/mongo/shell/session.js:368:27
[js_test:defragment_large_collection] runCommand@src/mongo/shell/session.js:468:25
[js_test:defragment_large_collection] DB.prototype._runCommandImpl@src/mongo/shell/db.js:167:12
[js_test:defragment_large_collection] DB.prototype.runCommand@src/mongo/shell/db.js:182:16
[js_test:defragment_large_collection] DBCollection.prototype._dbCommand@src/mongo/shell/collection.js:174:16
[js_test:defragment_large_collection] executeBatch@src/mongo/shell/bulk_api.js:904:22
[js_test:defragment_large_collection] Bulk/this.execute@src/mongo/shell/bulk_api.js:935:17
[js_test:defragment_large_collection] fillChunksToRandomSize@jstests/sharding/libs/defragmentation_util.js:75:30
[js_test:defragment_large_collection] createFragmentedCollection@jstests/sharding/libs/defragmentation_util.js:10:9
[js_test:defragment_large_collection] @jstests/sharding/defragment_large_collection.js:43:1
[js_test:defragment_large_collection] @jstests/sharding/defragment_large_collection.js:10:2
[js_test:defragment_large_collection] failed to load: jstests/sharding/defragment_large_collection.js



 Comments   
Comment by Githook User [ 22/Feb/22 ]

Author:

{'name': 'Allison Easton', 'email': 'allison.easton@mongodb.com', 'username': 'allisoneaston'}

Message: SERVER-63380 JSTest bulk api can create oversized batches

(cherry picked from commit 0e576ee656e40fa65995b835e471eb5ff5059ea5)
Branch: v5.3
https://github.com/mongodb/mongo/commit/d3fa3ac59b06100d4bec34c1a5273d805f01f7db

Comment by Githook User [ 17/Feb/22 ]

Author:

{'name': 'Allison Easton', 'email': 'allison.easton@mongodb.com', 'username': 'allisoneaston'}

Message: SERVER-63380 JSTest bulk api can create oversized batches
Branch: master
https://github.com/mongodb/mongo/commit/0e576ee656e40fa65995b835e471eb5ff5059ea5

Generated at Thu Feb 08 05:57:39 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.