[SERVER-12305] Allow command request and response BSON objects to exceed 16MB Created: 09/Jan/14  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Write Ops
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Greg Studer Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 12
Labels: asya, needs-scope, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File big-glemode.js    
Issue Links:
Depends
is depended on by DRIVERS-170 Upserts via write commands should all... Closed
Duplicate
is duplicated by SERVER-27013 Allow results of the aggregation fram... Closed
is duplicated by SERVER-28647 group() can't handle more than 20000 ... Closed
Related
related to SERVER-12598 Cannot upsert 16MB document from shell Closed
related to CDRIVER-589 Driver should sanity check the size o... Closed
is related to SERVER-12275 Add MaxWriteBatchSize to isMaster res... Closed
is related to SERVER-5937 Remove document size limits by transp... Open
is related to SERVER-5923 Increase max document size to at leas... Closed
Assigned Teams:
Query Execution
Participants:
Case:

 Description   

This is due to limitations in both the shell and command processing. This could be useful for things like allowing large write batches or allowing users to compute result documents larger than 16MB using the aggregation framework.

The correct fix here is probably to remove the generic BSONObj limit entirely, and only enforce the limit when serializing to disk (16MB) or the network (48MB).



 Comments   
Comment by Asya Kamsky [ 10/May/18 ]

Another use case - returning a large result to client in response to a command that gathers responses from many shards (in cases where it's not feasible to to return them as a cursor, example: explain on large sharded cluster with one of the more verbose options).

 

Comment by A. Jesse Jiryu Davis [ 03/Mar/14 ]

Another example: Inserting a nearly-16MB document with the shell's Bulk API succeeds with w: 1, but fails with a 100-character-long getLastErrorMode name:

Error: Converting from JavaScript to BSON failed: Object size 16777334 exceeds limit of 16777216 bytes.

Comment by Andreas Nilsson [ 03/Mar/14 ]

This is a more general problem than write commands.

Running the following command with a > 16MB length bigString

db.createRole({role:bigString,roles:[],privileges:[]});

gives the following error:

Error: Converting from JavaScript to BSON failed: Object size 16777284 exceeds limit of 16777216 bytes. at src/mongo/shell/query.js:81

Allthough the command would fail on the server it fails in the shell for the "wrong" reasons.

Generated at Thu Feb 08 03:28:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.