[SERVER-13013] write command overhead too large for large batch sizes Created: 03/Mar/14  Updated: 11/Jul/16  Resolved: 10/Mar/14

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 2.6.0-rc0
Fix Version/s: 2.6.0-rc2

Type: Bug Priority: Blocker - P1
Reporter: Mathias Stearn Assignee: Greg Studer
Resolution: Done Votes: 0
Labels: 26qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-13180 upgradeCheck hits BSONObj limit when ... Closed
Operating System: ALL
Participants:

 Description   

The spec says that drivers should only consider the "body" when sizing batches. For example, that means that for {delete: 'db.collection', deletes: [{q:{query: 1}, limit:0}]}}} only the size of {query: 1} counts to the size of the batch. The problem is that if you do 1000 (max ops per batch) deletes in a batch, the overhead that doesn't count for max batch size adds up to more than the difference between 16MB and BSONObjMaxInternalSize. This leads to an assert on the server for what is specified to be a valid operation. The assert is in a place that the server decides that there is irreparable network corruption and closes the connection.

I spoke with greg_10gen and we think the best solution would be to change the write-commands spec to say that either the entire command object with all overhead must fit under 16MB or it must only have a single op in the operations array.



 Comments   
Comment by Githook User [ 07/Mar/14 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-13013 Raise js->bson size limit to BSONObjMaxInternalSize

This is to allow write commands that include objects with BSONObjMaxUserSize.
Branch: master
https://github.com/mongodb/mongo/commit/7240d05b07aebe766a9490a31459c6112620a6f6

Comment by Greg Studer [ 05/Mar/14 ]

The shell also needs to change here and use the new size check.

Comment by Greg Studer [ 04/Mar/14 ]

Confusingly, the bulk api spec says something different that results in this not being a problem - each batch is sized until it hits 16MB + some overhead and then a new batch is broken off:

https://github.com/10gen/specifications/blob/master/source/driver-fluent-api.rst

Proposing a code review that fixes the write command spec to more closely match the fluent spec.

Generated at Thu Feb 08 03:30:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.