[SERVER-13013] write command overhead too large for large batch sizes Created: 03/Mar/14 Updated: 11/Jul/16 Resolved: 10/Mar/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Write Ops |
| Affects Version/s: | 2.6.0-rc0 |
| Fix Version/s: | 2.6.0-rc2 |
| Type: | Bug | Priority: | Blocker - P1 |
| Reporter: | Mathias Stearn | Assignee: | Greg Studer |
| Resolution: | Done | Votes: | 0 |
| Labels: | 26qa | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
The spec says that drivers should only consider the "body" when sizing batches. For example, that means that for {delete: 'db.collection', deletes: [{q:{query: 1}, limit:0}]}}} only the size of {query: 1} counts to the size of the batch. The problem is that if you do 1000 (max ops per batch) deletes in a batch, the overhead that doesn't count for max batch size adds up to more than the difference between 16MB and BSONObjMaxInternalSize. This leads to an assert on the server for what is specified to be a valid operation. The assert is in a place that the server decides that there is irreparable network corruption and closes the connection. I spoke with greg_10gen and we think the best solution would be to change the write-commands spec to say that either the entire command object with all overhead must fit under 16MB or it must only have a single op in the operations array. |
| Comments |
| Comment by Githook User [ 07/Mar/14 ] |
|
Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}Message: This is to allow write commands that include objects with BSONObjMaxUserSize. |
| Comment by Greg Studer [ 05/Mar/14 ] |
|
The shell also needs to change here and use the new size check. |
| Comment by Greg Studer [ 04/Mar/14 ] |
|
Confusingly, the bulk api spec says something different that results in this not being a problem - each batch is sized until it hits 16MB + some overhead and then a new batch is broken off: https://github.com/10gen/specifications/blob/master/source/driver-fluent-api.rst Proposing a code review that fixes the write command spec to more closely match the fluent spec. |