[SERVER-73536] Account for the size of the outgoing request in bulkWrite sub-batching logic Created: 02/Feb/23 Updated: 29/Oct/23 Resolved: 25/Jun/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.1.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Lingzhi Deng | Assignee: | Kaitlin Mahar |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | milestone-2 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Replication
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Repl 2023-02-20, Repl 2023-03-20, Repl 2023-05-15, Repl 2023-05-29, Repl 2023-06-12, Repl 2023-06-26 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
When constructing the sub-batch bulkWrite command to send to shards, we should accounts for the size of the outgoing request, similar to this. |
| Comments |
| Comment by Githook User [ 23/Jun/23 ] |
|
Author: {'name': 'Kaitlin Mahar', 'email': 'kaitlin.mahar@mongodb.com', 'username': 'kmahar'}Message: |
| Comment by Kaitlin Mahar [ 07/Jun/23 ] |
|
On further inspection, we currently always include all the namespaces from the incoming client request in the sub-batches, and don't factor in whether any writes in the sub-batch target each namespace. So instead we can just calculate up front the size of the nsInfo and factor that into our command size estimation. Per conversation with the QE team, we should also factor in top-level fields in the command which could be quite large, e.g. let (see We should also factor in the size of command metadata e.g. $audit but SERVER-53387 already covers fixing that problem for all commands so we will not attempt to tackle it here. |
| Comment by Kaitlin Mahar [ 26/Apr/23 ] |
|
Wanted to note while I've been looking at this code that making getWriteSizeBytes() work for bulkWrite may be a little tricky. Right now that method only produces estimates that are valid for how inserts, updates, and deletes are represented in the insert/update/delete commands respectively, and the method only has to think about the size of the op itself. |