[SERVER-53387] Large internal metadata can trigger BSONObjectTooLarge for commands under the BSON size limit Created: 16/Dec/20 Updated: 06/Oct/23 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Jack Mulrow | Assignee: | Backlog - Service Architecture |
| Resolution: | Unresolved | Votes: | 6 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Service Arch
|
||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||
| Story Points: | 3 | ||||||||||||||||||||||||
| Description |
|
Before an internal request is sent, metadata may be added to it through registered hooks (e.g. $clusterTime, $audit). Eventually this metadata is combined with the request's command body into an op msg, which will fail with BSONObjectTooLarge if the combined size of the metadata and command body is over the BSONObjMaxInternalSize limit. This is a problem for commands that are dynamically built within the server that can be near the size limit (like batch writes) because they don't currently account for the exact size of the metadata to be added, which can cause persistent BSONObjectTooLarge errors if the metadata is large enough, like with complicated LDAP roles (which leads to large $audit metadata). |
| Comments |
| Comment by Andrew Palmer (Inactive) [ 24/Feb/21 ] |
|
To add some additional context to this issue w.r.t. Change Events (HELP-22469), we wanted to bring up the potential case (although very unlikely) where there's a 16MB pre-image + 16MB update + 16 MB post-image + metadata in the change event, which will be over the 48MB wire limit. Although there's likely compression done over the wire, we wanted to highlight this case for the team. Thanks all! |
| Comment by Garaudy Etienne [ 18/Dec/20 ] |
|
jack.mulrow what's the downside to option 2? Option 1 seems like additional work. Is that true? |
| Comment by Jack Mulrow [ 16/Dec/20 ] |
|
Two approaches I can think of to solve this:
I prefer option 2), since that should solve this for all internal commands, but I'll leave both up and we can decide when this ticket is triaged. |