[SERVER-53387] Large internal metadata can trigger BSONObjectTooLarge for commands under the BSON size limit Created: 16/Dec/20  Updated: 06/Oct/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Jack Mulrow Assignee: Backlog - Service Architecture
Resolution: Unresolved Votes: 6
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Problem/Incident
Related
is related to SERVER-77653 Batch write size estimation logic on ... Open
is related to SERVER-55062 Change stream events can exceed 16MB ... Closed
is related to SERVER-64592 Make sure events with large documentK... Closed
Assigned Teams:
Service Arch
Operating System: ALL
Participants:
Case:
Story Points: 3

 Description   

Before an internal request is sent, metadata may be added to it through registered hooks (e.g. $clusterTime, $audit). Eventually this metadata is combined with the request's command body into an op msg, which will fail with BSONObjectTooLarge if the combined size of the metadata and command body is over the BSONObjMaxInternalSize limit.

This is a problem for commands that are dynamically built within the server that can be near the size limit (like batch writes) because they don't currently account for the exact size of the metadata to be added, which can cause persistent BSONObjectTooLarge errors if the metadata is large enough, like with complicated LDAP roles (which leads to large $audit metadata).



 Comments   
Comment by Andrew Palmer (Inactive) [ 24/Feb/21 ]

To add some additional context to this issue w.r.t. Change Events (HELP-22469), we wanted to bring up the potential case (although very unlikely) where there's a 16MB pre-image + 16MB update + 16 MB post-image + metadata in the change event, which will be over the 48MB wire limit.

Although there's likely compression done over the wire, we wanted to highlight this case for the team. Thanks all!

Comment by Garaudy Etienne [ 18/Dec/20 ]

jack.mulrow what's the downside to option 2? Option 1 seems like additional work. Is that true?

Comment by Jack Mulrow [ 16/Dec/20 ]

Two approaches I can think of to solve this:

  1. Change internally constructed requests to account for the exact size of the metadata to be added instead of using an estimate. We could start with just the commands known to hit this problem, like batch writes.
  2. Allow the combined metadata and command body to be over the max internal BSON size limit and instead only throw BSONObjectTooLarge if either is over the limit on its own. The max size for messages on our wire protocol is 48MB, so sending larger BSON should be possible. This can reuse the large BSON support added in SERVER-36606.

I prefer option 2), since that should solve this for all internal commands, but I'll leave both up and we can decide when this ticket is triaged.

Generated at Thu Feb 08 05:30:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.