Summary

The CRUD specifications do not clearly define how to batch documents for insert, delete, and update operations to handle cases where a document exceeds the maximum size of ~16MB in a non-initial batch, particularly when advancing past the first partition of the batch.

For example, when batching for an insert, the Go Driver partitions documents into ~16MB parts (this is a bug in the Go Driver, it should be 48 MB): p1, p2, p3, ... . If a part pi (where i > 1) fails with ErrDocumentTooLarge, then the operation terminates. In this case, partitions 1 through i - 1 would be inserted successfully, but the user would receive an error for partition i. In this case, should we still return a set of inserted IDs? Is this behavior generally acceptable?

Here is a repro using the Go Driver: https://gist.github.com/prestonvasquez/e28544554a872f4115389f1909f18ccd

Output:

2024/10/30 10:41:51 err:  an inserted document is too large
found:  100 

Motivation

Who is the affected end user?

Any drivers that validate CRUD batches as they are advanced for operation.

How does this affect the end user?

In the described scenario, users could insert a subset of the intended data before receiving an error noting that they are trying to insert a document that is too large. 

How likely is it that this problem or use case will occur?

Unknown but definitely possible

If the problem does occur, what are the consequences and how severe are they?

If analogues to ErrDocumentTooLarge are not associated with a list of inserted IDs, such as the Go Driver behavior, it can be quite annoying to avoid duplicating data when re-attempting the write.

Is this issue urgent?

NA

Is this ticket required by a downstream team?

No

Is this ticket only for tests?

No

Acceptance Criteria

• Define rules for batch splitting for insertMany and collection.bulkWrite. For collection.bulkWrite, this should include rules for grouping together writes of the same kind when unordered:true is specified.
• Define rules for reporting inserted IDs in the results of multi-write operations. See here for guidance.
• Add unified and prose tests that validate this behavior.

The implementer should reference existing rules for batch-splitting and error handling in the client bulk write specification.