[DRIVERS-2150] Add retryable write spec tests for batch split insert, update, and delete operations Created: 11/Mar/19 Updated: 31/Mar/22 |
|
| Status: | Backlog |
| Project: | Drivers |
| Component/s: | Retryability |
| Fix Version/s: | None |
| Type: | Spec Change | Priority: | Major - P3 |
| Reporter: | Kristofer Brandow (Inactive) | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Driver Changes: | Needed | ||||||||||||
| Description |
|
There are currently no spec tests for batch split insert, update, or delete operations that ensure that on failure only the current and remaining batches are retried, not all of the batches. There is currently no fail point that can support these tests since the onPrimaryTransactionalWrite fail point can accept either skip or tries but not both. So an update will need to be made to this fail point (or a new fail point introduced) that has this capability. |
| Comments |
| Comment by Jeremy Mikola [ 12/Mar/19 ] |
kris.brandow: You're not mistaken. Split Batch Tests is describing a subtly of onPrimaryTransactionalWrite with insert commands vs. update/delete. It states that YAML tests for bulk writes may split by command type and discusses how multi-statement writes might be split due to size factors; however, none of the YAML tests exercise those size boundaries. |
| Comment by Kristofer Brandow (Inactive) [ 11/Mar/19 ] |
|
shane.harvey that section looks like it’s describing how the YAML tests work, not prose tests. Am I missing something? jmikola I’ll take a look at failCommand and see if I can get it to do what I’m trying to test here. |
| Comment by Shane Harvey [ 11/Mar/19 ] |
|
There are already prose tests for these cases described in the test readme here: https://github.com/mongodb/specifications/tree/master/source/retryable-writes/tests#split-batch-tests |
| Comment by Jeremy Mikola [ 11/Mar/19 ] |
|
I acknowledge the desire for spec tests here, I don't think it's feasible unless the driver's test suite can configure the server to use artificially lower values for maxBsonObjectSize, maxMessageSizeBytes, and/or maxWriteBatchSize. As-is, we might be able to define a very large number of bulk write requests using YAML anchors but that would still lead to very large JSON files. I'd vote for using prose tests, which would also allow drivers to intelligently verify behavior using APM without spec tests needing to define large command assertions (in addition to the large operation). Changes to onPrimaryTransactionalWrite might also not be necessary if we can instead rely on failCommand to fail individual write commands, which may contain any number of requests. For instance, we can apply failCommand to insert and skip the first command (i.e. batch) and only trigger a failure on the second half of requests sent in the second command. |