[DRIVERS-2150] Add retryable write spec tests for batch split insert, update, and delete operations Created: 11/Mar/19  Updated: 31/Mar/22

Status: Backlog
Project: Drivers
Component/s: Retryability
Fix Version/s: None

Type: Spec Change Priority: Major - P3
Reporter: Kristofer Brandow (Inactive) Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to GODRIVER-871 Retryable Writes is broken for batch ... Closed
Driver Changes: Needed

 Description   

There are currently no spec tests for batch split insert, update, or delete operations that ensure that on failure only the current and remaining batches are retried, not all of the batches.

There is currently no fail point that can support these tests since the onPrimaryTransactionalWrite fail point can accept either skip or tries but not both. So an update will need to be made to this fail point (or a new fail point introduced) that has this capability.



 Comments   
Comment by Jeremy Mikola [ 12/Mar/19 ]

that section looks like it’s describing how the YAML tests work, not prose tests

kris.brandow: You're not mistaken. Split Batch Tests is describing a subtly of onPrimaryTransactionalWrite with insert commands vs. update/delete. It states that YAML tests for bulk writes may split by command type and discusses how multi-statement writes might be split due to size factors; however, none of the YAML tests exercise those size boundaries.

Comment by Kristofer Brandow (Inactive) [ 11/Mar/19 ]

shane.harvey that section looks like it’s describing how the YAML tests work, not prose tests. Am I missing something?

jmikola I’ll take a look at failCommand and see if I can get it to do what I’m trying to test here.

Comment by Shane Harvey [ 11/Mar/19 ]

There are already prose tests for these cases described in the test readme here: https://github.com/mongodb/specifications/tree/master/source/retryable-writes/tests#split-batch-tests

Comment by Jeremy Mikola [ 11/Mar/19 ]

I acknowledge the desire for spec tests here, I don't think it's feasible unless the driver's test suite can configure the server to use artificially lower values for maxBsonObjectSize, maxMessageSizeBytes, and/or maxWriteBatchSize. As-is, we might be able to define a very large number of bulk write requests using YAML anchors but that would still lead to very large JSON files.

I'd vote for using prose tests, which would also allow drivers to intelligently verify behavior using APM without spec tests needing to define large command assertions (in addition to the large operation). Changes to onPrimaryTransactionalWrite might also not be necessary if we can instead rely on failCommand to fail individual write commands, which may contain any number of requests. For instance, we can apply failCommand to insert and skip the first command (i.e. batch) and only trigger a failure on the second half of requests sent in the second command.

Generated at Thu Feb 08 08:24:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.