[DRIVERS-2399] Add spec test for large bulk_write ops inside transactions Created: 26/Jul/22 Updated: 26/Jul/22 |
|
| Status: | Backlog |
| Project: | Drivers |
| Component/s: | CRUD |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Unknown |
| Reporter: | Dmitry Rybakov | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Driver Changes: | Not Needed |
| Description |
SummaryWe should add a spec test that ensures that large bulk_write operations inside an explicit transaction work properly. When performing large bulk_writes, the write may internally be split into multiple smaller batches. When the bulk writes are executed inside an explicit transaction, there is a room for errors. For example, there was a user reported bug in Ruby driver, and as a result of this bug such bulk writes always failed - https://jira.mongodb.org/browse/RUBY-3044 MotivationWho is the affected end user?Users who uses large bulk write operations in transactions How does this affect the end user?There is a workaround - splitting the batch into smaller batches on the app side. But it must be very annoying, the driver should take care of this. How likely is it that this problem or use case will occur?We received only one bug report for the Ruby driver regarding this problem, even though this bug existed for quite a long time. So, it seems to be rather a rare case. Is this issue urgent?No. Is this ticket only for tests?This ticket is to create a spec tests. |
| Comments |
| Comment by Jeremy Mikola [ 26/Jul/22 ] | |
|
The closest thing that comes to mind is something like:
However, the main hurdle is that all of our special syntax is relegated to Evaluating Matches for result assertions. None of that applies to handling arguments/parameters for operations, which this would require. Common Options is the only place I'm aware of where we do special handling of parameters, but that's also limited to top-level options. In this case, a test runner would need to process special syntax within a bulkWrite operation's arguments.requests[].insertOne.document field. Presumably, we'd only lead with insertOne. | |
| Comment by Jeffrey Yemin [ 26/Jul/22 ] | |
|
jmikola@mongodb.com can you think about what a syntax for this might look like if we were to add it to the unified test syntax? I can imagine a lot of different tests around batch splitting in addition to this one, and it would be a lot less work to implement if they could all be spec tests. | |
| Comment by Jeremy Mikola [ 26/Jul/22 ] | |
|
Note: this would likely need to be a prose test. We don't have any syntax for specifying large payloads in unified spec tests, nor adjusting the client to use artificially lower values for batch limits. See comments in DRIVERS-2150 for prior discussion on that. |