[DRIVERS-2399] Add spec test for large bulk_write ops inside transactions Created: 26/Jul/22  Updated: 26/Jul/22

Status: Backlog
Project: Drivers
Component/s: CRUD
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: Dmitry Rybakov Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Driver Changes: Not Needed

 Description   

Summary

We should add a spec test that ensures that large bulk_write operations inside an explicit transaction work properly.

When performing large bulk_writes, the write may internally be split into multiple smaller batches. When the bulk writes are executed inside an explicit transaction, there is a room for errors. For example, there was a user reported bug in Ruby driver, and as a result of this bug such bulk writes always failed - https://jira.mongodb.org/browse/RUBY-3044

Motivation

Who is the affected end user?

Users who uses large bulk write operations in transactions

How does this affect the end user?

There is a workaround - splitting the batch into smaller batches on the app side. But it must be very annoying, the driver should take care of this.

How likely is it that this problem or use case will occur?

We received only one bug report for the Ruby driver regarding this problem, even though this bug existed for quite a long time. So, it seems to be rather a rare case.

Is this issue urgent?

No.

Is this ticket only for tests?

This ticket is to create a spec tests.



 Comments   
Comment by Jeremy Mikola [ 26/Jul/22 ]

The closest thing that comes to mind is something like:

{ $$repeat: { value: "a", times: 1000 }}

However, the main hurdle is that all of our special syntax is relegated to Evaluating Matches for result assertions. None of that applies to handling arguments/parameters for operations, which this would require.

Common Options is the only place I'm aware of where we do special handling of parameters, but that's also limited to top-level options. In this case, a test runner would need to process special syntax within a bulkWrite operation's arguments.requests[].insertOne.document field. Presumably, we'd only lead with insertOne.

Comment by Jeffrey Yemin [ 26/Jul/22 ]

jmikola@mongodb.com can you think about what a syntax for this might look like if we were to add it to the unified test syntax? I can imagine a lot of different tests around batch splitting in addition to this one, and it would be a lot less work to implement if they could all be spec tests.

Comment by Jeremy Mikola [ 26/Jul/22 ]

Note: this would likely need to be a prose test. We don't have any syntax for specifying large payloads in unified spec tests, nor adjusting the client to use artificially lower values for batch limits. See comments in DRIVERS-2150 for prior discussion on that.

Generated at Thu Feb 08 08:25:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.