[CSHARP-2385] Support lazy enumeration in BulkWrite Created: 13/Sep/18 Updated: 08/Feb/23 Resolved: 14/Dec/20 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | Write Operations |
| Affects Version/s: | 2.7.0 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Daniel Hegener | Assignee: | Unassigned |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
The BulkWrite method takes an IEnumerable for the bulk write requests. There are two issues currently with how the driver handles that IEnumberable. The first is that the IEnumerable is iterated multiple times. That will be covered in scope of CSHARP-1378. The second is that it eagerly iterates the IEnumerable instead of doing so lazily, which would support streaming use cases where the IEnumerable is computed lazily. We will consider this second issue as the scope of this Jira issue, and as such no longer will treat it as a duplicate of CSHARP-1378. Original Description The BulkWrite(Async) methods accept an IEnumerable which, unfortunately, gets enumerated multiple times. Here is the code:
So, I suggest to |
| Comments |
| Comment by Jeffrey Yemin [ 15/Dec/20 ] |
|
Just to clarify: I updated the description to reflect the two aspects of the issue, and so this is no longer considered a duplicate of CSHARP-1378. Still, we've decided for now to close this issue as Won't Fix, as there is implementation complexity in supporting an "infinite" stream of write requests, and we don't think the complexity is warranted given the limited use cases for the feature. Also, no other MongoDB driver supports this use case, so it would make .NET somewhat of an outlier and may limit our flexibility going forward if we were to support this. For applications that require lazy enumeration, it's always an option to implement it on top of the driver by batching requests into a series of BulkWrite operations. This has the benefit of giving applications greater control of batch sizes, which in the driver are hard coded to 1000 requests per batch. |
| Comment by Dmitry Lukyanov (Inactive) [ 14/Dec/20 ] |
|
Supporting lazy enumerating for BulkWrite is not in our road map for now. |
| Comment by Christopher Lombardi [ 07/Nov/19 ] |
|
Despite this being considered a, "duplicate" can we receive a response regarding why the implementation enumerates the entire collection and does not stream and batch during streaming? |
| Comment by Jeffrey Yemin [ 22/Oct/18 ] |
|
Closing as a duplicate of CSHARP-1378. |