[SERVER-81063] Streams: Async DLQ inserts with batching Created: 14/Sep/23  Updated: 29/Oct/23  Resolved: 25/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Task Priority: Major - P3
Reporter: Aadesh Patel (Inactive) Assignee: Aadesh Patel (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Atlas Streams
Backwards Compatibility: Fully Compatible
Sprint: Sprint 32
Participants:

 Description   

Right now we synchronously insert one document into the DLQ at a time, we should move the i/o work to a background thread and make all inserts into the DLQ async. The worker thread should push all documents destined for the DLQ into a queue that the DLQ background thread is constantly consuming from.

Ideally the work we do here can be reused to do something similar in the merge operator.

Slack context: https://mongodb.slack.com/archives/C05P740L0VC/p1694540160003889


Generated at Thu Feb 08 06:45:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.