Flush in-memory files when processing checkpoint message

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • Fully Compatible
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      For the S3EmitOperator private preview, we'd like to do a simple solution for checkpointing progress.

       

      As a simple temporary solution, we should flush all in-memory files to S3 whenever we process a checkpoint message.

       

      Later on, we will use https://jira.mongodb.org/browse/SERVER-99980 to track changes for the "real" checkpoint logic

              Assignee:
              Harendra Chawla
              Reporter:
              Andrew Chen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: