Flush in-memory files when processing checkpoint message

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      For the S3EmitOperator private preview, we'd like to do a simple solution for checkpointing progress.

       

      As a simple temporary solution, we should flush all in-memory files to S3 whenever we process a checkpoint message.

       

      Later on, we will use https://jira.mongodb.org/browse/SERVER-99980 to track changes for the "real" checkpoint logic

            Assignee:
            Harendra Chawla
            Reporter:
            Andrew Chen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: