Use BSONObj for queued documents for S3 emit operator

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.2.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Atlas Streams
    • Fully Compatible
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      We had to make a design decision that essentially puts a tradeoff on the usability of trigger.bytes and performance.

       

      Storing docs as strings allows us to implement trigger.bytes to be based on the output size that the user actually sees (without compression). However, that comes at the cost of needing to join potentially large strings together.

       

      Engineering had a discussion with Product and decided that it was better to prioritize performance. Thus, we will make a quick change to switch the internal representation of documents within the S3 emit operator to use BSONObj

       

              Assignee:
              Andrew Chen
              Reporter:
              Andrew Chen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: