Kafka key and header passthrough as BinData

XMLWordPrintableJSON

    • Atlas Streams
    • Fully Compatible
    • Sprint 45, Sprint 46
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      See this product description: https://docs.google.com/document/d/1bzZUxSZOd47nlc5i25hIqzsx3oS02U_wAt7MCnlWWOk/edit#heading=h.ml9azibz674l

      =====

      In the Scope document, we touched on headers lightly. Since then, It's come up with customers like Acoustic. They want access to the Kafka header information inside the stream. The best logical place is the _stream meta as outlined in the scope doc: https://docs.google.com/document/d/1WGjmWFABCYBRPRWnbgERuwjWcKmvowWU2M76OiSxZu4/edit

      // now we get messages that look like:
      {
        "sensor_id": 12,
        "temperature": 999,
        "_myts": "ada lovelace",
        "_ts": ISODate("2022-10-19T15:37:04.604Z"),
        "_stream_meta": {
           "type": "kafka",
           "key": "sensor_id", 
           "value": "11", 
           "timestamp": 1677855573409, 
           "partition": 4,  
           "offset": 999, 
           "headers":[] // add header information 
         }
      }
       

              Assignee:
              Rui Liu
              Reporter:
              Kenny Gorman
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: