Kafka key and header passthrough as BinData

XMLWordPrintableJSON

    • Atlas Streams
    • Fully Compatible
    • Sprint 45, Sprint 46
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      See this product description: https://docs.google.com/document/d/1bzZUxSZOd47nlc5i25hIqzsx3oS02U_wAt7MCnlWWOk/edit#heading=h.ml9azibz674l

      =====

      In the Scope document, we touched on headers lightly. Since then, It's come up with customers like Acoustic. They want access to the Kafka header information inside the stream. The best logical place is the _stream meta as outlined in the scope doc: https://docs.google.com/document/d/1WGjmWFABCYBRPRWnbgERuwjWcKmvowWU2M76OiSxZu4/edit

      // now we get messages that look like:
{
  "sensor_id": 12,
  "temperature": 999,
  "_myts": "ada lovelace",
  "_ts": ISODate("2022-10-19T15:37:04.604Z"),
  "_stream_meta": {
     "type": "kafka",
     "key": "sensor_id", 
     "value": "11", 
     "timestamp": 1677855573409, 
     "partition": 4,  
     "offset": 999, 
     "headers":[] // add header information 
   }
}

            Assignee:
            Rui Liu
            Reporter:
            Kenny Gorman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: