Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-87970

Kafka key and header passthrough as BinData

    • Atlas Streams
    • Fully Compatible
    • Sprint 45, Sprint 46

      See this product description: https://docs.google.com/document/d/1bzZUxSZOd47nlc5i25hIqzsx3oS02U_wAt7MCnlWWOk/edit#heading=h.ml9azibz674l

      =====

      In the Scope document, we touched on headers lightly. Since then, It's come up with customers like Acoustic. They want access to the Kafka header information inside the stream. The best logical place is the _stream meta as outlined in the scope doc: https://docs.google.com/document/d/1WGjmWFABCYBRPRWnbgERuwjWcKmvowWU2M76OiSxZu4/edit

      // now we get messages that look like:
      {
        "sensor_id": 12,
        "temperature": 999,
        "_myts": "ada lovelace",
        "_ts": ISODate("2022-10-19T15:37:04.604Z"),
        "_stream_meta": {
           "type": "kafka",
           "key": "sensor_id", 
           "value": "11", 
           "timestamp": 1677855573409, 
           "partition": 4,  
           "offset": 999, 
           "headers":[] // add header information 
         }
      }
       

            Assignee:
            rui.liu@mongodb.com Rui Liu
            Reporter:
            kenny.gorman@mongodb.com Kenny Gorman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: