Kafka mongo sink. Create _id from key message

XMLWordPrintableJSON

    • Type: Question
    • Resolution: Works as Designed
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Hello,

      I have the following kafka message: key content is {{

      {orderid: 123}

      }} and the message value as {{{}

      {name: "def", company: "abc"}

      {}}}. I want to insert in mongodb the following document: {_id: 123, name: "def", company: "abc"} where _id takes the value of orderid from the key message. I have tried the following config and it looks ok except that the _id is being an object like this: _id:{orderid: 123}{}. Any idead how we can achieve this? I have the following config:

      name = MongoSinkConnectorConnector_1
connector.class = com.mongodb.kafka.connect.MongoSinkConnector
topics = topicA

key.converter = org.apache.kafka.connect.storage.StringConverter
key.converter.schemas.enable = false

value.converter = org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable = false

connection.uri = mongodb://localhost:27017
database = foo
collection = mycollection

# Use a single message as a document
document.id.strategy=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialKeyStrategy

# Post processing of the sink documents
post.processor.chain=com.mongodb.kafka.connect.sink.processor.DocumentIdAdder

# Document ID strategy using the Partial Key Strategy
document.id.strategy.overwrite.existing=true

# Provide a custom setting for which key values to consider for the _id
document.id.strategy.partial.key.projection.list=orderid,customerid
document.id.strategy.partial.key.projection.type=AllowList

      Thank you!

      Ralph

       

            Assignee:
            Robert Walters (Inactive)
            Reporter:
            Ralph Sawaya
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: