Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-394

Delete.on.null.values has no effect

    XMLWordPrintableJSON

Details

    • Icon: Question Question
    • Resolution: Works as Designed
    • Icon: Unknown Unknown
    • None
    • 1.11.1
    • None
    • None

    Description

      Tombstone messages do not trigger a deletion even if `delete.on.null.values` is set to true. 

      Configuration:

      topics: "inputTopic"
      connection.uri: "mongodb://blafoo:27017"
      key.converter: "org.apache.kafka.connect.storage.StringConverter"
      value.converter: "org.apache.kafka.connect.storage.StringConverter"
      value.converter.schemas.enable: false
      key.converter.schemas.enable: false
      transforms: "PrepareId, PrepareValue"
      transforms.PrepareId.type: "org.apache.kafka.connect.transforms.HoistField$Key"
      transforms.PrepareId.field: "_id"
      transforms.PrepareValue.type: "org.apache.kafka.connect.transforms.HoistField$Value"
      transforms.PrepareValue.field: "value"
      document.id.strategy: "com.mongodb.kafka.connect.sink.processor.id.strategy.ProvidedInKeyStrategy"
      document.id.strategy.overwrite.existing: true
      delete.on.null.values: true
      database: "test"
      collection: "test" 

      How to reproduce:

       

      1. Send a kafka record with key: "test" and value: "test" to the inputTopic
        1. We see the following entry in MongoDB {"_id": "test, "value": "test"}
      2. Send a kafka record with key: "test" and value: null to the inputTopic 
        1. We see the following entry in MongoDB {"_id": "test, "value": null}

      Expected behaviour:

      After ** step 2 we would expect that we do not find any entry with {"_id": "test", ...} in MongoDB.

      Attachments

        Activity

          People

            ross@mongodb.com Ross Lawley
            fabianx.bell@gmail.com Fabian Bell
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: