New id strategy that doesn't reuse configuration parameters.

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Major - P3
    • 1.2.0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Needed
    • Hide

      `PartialValueStrategy` and `PartialKeyStrategy` now have namespaced configurations. So use:

      `document.id.strategy.partial.value.projection.type`, `document.id.strategy.partial.value.projection.list`,
      `document.id.strategy.partial.key.projection.type` and `document.id.strategy.partial.key.projection.list`.

      instead of [key|value].projection.[type|list].

      Remove note about PROJECTION POST PROCESSORS ARE NOT COMPATIBLE WITH PARTIALVALUESTRATEGY

      Show
      `PartialValueStrategy` and `PartialKeyStrategy` now have namespaced configurations. So use: `document.id.strategy.partial.value.projection.type`, `document.id.strategy.partial.value.projection.list`, `document.id.strategy.partial.key.projection.type` and `document.id.strategy.partial.key.projection.list`. instead of [key|value] .projection. [type|list] . Remove note about PROJECTION POST PROCESSORS ARE NOT COMPATIBLE WITH PARTIALVALUESTRATEGY
    • None
    • None
    • None
    • None
    • None
    • None

      We want to support a solution to config more than one processor on the chain of post processors.

      Consider the following example:

        "topic.override.source.document.id.strategy":"com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy",
     "topic.override.source.collection":"sink",
     "topic.override.source.value.projection.type":"whitelist",
     "topic.override.source.value.projection.list":"attuid",
     "topic.override.source.writemodel.strategy": "com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy",

     "topic.override.source.post.processor.chain":"com.mongodb.kafka.connect.sink.processor.WhitelistValueProjector",
     "topic.override.source.collection":"sink",
     "topic.override.source.value.projection.type":"whitelist",
     "topic.override.source.value.projection.list":"attuid, name, pc",
     "topic.override.source.batch.size":"100","name":"mongo-sink"}

      Currently, this fails to update correctly the MongoDB documents. The current design only allows one processor to be configured as it is not possible to configure settings for each of the processors.

      This work is to implement a new id strategy that does not reuse configuration parameters.

            Assignee:
            Ross Lawley
            Reporter:
            Seth Payne
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: