Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-214

Allow for partitioning based on documentId

    • Type: Icon: Improvement Improvement
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Needed
    • Hide

      Presumably, documentation would be needed to explain how to configure it.

      Show
      Presumably, documentation would be needed to explain how to configure it.

      The problem:  

      If your source is publishing to a topic with more than 1 partition, we cannot guarantee that sinks will consume a document's changes in the order in which they occur.  Create could go to partition 0, update to 1, and delete to 2.  A slow sink on 0 and a fast sink on 2 means that we could get a delete before a create.

      If we could assign the partition based on the documentId, we could ensure that a given document is processed in the right order.  

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            brian.begy@kazoohr.com Brian Begy
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: