Uploaded image for project: 'Kafka Connector'
  1. Kafka Connector
  2. KAFKA-272

Allow connector to only do Copy Existing and no Change Streams

    XMLWordPrintableJSON

Details

    • Icon: New Feature New Feature
    • Resolution: Works as Designed
    • Icon: Unknown Unknown
    • None
    • None
    • None

    Description

      We have to read a lot of historic data (2+ TB) and it is taking too long because of CPU limitations (can't get connector to use all 4 cores), so we are thinking about having multiple instances of the connector to import different portions of the historic data but only one instance to do the change streams but we can't find a clean way to configure this since it seems all instances would automatically switch to the change stream once they are done with their corresponding Copy Existing portion. Thanks for any advice or help you can give us with this.

      Attachments

        Activity

          People

            robert.walters@mongodb.com Robert Walters
            daniel@haystack.tv Daniel Barreto
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: