MongoSink Connector is missing messages from Topic

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Gone away
    • Priority: Unknown
    • None
    • Affects Version/s: 1.5.0
    • Component/s: None
    • Environment:
      Confluent Kafka
    • None
    • None
    • None
    • None
    • None
    • None

      Hi ,

      Mongo Sink connector is missing messages (present in mongo changestream and topic in Kafka) from topic .

      i could see the whole message in the topic as well as in the changestream.

      What could be the reason.

      {"_data": "8260CC87A9000000092B022C0100296E5A1004A1F0D3C981B54CD59F248E484B6EEC3746645F6964006460CC4F9343617B00086257740004"},
      

      this token is present, but the sync isnt happening .

      the sync conf:

      {
        "name": "prod-organizationmanagementSink",
        "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector",
        "tasks.max": "1",
        "topics.regex": "^prod.REMOVED.*",
        "connection.uri": "REMOVED",
        "database": "REMOVED,
        "change.data.capture.handler": "com.mongodb.kafka.connect.sink.cdc.mongodb.ChangeStreamHandler"
      }
      
      

      I have removed the sensitive information.

      Its happening for few documents (randomly) not for all .

      Any lead would be appreciated. Let me know if anything else is needed .

      BR//
      Nitin

            Assignee:
            Ross Lawley
            Reporter:
            Nitin Kapoor
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: