[KAFKA-216] Source task should fail when not able to resume change stream Created: 05/Apr/21  Updated: 27/Oct/23  Resolved: 18/May/21

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Nigel Liang Assignee: Ross Lawley
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The current implementation of `MongoSourceTask::tryCreateCursor` ends with

if (resumeTokenNotFound(e)) {
      throw new ConnectException(
          "ResumeToken not found. Cannot create a change stream cursor", e);
    }
  }
  return null;
}

In the event that the cursor was not able to be created for a reason other than the "resume token not found" error, the connector does not fail and instead starts and processes no messages.



 Comments   
Comment by Backlog - Core Eng Program Management Team [ 18/May/21 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by Ross Lawley [ 04/May/21 ]

Hi nigel@nigelliang.com,

Thanks for the ticket. It was requested that the connector makes deploying it as easy as possible. This has been achieved by decoupling the deployment of the kafka system and the MongoDB database/collection. Databases / Collections can be watched before they exist and all change stream messages can be caught.

Is this causing you an issue? or was it more of a surprise that it didn't immediately fail?

Ross

Comment by Esha Bhargava [ 06/Apr/21 ]

nigel@nigelliang.com Thank you for reporting this issue! We'll look into it and get back to you soon.

Generated at Thu Feb 08 09:05:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.