[KAFKA-128] Mongo source connector offset partition map Created: 12/Jul/20  Updated: 28/Oct/23  Resolved: 03/Sep/20

Status: Closed
Project: Kafka Connector
Component/s: Source
Affects Version/s: None
Fix Version/s: 1.3.0

Type: New Feature Priority: Major - P3
Reporter: Andrey B Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Currently offset creates as a concatenation of connection.uri, database and collection parameters.
https://github.com/mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/source/MongoSourceTask.java#L309-L317

This approach has a couple of disadvantages.

1) such offset could contain mongo username and password, which will be written to kafka connect offsets topic. I guess it's not very secure.
2) if the mongo password or username was changed, the offset will be lost.

Is it make sense to use a static partition name? Or in the future, there is could be more than one partition?



 Comments   
Comment by Githook User [ 01/Sep/20 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Sanitized the connection string in the partion map

KAFKA-128
Branch: master
https://github.com/mongodb/mongo-kafka/commit/01f3a71b724e1c022c490de4a87ec4c50634b7cb

Comment by Githook User [ 28/Aug/20 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Sanitized the connection string in the partion map

KAFKA-128
Branch: k128
https://github.com/mongodb/mongo-kafka/commit/1611cd60f867ea1e96e673e48ae0f90aed9b4ff3

Comment by Ross Lawley [ 24/Aug/20 ]

PR: https://github.com/mongodb/mongo-kafka/pull/31

Generated at Thu Feb 08 09:05:39 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.