Details
-
Question
-
Resolution: Gone away
-
Unknown
-
None
-
1.5.0
-
None
Description
We are trying to leverage Mongo Source Connector for copying the data from a collection.
Below configuration was used and it captures only the changed documents and not the documents from beginning.
// Configuration
|
|
|
{
|
"name": "MongoSourceConnectorConnector_0", |
"config": { |
"connector.class": "com.mongodb.kafka.connect.MongoSourceConnector", |
"key.converter": "org.apache.kafka.connect.storage.StringConverter", |
"value.converter": "org.apache.kafka.connect.storage.StringConverter", |
"connection.uri": "", |
"database": "demo", |
"collection": "identity", |
"pipeline": "[ { '$match': { 'operationType': {'$in': ['insert', 'update', 'replace'], } } }, { '$project': { '_id': 1, 'fullDocument': 1, 'ns': 1, } } ]", |
"publish.full.document.only": "true", |
"topic.namespace.map": "{\"*\":\"demo.identity\"}", |
"copy.existing": "true" |
}
|
}
|
Is there a provision for us to copy the data from the start and do we have any restriction of how far in the past it can support.