-
Type: Task
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 1.1
-
Component/s: None
-
None
-
Environment:MongoDB: 3.6.17, Kafka Connector 1.1
-
(copied to CRM)
Hi, I am using MongoDB: 3.6.17 in my setup. since MongoDB version < 4.0 does not support change streams in database level. I am registering multiple connector instances on my setup. I have 10 connector instances listening to 10 collections in my setup. I try to load around 5000 documents in the collections. After some time I start seeing failures like. and the connector fails and the related Kafka topic does not only partial messages.
Also when I look into the connector status in curl -X GET http://localhost:8083/connectors/mongo-source-2/status the result says its running. but the connector is in a failed state.
2020-03-30 17:00:15,513] WARN [Producer clientId=confluent.monitoring.interceptor.connector-producer-mongo-source-7-0] Error while fetching metadata with correlation id 3 : {_confluent-monitoring=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
com.mongodb.MongoQueryException: Query failed with error code 136 and error message 'Error in $cursor stage :: caused by :: errmsg: "CollectionScan died due to failure to restore tailable cursor position. Last seen record id: RecordId(6810047002907247540)"' on server mongo1:27017
com.mongodb.MongoQueryException: Query failed with error code 136 and error message 'Error in $cursor stage :: caused by :: errmsg: "CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(6810046998612279328)"' on server mongo1:27017
com.mongodb.MongoQueryException: Query failed with error code 136 and error message 'Error in $cursor stage :: caused by :: errmsg: "CollectionScan died due to failure to restore tailable cursor position. Last seen record id: RecordId(6810047020087116556)"' on server mongo1:27017
com.mongodb.MongoQueryException: Query failed with error code 136 and error message 'Error in $cursor stage :: caused by :: errmsg: "CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(6810046998612281250)"' on server mongo1:27017
com.mongodb.MongoQueryException: Query failed with error code 136 and error message 'Error in $cursor stage :: caused by :: errmsg: "CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(6810047015792149212)"' on server mongo1:27017
com.mongodb.MongoQueryException: Query failed with error code 136 and error message 'Error in $cursor stage :: caused by :: errmsg: "CollectionScan died due to position in capped collection being deleted. Last seen record id: RecordId(6810047020087116658)"' on server mongo1:27017
- duplicates
-
KAFKA-76 Reuse the postBatchResumeToken
- Closed