[KAFKA-230] Change stream doesn't recover with errors.tolerance=all on Mongo 4 Created: 02/Jun/21 Updated: 28/Oct/23 Resolved: 09/Jul/21 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | Source |
| Affects Version/s: | 1.5.1, 1.6.0 |
| Fix Version/s: | 1.6.0 |
| Type: | Bug | Priority: | Unknown |
| Reporter: | Tim Patterson | Assignee: | Ross Lawley |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | external-user | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Confluent 5.5.4 (kafka 2.4.x I believe) |
||
| Description |
|
Currently if we fall off the end of the oplog for whatever reason we get the following error on mongo 4:
With errors.tolerance=all the source should auto recover starting with a fresh "change stream". However this doesn't happen due to:
This minimal patch is enough to fix for us https://github.com/tim-patterson/mongo-kafka/commit/a281a6c42256efdea6613e5a2a91bca61cfa75c6 (I'm currently chasing up on my companies open source policies so I'm not sure if I'm going to be in a position to submit a fix/PR) |
| Comments |
| Comment by Githook User [ 09/Jul/21 ] |
|
Author: {'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}Message: Fixed Source Connector recovery during getMore call (#79)
|
| Comment by Ross Lawley [ 07/Jul/21 ] |
| Comment by Esha Bhargava [ 04/Jun/21 ] |
|
tpatterson@sailthru.com Thank you for reporting this issue! We'll look into it and get back to you soon. |