[KAFKA-95] "Unrecognized field: startAfter" with Source connector Created: 06/Apr/20 Updated: 28/Oct/23 Resolved: 08/Apr/20 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | 1.0.1 |
| Fix Version/s: | 1.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Abhishek Gupta | Assignee: | Ross Lawley |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
MongoDB 3.6 |
||
| Description |
|
I am trying to configure a Source connector. Here are the steps
INFO Resuming the change stream after the previous offset (com.mongodb.kafka.connect.source.MongoSourceTask:234) I am using MongoDB 3.6 and I see that startAfter is supported for 4.2 and above (https://docs.mongodb.com/manual/changeStreams/#startafter-for-change-streams). I checked the code[https://github.com/mongodb/mongo-kafka/blob/d89f0c0ba0b8a365fbcc0536dfa93d1728e4d933/src/main/java/com/mongodb/kafka/connect/source/MongoSourceTask.java#L253] but it's difficult to figure out since all the exception handling is based on error codes. Not sure why startAfter is coming into picture in this case |
| Comments |
| Comment by Abhishek Gupta [ 12/Apr/20 ] |
|
Thank you! Good to see its included in 1.1 release. Out of curiosity, when is the 1.1 release going out? |
| Comment by Ross Lawley [ 08/Apr/20 ] |
|
I've added an extra check for errorCode 9 - Unrecognized field. This should ensure that resumeAfter is used instead. |
| Comment by Githook User [ 08/Apr/20 ] |
|
Author: {'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}Message: Updated startAfter support check
|
| Comment by Ross Lawley [ 07/Apr/20 ] |
|
I've tested on both 3.6.10 and 3.6.17 and I don't the "Unrecognized field: startAfter 9 " error. Please could you confirm the exact version of MongoDB you are using? Kind Regards, Ross |
| Comment by Ross Lawley [ 07/Apr/20 ] |
|
Hi, Apologies, that is definitely a bug with the handling of resuming a changestream. Restarting the connector will attempt to restart from the last seen offset. However, MongoDB 3.6 handles the startAfter field differently than expected - a fix will be issued in the next release. Ross |