[KAFKA-145] org.bson.BsonInvalidOperationException: Value expected to be of type DOCUMENT is of unexpected type NULL Created: 21/Aug/20 Updated: 28/Oct/23 Resolved: 07/Sep/20 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | Source |
| Affects Version/s: | 1.2.0 |
| Fix Version/s: | 1.3.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Rajaramesh Yaramati | Assignee: | Ross Lawley |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
kafka version: 2.6 |
||
| Attachments: |
|
| Description |
|
My source connector terminating with this error. [2020-08-20 20:05:52,666] ERROR WorkerSourceTask{id=mongo-source-assets-mongos-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:187) ] to 10.74.3.104:27017 because the pool has been closed. (org.mongodb.driver.connection:71) Can someone please take a look at this error and tell me if this is something expected? Thanks, Rajaramesh |
| Comments |
| Comment by Gil De Grove [ 25/Sep/20 ] | |||||||||||||||||||||||||||||||
|
We discover the same bug,
We think that this may be related to what is explained in the connector documentation. If you have a lot of update on a document, and have a delete on the document then it is send as a null full_document. I think the check you added will totally allow to prevent this issue.
| |||||||||||||||||||||||||||||||
| Comment by Githook User [ 07/Sep/20 ] | |||||||||||||||||||||||||||||||
|
Author: {'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}Message: Ensure the fullDocument field is a document
| |||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 07/Sep/20 ] | |||||||||||||||||||||||||||||||
|
From that error message, it looks like the source connector is encountering a fullDocument field that is not a document but rather is set as null. I'm not sure in what scenario the field would be published with a null value. What version of MongoDB are you running? I'll add an extra check in 1.3 Ross | |||||||||||||||||||||||||||||||
| Comment by Ross Lawley [ 07/Sep/20 ] | |||||||||||||||||||||||||||||||
|
Hi there, thank you for reaching out. As this sounds like a support issue, I wanted to give you some resources to get this questioned answered more quickly:
Thank you!
| |||||||||||||||||||||||||||||||
| Comment by Rajaramesh Yaramati [ 22/Aug/20 ] | |||||||||||||||||||||||||||||||
|
My source connector config:
|