[KAFKA-285] Mongo source connector ads binary symbols before json Created: 26/Jan/22  Updated: 28/Oct/23  Resolved: 04/Feb/22

Status: Closed
Project: Kafka Connector
Component/s: docker
Affects Version/s: None
Fix Version/s: 1.7.0

Type: Bug Priority: Minor - P4
Reporter: Gleb Abroskin Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2022-01-25 at 15.20.55.png    

 Description   

To reproduce:

cd docker
./run.sh
# manually add any records into the pageviews table

Then, in connect web UI go the mongo.test.pageviews and I see there very strange behaviour (pic in attachments), although I have not changed anything in the example.

TI've checked the message in the kafka itself in contains 4 null characters and 3 other binary ones.



 Comments   
Comment by Ross Lawley [ 04/Feb/22 ]

Thanks gabroskin@fun.co, for the ticket and fix.

Comment by Githook User [ 04/Feb/22 ]

Author:

{'name': 'Gleb Abroskin', 'email': 'abroskingleb@gmail.com', 'username': 'gasabr'}

Message: fixed source setup in docker example (#99)

Added json key and value converter to the MongoDB Kafka
Source Connector, the default Avro converter does not
handle mongo jsons properly adding binary values in front
of each key and value. The behaviour could be verified
by running ./run.sh from the docker folder

KAFKA-285

Co-authored-by: Gleb Abroskin <gabroskin@fun.co>
Branch: master
https://github.com/mongodb/mongo-kafka/commit/df312f655ae44a2472d29baa0be25c051e1ede70

Comment by Gleb Abroskin [ 26/Jan/22 ]

I've found the error in the source configuration and submitted PR https://github.com/mongodb/mongo-kafka/pull/99

Comment by Ross Lawley [ 26/Jan/22 ]

Hi gabroskin@fun.co

Thank you for reaching out. As this sounds like a support issue, I wanted to give you some resources to get this question answered more quickly:

  • Our MongoDB support portal, located at support.mongodb.com
  • Our MongoDB community portal, located here
  • If you are an Atlas customer, there is free support offered 24/7 in the lower right hand corner of the UI

Just in case you have already opened a support case and are not receiving sufficient help, please let me know and I can facilitate escalating your issue.

If you are yet to open a support or community thread - please include your kafka configs - especially with regards to the converters you are using for the key / value.

All the best,

Ross Lawley

Generated at Thu Feb 08 09:06:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.