[KAFKA-352] The data obtained through the connector is an un-parsable json string with " and \ and escape symbols, not the JSON string obtained by json.toString() Created: 14/Feb/23  Updated: 27/Oct/23  Resolved: 07/Apr/23

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: 1.9.1
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: 晓阳 刘 Assignee: Ross Lawley
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Original title: 通过连接器获取的数据,是带有“ and \ 和转义符号的无法解析的json字符串,而不是json.tostring()所获取的jsonString

This is the description of the source connection:

{"connector.class":"com.mongodb.kafka.connect.MongoSourceConnector",
 "mongo.errors.deadletterqueue.topic.name":"conn_error_data",
  "publish.full.document.only":"false", 
  " mongo.errors.log.enable":"true",
  "startup.mode":"copy_existing",
  "tasks.max":"2",
  "change.stream.full.document":"updateLookup",
  "collection" :"",
  "mongo.errors.tolerance":"all",
  "database":"****",
  "topic.prefix":"mongo-cdc1",
  "topic.separator":"_",
  " connection.uri":"mongodb://user:pass@mongo1:27017,mongo2:27017,mongo3:27017" ,
  “姓名”:"mongo-cdc-chatbot"}

以下是获取的数据: (The following is the data obtained )

“{\“_ID\”: {\"_data\": \"8263EA11F300000A002B022C0100296E5A10044626960F9B9E4F8A8CCB5B7FB645F88746645F6964006463E30D187FA0165B25002E010004\"}
, \"operationType\": \"delete\", \"clusterTime\": \{\"$timestamp\": {\"t\": 1676284403, \"i\": 2560}}, \"ns\ ": \{\"db\": \"chatbot\", \"coll\": \"test\"}, \"documentKey\": {\"_id\":
{\"$oid\": \"63e30d187fa0165b25002e01\"}}}"

When parsing:

通过BSONDocumeng.parse(str)解析是出现的错误:
 
readStartDocument 只能在 CurrentBSONType 为 DOCUMENT 时调用,不能在 CurrentBSONType 为 STRING 时调用。



 Comments   
Comment by PM Bot [ 07/Apr/23 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by PM Bot [ 13/Mar/23 ]

Hey liuxiaoyang904.61@gmail.com, We need additional details to investigate the problem. If this is still an issue for you, please provide the requested information.

Comment by 晓阳 刘 [ 06/Mar/23 ]

问题已经解决掉了,异常情况是因为Kafka connect 转换器的问题导致的,非常感谢你的提醒。
将json转换器更改为string 转换器就正常了,希望你们可以将此问题更新至文档,应该可以减少一些不必要的问题出现。
----------------------------------------------------------------------------------------------------------------------------------
The problem has been solved. The abnormal situation is caused by the problem of the Kafka connect converter. Thank you for your reminder.
It is normal to change the json converter to a string converter. I hope you can update this issue to the document, which should reduce some unnecessary problems.

Comment by 晓阳 刘 [ 28/Feb/23 ]

很抱歉,我间隔这么长时间才回复。我使用的kafka连接器转换器是org.apache.kafka.connect.json.JsonConverter,是之前使用debezium是配置的,我在启动mongo-kafka-conn时并未改动这个参数。
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sorry for taking so long to reply. The kafka connector converter I use is org.apache.kafka.connect.json.JsonConverter, which was configured using debezium before. I did not change this parameter when I started mongo-kafka-conn.

Comment by Ross Lawley [ 17/Feb/23 ]

Hi liuxiaoyang904.61@gmail.com,

Thank you for the ticket. I will include a google translate copy of my response hopefully it will work!

Could you let me know what schema you have set and what kafka converters you are using?

Ross


感谢您的票(jira 票)。我会附上我的回复的谷歌翻译副本,希望它有效!

你能告诉我你设置了什么模式(Schemata)吗?你使用的是什么kafka转换器?

罗斯

Comment by Jeffrey Yemin [ 16/Feb/23 ]

Google translate of the description:

"The data obtained through the connector is an un-parsable json string with " and \ and escape symbols, not the JSON string obtained by json.toString()"

Generated at Thu Feb 08 09:06:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.