[KAFKA-99] Support relaxed Json format in the source connector Created: 21/Apr/20  Updated: 28/Oct/23  Resolved: 27/Jul/20

Status: Closed
Project: Kafka Connector
Component/s: Source
Affects Version/s: None
Fix Version/s: 1.3.0

Type: Improvement Priority: Major - P3
Reporter: Ross Lawley Assignee: Ross Lawley
Resolution: Fixed Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
Duplicate
is duplicated by KAFKA-97 Configuration for MongoDB Kafka Conne... Closed
Related
Epic Link: Support schema for Source connector
Case:
Documentation Changes Summary:

Add output.json.formatter configuration.

output.format=[bson,json,schema]

// Json specific configuration
output.json.formatter=[extendedJson,simplifiedJson,someUserProvidedClass]

From the Spec: Json formatter
For Json strings, to provide the user more flexibility the following Json writer settings providers will be introduced:

  • extendedJson: Canonical extended json
  • simplifiedJson: as relaxed extended json but with the following conversions:
  • ObjectId: Hex string
  • Symbol: String
  • Decimal: String
  • DateTimes: ISO formatted String: yyyy-MM-ddTHH:mm:ss.SSSz
  • Binary: Base64 string
  • someUserProvidedClass: Users will be able to implement their own settings if desired and provide the path to the class.

 Description   

Extended Json has a relaxed format which makes numbers more accessible.

Support relaxed numbers, dates, objectIds and binary values in the Json output. So the data is consumable by non MongoDB services.



 Comments   
Comment by Githook User [ 27/Jul/20 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Added configurable json formatting

New configuration option: output.json.formatter

There are 3 available formatters:

  • com.mongodb.kafka.connect.source.json.formatter.DefaultJson: The legacy strict json formatter
  • com.mongodb.kafka.connect.source.json.formatter.ExtendedJson: The fully type safe extended json formatter
  • com.mongodb.kafka.connect.source.json.formatter.SimplifiedJson: Simplified Json
    with ObjectId, Decimals, Dates and Binary values all represented as strings.

Users can also provide their own implementation of the com.mongodb.kafka.connect.source.json.formatter
which then must be added to the classpath.

KAFKA-99
Branch: master
https://github.com/mongodb/mongo-kafka/commit/80ced36c266df2eac4737c1c8a954050fa08261d

Comment by Ross Lawley [ 21/Jul/20 ]

PR: https://github.com/rozza/mongo-kafka/pull/27

Generated at Thu Feb 08 09:05:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.