[KAFKA-45] Sink connector expects both the key and value to be convertible to a Bson Document Created: 18/Jul/19  Updated: 28/Oct/23  Resolved: 06/Apr/20

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1

Type: Improvement Priority: Critical - P2
Reporter: Ross Lawley Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Case:

 Description   

That may not always be the case - the datagen connector can produce keys that are just strings.



 Comments   
Comment by Githook User [ 06/Apr/20 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Added LazyBsonDocument and made SinkDocument key/values lazily decoded.

KAFKA-45
Branch: master
https://github.com/mongodb/mongo-kafka/commit/f4df234076a140a07da2d9adce51f5fe6aaf0f36

Comment by Ross Lawley [ 01/Apr/20 ]

PR: https://github.com/rozza/mongo-kafka/pull/14

Comment by Ross Lawley [ 05/Sep/19 ]

Thanks grahslhp@gmail.com,

Looks like this could just be a documentation issue then that can be fixed by an SMT.

Comment by H.P. Grahsl [ 29/Aug/19 ]

my 2 cents on this: I never saw this as a real restriction. The way to go is to simply configure an SMT that wraps e.g. a plain string into a "document" structure and that's it. The advantage in doing so is to keep the options w.r.t key-related "manipulations": extract parts, project fields etc. which wouldn't be possible when employing e.g. a plain string or a simple number. Your mileage may vary though

Generated at Thu Feb 08 09:05:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.