[KAFKA-386] Kafka mongo sink. Create _id from key message Created: 20/Sep/23 Updated: 27/Oct/23 Resolved: 10/Oct/23 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Question | Priority: | Minor - P4 |
| Reporter: | Ralph Sawaya | Assignee: | Robert Walters |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
Hello, I have the following kafka message: key content is {{ {orderid: 123}}} and the message value as {{{} {name: "def", company: "abc"}{}}}. I want to insert in mongodb the following document: {_id: 123, name: "def", company: "abc"} where _id takes the value of orderid from the key message. I have tried the following config and it looks ok except that the _id is being an object like this: _id:{orderid: 123}{}. Any idead how we can achieve this? I have the following config:
Thank you! Ralph
|
| Comments |
| Comment by Ralph Sawaya [ 26/Sep/23 ] |
|
Thanks Robert. This works well if the key message contains the _id field such as {_id: 123} but in my case the key message can contain either orderid or itemid. I will check with the customer if he is willing to make a change on the message structure. Otherwise I believe, after much investigation, we are just left with custom transformation to achieve the desired result. It is anyway ok for the customer to keep things as they are but it would have been better if there was a staighforward solution so that _id contains directly the value instead of the object with the value. |
| Comment by Robert Walters [ 25/Sep/23 ] |
|
Consider using the document.id.strategy field as defined below:
document.id.strategy": "com.mongodb.kafka.connect.sink.processor.id.strategy.ProvidedInKeyStrategy"
|