[KAFKA-287] How to get the operationType when publish.full.document.only=true Created: 28/Jan/22  Updated: 27/Oct/23  Resolved: 14/Mar/22

Status: Closed
Project: Kafka Connector
Component/s: Configuration
Affects Version/s: 1.6.1
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: jiazhi zhou Assignee: Robert Walters
Resolution: Gone away Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

As the title mentioned, I want to get operationType in full-document mode. Although I can get the field if i set publish.full.document.only=false, the json message will become more complicated, which the fullDocument field was put in the inner json of a field named payload.
i am only interested in operationType *and *fullDocument, and i want to get a json without payload filed like below.

{
"fullDocument":"data...",
"operationType":"insert or update"
}

Is there any simple way to get that json through config ?
It is better if you could give me a sample of whole config for that. Thanks



 Comments   
Comment by PM Bot [ 14/Mar/22 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by jiazhi zhou [ 16/Feb/22 ]

Thanks, seems working

Comment by Robert Walters [ 09/Feb/22 ]

zhoujiazhiwork@163.com You can use a pipeline the source

something like

...
"publish.full.document.only": false,
"pipeline":"\\{'$project': {'_id':1, 'ns':1,'operationType': 1, 'fullDocument':1 }}",
 
Note: you need to project _id because that is how the connector tracks the resume token, you need ns because that is how it knows the topic to use. 
 

Generated at Thu Feb 08 09:06:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.