[KAFKA-341] Pipeline doesn't work in mongo source connector when Publish full document only: true Created: 15/Nov/22 Updated: 17/Nov/22 Resolved: 16/Nov/22 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Question | Priority: | Minor - P4 |
| Reporter: | Jayanth Culli | Assignee: | Ross Lawley |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||
| Issue Links: |
|
||||
| Description |
|
I have set publish full document only to true. When I try to exclude fields from the full document using the pipeline, it doesn't work. Is there a workaround for excluding fields when publishing full document? |
| Comments |
| Comment by Jayanth Culli [ 17/Nov/22 ] | |
|
When we set publish full document only: true, the pipeline suggested by you doesn't work. Verified it myself.
| |
| Comment by Ross Lawley [ 16/Nov/22 ] | |
|
Thank you for reaching out. The pipeline is applied to the change stream event and not just the fullDocument. So you will have to update the configuration with that in mind eg:
I've opened a docs ticket to clarify what publish.full.document.only does and its only a helper that returns just the fullDocument field from the change stream event. For future reference I wanted to give you some better resources to get questions answered:
All the best, Ross Lawley |