[KAFKA-303] Support database.viewname in DefaultTopicMapper Created: 22/Mar/22  Updated: 27/Oct/23  Resolved: 30/Mar/22

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: H S Assignee: Robert Walters
Resolution: Works as Designed Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Currently only:

  • Exact matches - maps to the exact match
  • Database matches - maps to database match + separator + collection name
  • Wildcard matches - maps everything

Users should be able to map mongodb "view" as source.  We would like to build views and want views to serve as source for kafka connector



 Comments   
Comment by Robert Walters [ 30/Mar/22 ]

harini.srinivasan3@inter.ikea.com MongoDB Kafka connector source opens a change stream to MongoDB.  Since change streams can't be opened on a view this won't be possible within our connector.  Perhaps you can enable client side field level encryption if you'd like to protect data within a collection from others who also have access to the collection.

Comment by Robert Walters [ 29/Mar/22 ]

harini.srinivasan3@inter.ikea.com Hi, view definitions for MongoDB collections are just aggregation pipeline definitions.  You could create a connector one for each "view" and simply paste in the view definition for the pipeline.  Would this work for your scenario?

Comment by Esha Bhargava [ 25/Mar/22 ]

harini.srinivasan3@inter.ikea.com Thank you for reporting this issue. We'll look into it and get back to you soon.

Generated at Thu Feb 08 09:06:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.