Details
-
Task
-
Resolution: Fixed
-
Major - P3
-
None
-
None
-
None
Description
Currently mapping is confusing:
#Kafka topics to consume from
|
topics=foo-t,blah-t
|
|
|
#MongoDB collections to write to
|
mongodb.collections=foo-c,blah-c
|
|
|
#Named topic <-> collection mappings
|
mongodb.collection.foo-t=foo-c
|
mongodb.collection.blah-t=blah-c
|
|
|
|
|
# specific processing settings for topic 'foo-t' -> collection 'foo-c'
|
mongodb.document.id.strategy.foo-c=c.m.k.c.processor.id.strategy.UuidStrategy
|
mongodb.post.processor.chain.foo-c=c.m.k.c.processor.DocumentIdAdder,c.m.k.c.processor.BlacklistValueProjector
|
mongodb.value.projection.type.foo-c=blacklist
|
mongodb.value.projection.list.foo-c=k2,k4
|
mongodb.max.batch.size.foo-c=100
|
Theres a disconnect:
#MongoDB collections to write to
|
mongodb.collections=foo-c,blah-c
|
|
|
# topic <-> collection mapping
|
mongodb.collection.<*topicName*>=<collectionName>
|
|
|
# topic <-> collection overrides
|
mongodb.<configuration>.<*collectionName*>=someValue
|
I propose simplifying the configuration, so to do without mongodb.collections. I also propose making the configuration topic centric rather than collection - as topics are required.
topics=foo-t,blah-t
|
|
|
# specific processing settings for topics
|
mongodb.overrides.<topicName>.<config>=someValue
|
Everything apart from the topics list and the connection String can be overridden at a topic level.