[KAFKA-113] Writemodel upsert strategy which can target multiple fields Created: 10/Jun/20 Updated: 27/Oct/23 Resolved: 08/Sep/20 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | Sink |
| Affects Version/s: | None |
| Fix Version/s: | 1.3.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Pat W | Assignee: | Ross Lawley |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Case: | (copied to CRM) |
| Description |
|
Hello again, We recently upgraded our sharded mongodb cluster to 4.2.0+ and encountered an issue where we wanted upserts but the mongodb connector creates upserts which only match on the _id field and our sharded collections have compound shard keys. I created a writemodel strategy which creates upserts that can be configured to target multiple fields, hopefully it can be of use. As before I have included tests, https://github.com/mongodb/mongo-kafka/pull/16 Thanks! -Pat |
| Comments |
| Comment by Ross Lawley [ 08/Sep/20 ] |
|
Closing as works as designed for now but happy to review if there is a scenario where that wont be the case. |
| Comment by Ross Lawley [ 08/Sep/20 ] |
|
In review you should be able to use the PartialValueStrategy to generate the shardkey match. See: writemodel strategy business keys.
I think the combination of the PartialValueStrategy and the fix coming in Ross |