[KAFKA-218] LazyBsonDocument clone ignores changes Created: 21/Apr/21  Updated: 28/Oct/23  Resolved: 14/Jun/21

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: 1.6.0

Type: Bug Priority: Major - P3
Reporter: Ross Lawley Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
Related

 Description   

Was:  Change of behaviour in 1.5 in sink for mapping / document id strategy

 There has been reported a change in the behaviour between 1.4 & 1.5 for a complex mapping / document id strategy.

    "writemodel.strategy": "com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneBusinessKeyTimestampStrategy",
    "post.processor.chain": "com.mongodb.kafka.connect.sink.processor.field.renaming.RenameByMapping,com.mongodb.kafka.connect.sink.processor.DocumentIdAdder",
    "field.renamer.mapping": "[{\"oldName\":\"value.c\",\"newName\":\"_id\"}]",
  "document.id.strategy": "com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy",
    "document.id.strategy.overwrite.existing": "true",
    "document.id.strategy.partial.value.projection.type": "allowlist",
    "document.id.strategy.partial.value.projection.list": "a, b, _id",

The UpdateOneBusinessKeyTimestampStrategy appears to be adding a new ObjectId instead of using the mapped one as defined by field.renamer.mapping.

The workaround has been to use SMT to handle the remapping:

"transforms": "RenameField",
    "transforms.RenameField.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",
    "transforms.RenameField.renames": "c:_id",



 Comments   
Comment by Githook User [ 14/Jun/21 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Fixed bug in LazyBsonDocument#clone ignoring any changes made once unwrapped.

KAFKA-218
Branch: master
https://github.com/mongodb/mongo-kafka/commit/7fb677e24d94c01e0c6ffc27b881f90646a49891

Comment by Ross Lawley [ 10/Jun/21 ]

I was unable to replicate the change of behaviour between 1.4 & 1.5 however, I did find a bug where changes in the field renaming weren't in effect in the DocumentIdAdder due to a logic bug in cloning the LazyBsonDocument. 

Generated at Thu Feb 08 09:05:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.