[KAFKA-207] Heartbeat make messages tombstone messages Created: 15/Mar/21  Updated: 28/Oct/23  Resolved: 16/Mar/21

Status: Closed
Project: Kafka Connector
Component/s: Source
Affects Version/s: None
Fix Version/s: 1.5.0

Type: Improvement Priority: Major - P3
Reporter: Ross Lawley Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Case:

 Description   

Reduces the size of the data stored in Kafka.



 Comments   
Comment by Githook User [ 29/Mar/21 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Ensure the heartbeat message schema is optional

A non optional schema breaks value conversion.

KAFKA-207 KAFKA-208
Branch: master
https://github.com/mongodb/mongo-kafka/commit/00cfc5e4ee241159acf48fc90a1d6a623dfdfba0

Comment by Githook User [ 15/Mar/21 ]

Author:

{'name': 'Ross Lawley', 'email': 'ross.lawley@gmail.com', 'username': 'rozza'}

Message: Improve heartbeat message efficiency.

Set the value payload to null. Making them a tombstone message.
This allows Kafka to delete the previous messages and reduces storage.

KAFKA-207
Branch: master
https://github.com/mongodb/mongo-kafka/commit/6330cb9336924e81dceef72b8e383cfd5355cbb9

Generated at Thu Feb 08 09:05:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.