<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:06:12 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-362] Source Connector - Exactly Once Semantics (KIP-618)</title>
                <link>https://jira.mongodb.org/browse/KAFKA-362</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;We&apos;re wondering if it would be feasible to implement the introduced properties by [KIP-618|&lt;a href=&quot;https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors&lt;/a&gt;|https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors]&lt;/p&gt;

&lt;p&gt;that got deployed in Kafka 3.3.&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;exactly.once.support&lt;/tt&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;transaction.boundary&lt;/tt&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;offsets.storage.topic&lt;/tt&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;transaction.boundary.interval.ms&lt;/tt&gt;&#160;&lt;/p&gt;

&lt;p&gt;into the source connector or if that for some reason wouldn&apos;t work.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2300820">KAFKA-362</key>
            <summary>Source Connector - Exactly Once Semantics (KIP-618)</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="robert.walters@mongodb.com">Robert Walters</assignee>
                                    <reporter username="robin.fehr@dormakaba.com">Robin Fehr</reporter>
                        <labels>
                    </labels>
                <created>Wed, 29 Mar 2023 13:31:02 +0000</created>
                <updated>Fri, 27 Oct 2023 19:45:06 +0000</updated>
                            <resolved>Thu, 20 Apr 2023 12:01:01 +0000</resolved>
                                                                    <component>Sink</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="5362055" author="dbeng-pm-bot" created="Thu, 20 Apr 2023 12:01:02 +0000"  >&lt;p&gt;There hasn&apos;t been any recent activity on this ticket, so we&apos;re resolving it. Thanks for reaching out! Please feel free to comment on this if you&apos;re able to provide more information.&lt;/p&gt;</comment>
                            <comment id="5340867" author="dbeng-pm-bot" created="Wed, 12 Apr 2023 12:01:01 +0000"  >&lt;p&gt;Hey &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=robin.fehr%40dormakaba.com&quot; class=&quot;user-hover&quot; rel=&quot;robin.fehr@dormakaba.com&quot;&gt;robin.fehr@dormakaba.com&lt;/a&gt;, We need additional details to investigate the problem. If this is still an issue for you, please provide the requested information.&lt;/p&gt;</comment>
                            <comment id="5323383" author="JIRAUSER1273309" created="Tue, 4 Apr 2023 14:51:16 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=robert.walters%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;robert.walters@mongodb.com&quot;&gt;robert.walters@mongodb.com&lt;/a&gt; &lt;/p&gt;

&lt;p&gt;we currently use it in a use case where we write to a MongoDB collection in an append only fashion. (e.g. logs / event-stores etc.)&lt;br/&gt;
we then attach a connector to those events and forward them to a kafka topic from where we consume via the consumer API from within different micro services. these micro services process the events and write the data along with the offset within a transaction to the database. if the connector would be able to support exactly-once the micro services could rely on the offset and do the deduplication by verifying the offset of a partition rather than doing the deduplication by saving partial data of the events (e.g. IDs) to the database and compare incoming events against the already processed IDs stored and indexed. It would basically make exactly once way easier from a consumer api users&apos; perspective since the storing and comparing offsets per partition is way easier than for example pulling off the deduplication using a capped collection in MongoDB (assuming the particular micro service is using MongoDB). Depending on the amount of partitions it might also be hard to keep the IDs in memory which comes with cache invalidation complexity. This burden is immediately relaxed if we can rely on the offset.&#160;&lt;/p&gt;</comment>
                            <comment id="5322649" author="robert.walters" created="Tue, 4 Apr 2023 11:04:45 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=robin.fehr%40dormakaba.com&quot; class=&quot;user-hover&quot; rel=&quot;robin.fehr@dormakaba.com&quot;&gt;robin.fehr@dormakaba.com&lt;/a&gt; Can you provide your use case where you would need exactly once ?&#160; Would this be for a Mongo to Mongo replication scenario or ?&#160;&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr3mmh:0400000950m</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>