<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:06:02 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-293] record will be lost if offset submitted earlier</title>
                <link>https://jira.mongodb.org/browse/KAFKA-293</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;In the class - org.apache.kafka.connect.runtime.WorkerSourceTask&lt;br/&gt;
enter function sendRecords() &lt;br/&gt;
we know that: &lt;/p&gt;

&lt;p&gt;firstly, execute &lt;br/&gt;
-&lt;br/&gt;
// Offsets are converted &amp;amp; serialized in the OffsetWriter&lt;br/&gt;
offsetWriter.offset(record.sourcePartition(), record.sourceOffset());&lt;br/&gt;
-&lt;br/&gt;
then execute&lt;br/&gt;
producer.send(...)&lt;/p&gt;

&lt;p&gt;but if producer.send(...) fail, but the offset of the record has been submit to offsetWriter&lt;/p&gt;

&lt;p&gt;and we know that, &lt;br/&gt;
org.apache.kafka.connect.runtime.SourceTaskOffsetCommitter.schedule(ConnectorTaskId id, WorkerSourceTask workerTask)&#160; is invoked periodically&#65281;&#65281;&#65281;&lt;br/&gt;
then it is possible that the offset of fail record has been recroded into the default file : /tmp/connect.offsets&lt;/p&gt;

&lt;p&gt;so it is wrong&lt;/p&gt;


&lt;p&gt;Is this a bug?&lt;/p&gt;</description>
                <environment></environment>
        <key id="1984527">KAFKA-293</key>
            <summary>record will be lost if offset submitted earlier</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="ross@mongodb.com">Ross Lawley</assignee>
                                    <reporter username="837500869@qq.com">Liu Zhiqiang</reporter>
                        <labels>
                            <label>external-user</label>
                    </labels>
                <created>Wed, 16 Feb 2022 11:45:11 +0000</created>
                <updated>Fri, 27 Oct 2023 11:54:12 +0000</updated>
                            <resolved>Tue, 15 Mar 2022 16:39:47 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="4413348" author="ross@10gen.com" created="Tue, 15 Mar 2022 16:39:23 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=837500869%40qq.com&quot; class=&quot;user-hover&quot; rel=&quot;837500869@qq.com&quot;&gt;837500869@qq.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thanks for the ticket, the offset mechanism is part of Kafka&apos;s resilience and how the Kafka connect framework works with it is outside the remit of the MongoDB Kafka Connector.&lt;/p&gt;

&lt;p&gt;I think your question is better suited to the Kafka mailing list / development team.&lt;/p&gt;

&lt;p&gt;All the best,&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                            <comment id="4377219" author="JIRAUSER1264944" created="Fri, 25 Feb 2022 16:11:41 +0000"  >&lt;p&gt;After I read the code of&#160; kafka connect - mongodb source ,&lt;/p&gt;

&lt;p&gt;I have rewrited it by myself ,&lt;/p&gt;

&lt;p&gt;please see &lt;a href=&quot;https://github.com/lzqdename/pulsar-mongodb-source-connector&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/lzqdename/pulsar-mongodb-source-connector&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;can everyone give me some improve suggestions?&#160;&lt;/p&gt;

&lt;p&gt;Thank you very much !&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="4369206" author="esha.bhargava" created="Tue, 22 Feb 2022 18:22:47 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=837500869%40qq.com&quot; class=&quot;user-hover&quot; rel=&quot;837500869@qq.com&quot;&gt;837500869@qq.com&lt;/a&gt; Thank you for reporting this issue! We&apos;ll look into it and get back to you soon.&lt;/p&gt;</comment>
                            <comment id="4358669" author="JIRAUSER1264944" created="Wed, 16 Feb 2022 14:53:24 +0000"  >&lt;p&gt;title is wrong, should be &quot;record will be lost if offset submitted earlier&quot;&lt;/p&gt;</comment>
                            <comment id="4358372" author="JIRAUSER1264944" created="Wed, 16 Feb 2022 12:29:29 +0000"  >&lt;p&gt;Affected version-Mongo Kafka Connector 1.6.1&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i04ex6:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>