<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:05:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-179] copy.existing does not copy all of the existing documents</title>
                <link>https://jira.mongodb.org/browse/KAFKA-179</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;Given a database and collection with existing data, when the copy.existing flag is set to true, upon initial configuration of the connector only 10-20% of the documents will be sourced from mongodb into the kafka topic.&lt;/p&gt;

&lt;p&gt;repro:&lt;br/&gt;
1. insert at least 1000 documents into the Stocks.StockData collection&lt;br/&gt;
2. configure the kafka source connector as follows:&lt;/p&gt;


&lt;p&gt;     &quot;tasks.max&quot;:&quot;1&quot;,&lt;br/&gt;
     &quot;connector.class&quot;:&quot;com.mongodb.kafka.connect.MongoSourceConnector&quot;,&lt;br/&gt;
     &quot;key.converter&quot;:&quot;org.apache.kafka.connect.storage.StringConverter&quot;,&lt;br/&gt;
     &quot;value.converter&quot;:&quot;org.apache.kafka.connect.storage.StringConverter&quot;,&lt;br/&gt;
     &quot;publish.full.document.only&quot;: false,&lt;br/&gt;
     &quot;copy.existing&quot; : true,&lt;br/&gt;
     &quot;connection.uri&quot;:&quot;mongodb://mongo1:27017,mongo2:27017,mongo3:27017&quot;,&lt;br/&gt;
     &quot;topic.prefix&quot;:&quot;stockdata&quot;,&lt;br/&gt;
     &quot;database&quot;:&quot;Stocks&quot;,&lt;br/&gt;
     &quot;collection&quot;:&quot;StockData&quot;&lt;/p&gt;

&lt;p&gt;3. view the messages on the kafka topic, you&apos;ll see only about 100 show up even after the connector is left running for a long time&lt;/p&gt;
</description>
                <environment></environment>
        <key id="1558616">KAFKA-179</key>
            <summary>copy.existing does not copy all of the existing documents</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="ross@mongodb.com">Ross Lawley</assignee>
                                    <reporter username="robert.walters@mongodb.com">Robert Walters</reporter>
                        <labels>
                    </labels>
                <created>Thu, 3 Dec 2020 17:35:21 +0000</created>
                <updated>Fri, 27 Oct 2023 19:45:03 +0000</updated>
                            <resolved>Tue, 22 Dec 2020 12:00:05 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="3539382" author="backlog-server-pm" created="Tue, 22 Dec 2020 12:00:05 +0000"  >&lt;p&gt;There hasn&apos;t been any recent activity on this ticket, so we&apos;re resolving it. Thanks for reaching out! Please feel free to comment on this if you&apos;re able to provide more information.&lt;/p&gt;</comment>
                            <comment id="3518732" author="ross@10gen.com" created="Mon, 7 Dec 2020 16:20:51 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=robert.walters&quot; class=&quot;user-hover&quot; rel=&quot;robert.walters&quot;&gt;robert.walters&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I&apos;ve not been able to reproduce this issue locally.  My test environment is: &lt;a href=&quot;https://github.com/rozza/mongo-kafka-docker/tree/k176&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/rozza/mongo-kafka-docker/tree/k176&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In one terminal tab run: &lt;tt&gt;./run.sh&lt;/tt&gt; to setup the kafka and mongodb environment&lt;br/&gt;
In another terminal tab run: &lt;tt&gt;./k176.sh&lt;/tt&gt; to insert 1000 documents into the &lt;tt&gt;test.testCopyExisting&lt;/tt&gt; namespace, add the connector and then output all messages on the &lt;tt&gt;mongo.test.testCopyExisting&lt;/tt&gt;.  Here you can see all 1000 messages are added to the topic.&lt;/p&gt;

&lt;p&gt;Can you confirm that the test docker scenario works?  Otherwise I&apos;ll need mongodb and kafka logs to see if I can replicate it.&lt;/p&gt;

&lt;p&gt;Ross&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy63un:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>