<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:03:25 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-3569] Find operation does not return all documents that are previously inserted into a sharded environment while chunks are moved.</title>
                <link>https://jira.mongodb.org/browse/SERVER-3569</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;To test the mongoDb sharding capabilities we have continuously inserted data into a two shards environment. Each shard consists of &lt;br/&gt;
one replica set (1 Master, 1 Secondary and 1 Arbiter). At a given point we perform some find operations to verify whether all data is returned. The problem is that there are sometimes gaps in the result. E.g. the result contains document 5200 and 5202 but not document 5201. We are using the mongo java driver 2.6.2 but could also experience the behavior with the php driver.&lt;/p&gt;

&lt;p&gt;These gaps are independent of the used write concern.&lt;/p&gt;

&lt;p&gt;To reproduce the problem a java maven project is attached to this issue:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Follow the steps in the README.txt to run the test&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Details:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;The test application provides 3 classes with a main method located in src\main\java\com\seitenbau\testing\mongo\runner: Inserter, VerifierSharding, VerifierFinding&lt;/li&gt;
	&lt;li&gt;The Inserter inserts documents that contain a field storing a counter (the counter is incremented each time a document is inserted). The script generates the file &quot;counter-state&quot; which stores the current value of the counter.&lt;/li&gt;
	&lt;li&gt;The VerifierSharding logs some information on sharding and counting (verifier-sharding-info.log).&lt;/li&gt;
	&lt;li&gt;The VerifierFinding performs a find operation to find all documents that are saved in the target collection at the moment.&lt;br/&gt;
  The result set is sorted and then logged in a file with the format &amp;lt;timestamp&amp;gt;-node-data. Additionally the script generates the log file &quot;verifier-finding-info.log&quot;.&lt;/li&gt;
	&lt;li&gt;To configure the application check the &quot;config.properties&quot; file&lt;/li&gt;
	&lt;li&gt;You have to abort the scripts manually as they perform their intended job inside a while loop.&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment>Ubunutu 10.10 x86_64</environment>
        <key id="20801">SERVER-3569</key>
            <summary>Find operation does not return all documents that are previously inserted into a sharded environment while chunks are moved.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="randolph@mongodb.com">Randolph Tan</assignee>
                                    <reporter username="michaelsb">Michael Wagner</reporter>
                        <labels>
                    </labels>
                <created>Wed, 10 Aug 2011 11:45:01 +0000</created>
                <updated>Wed, 10 Dec 2014 23:04:56 +0000</updated>
                            <resolved>Tue, 28 May 2013 19:41:25 +0000</resolved>
                                    <version>1.8.2</version>
                    <version>1.9.1</version>
                    <version>2.0.0</version>
                                                    <component>Sharding</component>
                                        <votes>5</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="55899" author="scotthernandez" created="Thu, 22 Sep 2011 15:04:41 +0000"  >&lt;p&gt;The issue is the chunk distribution (based on the shard key) and not the number of shards. If there are few chunks, like the initial one when you shard the collection before there is much data, it will have to do lots of migration until the distribution of chunks is even among all shards. Even then if the shard key isn&apos;t well distributed there can be host spots/chunk will require more splits, and will cause the balancer to move them around, which will cause more of these issues. With a load and well distributed system this will be much, much less likely over time.&lt;/p&gt;

&lt;p&gt;You could manage this a bit by only doing balancing in maint. windows which will make this behavior more predictable.&lt;/p&gt;</comment>
                            <comment id="55891" author="michaelsb" created="Thu, 22 Sep 2011 14:50:06 +0000"  >&lt;p&gt;We could reproduce this issue with mongoDb version 2.0.0. To create the test environment we have stopped all mongo processes, replaced the old binaries, performend a clean up upon all data and finally started all mongo processes again. &lt;/p&gt;</comment>
                            <comment id="55890" author="michaelsb" created="Thu, 22 Sep 2011 14:44:11 +0000"  >&lt;p&gt;The issue also appears if both shards are already activated at the beginning of our test.&lt;/p&gt;</comment>
                            <comment id="55882" author="scotthernandez" created="Thu, 22 Sep 2011 13:57:57 +0000"  >&lt;p&gt;That pretty much makes the issue one of chunk migration with the queued write-backs happening at the end of the migration. &lt;/p&gt;

&lt;p&gt;With a better balanced system, or more stable distribution of chunk, this should be less of an issue as compared to a simple test starting from an empty data-set.&lt;/p&gt;</comment>
                            <comment id="55881" author="michaelsb" created="Thu, 22 Sep 2011 13:55:17 +0000"  >&lt;p&gt;The test environment is equal to the environment used to reproduce &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-3568&quot; title=&quot;MongoException &amp;quot;norepl&amp;quot; is thrown when data is inserted in a sharded environment using WriteConcern &amp;quot;ReplicasSafe&amp;quot;.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-3568&quot;&gt;&lt;del&gt;SERVER-3568&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="55880" author="michaelsb" created="Thu, 22 Sep 2011 13:53:55 +0000"  >&lt;p&gt;In case the balancing is turned off the described issue does not appear.&lt;/p&gt;</comment>
                            <comment id="55855" author="scotthernandez" created="Thu, 22 Sep 2011 11:36:06 +0000"  >&lt;p&gt;If balancing is turned off before running the tests, does the issue still appear?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="66482">SERVER-8741</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="12576" name="mongo-sharding-test.zip" size="17551" author="michaelsb" created="Wed, 10 Aug 2011 11:45:01 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 22 Sep 2011 11:36:06 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 21 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 21 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>michaelsb</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrou5j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrib9r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>20471</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrrgsf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>