<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:04:54 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-44062] Add zone operations concurrency testing for refineCollectionShardKey</title>
                <link>https://jira.mongodb.org/browse/SERVER-44062</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;h2&gt;&lt;a name=&quot;Background&quot;&gt;&lt;/a&gt;Background&lt;/h2&gt;
&lt;p&gt;Concurrency tests for refining a collection shard key should be added to verify a refine atomically takes effect and does not interfere with concurrent zone operations.&lt;/p&gt;

&lt;p&gt;In particular, workloads should be added that:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Repeatedly refines a shard key with concurrent zone operations (updateZoneKeyRange, addShardToZone, removeShardFromZone) in addition to concurrent CRUD ops.
	&lt;ul&gt;
		&lt;li&gt;Note updateZoneRange is the most important to test, so testing with the other two zone commands can be split out and re-prioritized if necessary.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;In addition to the default concurrency suites, these workloads should run in suites with the balancer enabled and failovers. It should be verified that the balancer will actually attempt to move chunks, possibly enforced by inserting enough data to imbalance the cluster, beginning with lopsided zones, and/or adding fsm stages that explicitly wait for a balancer round.&lt;/p&gt;

&lt;h2&gt;&lt;a name=&quot;ProposedSolution&quot;&gt;&lt;/a&gt;Proposed Solution&lt;/h2&gt;

&lt;h3&gt;&lt;a name=&quot;HighLevelExplanation&quot;&gt;&lt;/a&gt;High-Level Explanation&lt;/h3&gt;
&lt;p&gt;This workload will be built upon the underlying idea of refining a list of collections controlled by a latch. &lt;/p&gt;

&lt;p&gt;Each thread will have two zones and two zone ranges. The thread will bounce the zones in between two shards, and will bounce the ranges in between the two zones. This means that any given time, a zone will belong to only one shard and will have a specified range. While this is happening, CRUD operations borrowed from the broadcast update/delete transaction states will be operating in documents belonging to the ranges owned by the thread. Every so often, the collection shard key will be refined, and then we will move to the next collection.&lt;/p&gt;

&lt;h3&gt;&lt;a name=&quot;Stepsforsettinguptheworkload&quot;&gt;&lt;/a&gt;Steps for setting up the workload&lt;/h3&gt;
&lt;ol&gt;
	&lt;li&gt;Create a number of zones equal to twice the amount of threads, so that each thread may have two zones.&lt;/li&gt;
	&lt;li&gt;Create a number of ranges equal to twice the amount of threads, so that each thread may take ownership of two ranges.&lt;/li&gt;
	&lt;li&gt;Assign to each thread two zones and two ranges for those zones.&lt;/li&gt;
	&lt;li&gt;Shard the collection with the initial zones, which will create the chunks automatically.&lt;/li&gt;
	&lt;li&gt;Assign each zone to only one shard. &lt;em&gt;If the balancer is on, wait for the chunks to be distributed.&lt;/em&gt;&lt;/li&gt;
	&lt;li&gt;Fill owned ranges with documents to be used for CRUD ops.&lt;/li&gt;
&lt;/ol&gt;


&lt;h3&gt;&lt;a name=&quot;Stepsforthreadinitializationfortheworkload.&quot;&gt;&lt;/a&gt;Steps for thread initialization for the workload.&lt;/h3&gt;
&lt;ol&gt;
	&lt;li&gt;Cache into memory each thread&apos;s assigned zones, assigned ranges, and assigned documents.&lt;/li&gt;
&lt;/ol&gt;


&lt;h3&gt;&lt;a name=&quot;States&quot;&gt;&lt;/a&gt;States&lt;/h3&gt;
&lt;ul&gt;
	&lt;li&gt;sendZoneToOtherShard() &amp;#8211; Picks a random zone assigned to this thread, removes it from the current shard, and assigns it to the other shard. Verifies that the first shard no longer has the zone, and that the second shard has the zone. &lt;em&gt;If the balancer is on, waits for the chunks to be moved.&lt;/em&gt;&lt;/li&gt;
	&lt;li&gt;swapZoneRange() &amp;#8211; Removes the ranges from each of the zones assigned to this thread, and swaps them, such that each range is now assigned to the opposite zone. &lt;em&gt;If the ranges are owned by different shards and the balancer is on, waits for the chunks to be moved.&lt;/em&gt;&lt;/li&gt;
	&lt;li&gt;refineCollectionShardKey() &amp;#8211; Refines the collection&apos;s shard key and decreases the latch count for collections to refine.&lt;/li&gt;
	&lt;li&gt;update/delete in transaction states &amp;#8211; States borrowed from the broadcast update/delete transaction states.&lt;/li&gt;
&lt;/ul&gt;

</description>
                <environment></environment>
        <key id="972084">SERVER-44062</key>
            <summary>Add zone operations concurrency testing for refineCollectionShardKey</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="blake.oler@mongodb.com">Blake Oler</assignee>
                                    <reporter username="blake.oler@mongodb.com">Blake Oler</reporter>
                        <labels>
                    </labels>
                <created>Thu, 17 Oct 2019 15:23:42 +0000</created>
                <updated>Sun, 29 Oct 2023 22:16:01 +0000</updated>
                            <resolved>Fri, 1 Nov 2019 00:27:07 +0000</resolved>
                                                    <fixVersion>4.3.1</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="2512884" author="xgen-internal-githook" created="Thu, 31 Oct 2019 17:17:07 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;BlakeIsBlake&apos;, &apos;email&apos;: &apos;blake.oler@mongodb.com&apos;, &apos;name&apos;: &apos;Blake Oler&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-44062&quot; title=&quot;Add zone operations concurrency testing for refineCollectionShardKey&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-44062&quot;&gt;&lt;del&gt;SERVER-44062&lt;/del&gt;&lt;/a&gt; Add zone operations concurrency testing for refineCollectionShardKey&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/ae9747e4572976c713f2bc2eb02436eae4f36352&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/ae9747e4572976c713f2bc2eb02436eae4f36352&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2490743" author="blake.oler" created="Fri, 18 Oct 2019 21:02:58 +0000"  >&lt;p&gt;It turns out we can&apos;t verify that the balancer has balanced zones correctly en masse at the moment, due to the lack of ability to see into the balancer&apos;s &quot;quietness&quot; or its verification that nodes are completely balanced with regards to zones. So we&apos;ll have to hold off on verifying zones are on the right shards until then.&lt;/p&gt;

&lt;p&gt;Refer to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-43990&quot; title=&quot;A command to confirm that all chunks have been moved to the right zone after the initial split&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-43990&quot;&gt;&lt;del&gt;SERVER-43990&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2490278" author="blake.oler" created="Fri, 18 Oct 2019 17:48:47 +0000"  >&lt;ol&gt;
	&lt;li&gt;&lt;tt&gt;swapZoneRange&lt;/tt&gt; should be able to emulate this.&lt;/li&gt;
	&lt;li&gt;You&apos;re right there, it&apos;s not super interesting. It&apos;s to gain a bit of coverage with zones in concurrency testing altogether, since we currently have no zone concurrency testing.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="2488486" author="kaloian.manassiev" created="Fri, 18 Oct 2019 05:56:29 +0000"  >&lt;p&gt;Will there be separate states to ensure that adding zones concurrently with refining the shard key ends up with all zones containing the correct number of fields? Or is it the intention that &lt;tt&gt;swapZoneRange&lt;/tt&gt; will emulate that?&lt;/p&gt;

&lt;p&gt;The &lt;tt&gt;sendZoneToOtherShard&lt;/tt&gt; operation is an addShardToZone/removeShardFromZone combo, right? If this is the case, why is it interesting that it is tested in combination with refining the shard key given that it doesn&apos;t change the bounds? Maybe it has something to do with the refine operation potentially changing more than just the zone boundaries and overwriting the shard.&lt;/p&gt;

&lt;p&gt;Otherwise LGTM.&lt;/p&gt;</comment>
                            <comment id="2487621" author="blake.oler" created="Thu, 17 Oct 2019 20:41:07 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jack.mulrow&quot; class=&quot;user-hover&quot; rel=&quot;jack.mulrow&quot;&gt;jack.mulrow&lt;/a&gt;,&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;I was going to mix in the CRUD ops on top of a new workload, which should be easy to do with the way that I&apos;ve pulled out the broadcast states. It&apos;ll be easier not to implement the CRUD ops at all, as this will cut down the work required for the workload.&lt;/li&gt;
	&lt;li&gt;Added a comment on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-43099&quot; title=&quot;Reenable random chunk migration failpoint for concurrency with_balancer suites&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-43099&quot;&gt;&lt;del&gt;SERVER-43099&lt;/del&gt;&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Got it.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="2487602" author="jack.mulrow" created="Thu, 17 Oct 2019 20:24:46 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=blake.oler&quot; class=&quot;user-hover&quot; rel=&quot;blake.oler&quot;&gt;blake.oler&lt;/a&gt;, the approach LGTM. This should give us really useful coverage.&lt;/p&gt;

&lt;p&gt;A couple small notes:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;We might not need the CRUD ops since we&apos;re really only testing that chunks are migrated to the right shards, not how the migrations actually affect concurrent activity, but if it&apos;s easier to implement by extending one of the transaction workloads then I&apos;m fine with keeping CRUD ops in.&lt;/li&gt;
	&lt;li&gt;When &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-43099&quot; title=&quot;Reenable random chunk migration failpoint for concurrency with_balancer suites&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-43099&quot;&gt;&lt;del&gt;SERVER-43099&lt;/del&gt;&lt;/a&gt; re-enables a random balancer policy, we might need to do work to disable the randomness for this workload. I&apos;m not sure we can do anything about that now, but I&apos;d put a quick comment on that ticket for whoever ends up implementing it.&lt;/li&gt;
	&lt;li&gt;I thought it might be tricky to handle a refine during swapZoneRange(), but we already implicitly extend tag boundaries from a user that don&apos;t include all shard key fields &lt;a href=&quot;https://github.com/mongodb/mongo/blob/cff25ee25c/src/mongo/db/s/config/sharding_catalog_manager_zone_operations.cpp#L167-L168&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;, so I don&apos;t think this will be a problem, just mentioning in case it might save some investigation later.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="2487523" author="blake.oler" created="Thu, 17 Oct 2019 19:40:15 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jack.mulrow&quot; class=&quot;user-hover&quot; rel=&quot;jack.mulrow&quot;&gt;jack.mulrow&lt;/a&gt; Wrote up a proposal for a workload for zone operations. Can I get an LGTM?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 17 Oct 2019 20:24:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 14 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1327</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 14 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>blake.oler@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jack.mulrow@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvxwr3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hvmdlj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3307">Sharding 2019-11-04</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvxj0f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>