<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:44:27 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-58413] Create/Find a workload that does CRUD ops while moving chunks around</title>
                <link>https://jira.mongodb.org/browse/SERVER-58413</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;This workload should try to take advantage of the Fine-grained Collection Critical Sections that would be implemented as part of PM-2098. This means that the CRUD operations should target a set of chunks that doesn&apos;t include chunks being migrated.&lt;/p&gt;

&lt;p&gt;There are some chunk-related workloads on the &lt;a href=&quot;https://github.com/10gen/workloads&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;workloads repo&lt;/a&gt;.  &lt;a href=&quot;https://github.com/10gen/workloads/blob/master/workloads/move_chunk_with_load.js&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;This one looks promising.&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="1814805">SERVER-58413</key>
            <summary>Create/Find a workload that does CRUD ops while moving chunks around</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="antonio.fuschetto@mongodb.com">Antonio Fuschetto</assignee>
                                    <reporter username="sergi.mateo-bellido@mongodb.com">Sergi Mateo Bellido</reporter>
                        <labels>
                    </labels>
                <created>Mon, 12 Jul 2021 06:59:27 +0000</created>
                <updated>Thu, 29 Jul 2021 14:05:08 +0000</updated>
                            <resolved>Thu, 29 Jul 2021 13:47:19 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="3962157" author="JIRAUSER1259062" created="Mon, 26 Jul 2021 08:58:41 +0000"  >&lt;p&gt;The previously identified test was rewritten to run very quickly on personal workstations and without the heavy infrastructure required by the &lt;em&gt;workload&lt;/em&gt; framework.&lt;/p&gt;

&lt;p&gt;It has been also changed in order to:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Setup 2 shards and 1 sharded collection splitted into 3 chunks: 2 chunks contain 25K documents and are spread over the 2 shards, while 1 chunk contains 1 document and is stored on the first shard&lt;/li&gt;
	&lt;li&gt;Start &quot;n&quot; background threads that continuously&#160;execute &lt;tt&gt;insert&lt;/tt&gt;, &lt;tt&gt;remove&lt;/tt&gt;, &lt;tt&gt;update&lt;/tt&gt; and &lt;tt&gt;findOne&lt;/tt&gt; operations on the sharded collection&lt;/li&gt;
	&lt;li&gt;Move the the chunk containing 1 document back and forth between the first and second shards&lt;/li&gt;
	&lt;li&gt;Record the throughput of CRUD and&#160;&lt;tt&gt;moveChunk&lt;/tt&gt; operations, and the duration of the &lt;tt&gt;moveChunk&lt;/tt&gt;&apos;s critical section at each step&lt;/li&gt;
	&lt;li&gt;Reiterate the tests with different thread numbers (i.e., 0, 8, 16, 32, 64, 128, 256)&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The implementation of the test is available at &lt;a href=&quot;https://github.com/afuschetto/mongo-scripts/blob/main/performance/move_chunk_with_load.js&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/afuschetto/mongo-scripts/blob/main/performance/move_chunk_with_load.js&lt;/a&gt; (read permissions are required).&lt;/p&gt;</comment>
                            <comment id="3932364" author="JIRAUSER1259062" created="Wed, 14 Jul 2021 07:28:11 +0000"  >&lt;p&gt;As initially assumed, a very good candidate to test the performance of the &lt;em&gt;migration protocol&lt;/em&gt; and concurrent &lt;em&gt;CRUD operations&lt;/em&gt; is the &lt;a href=&quot;https://github.com/10gen/workloads/blob/master/workloads/move_chunk_with_load.js&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;move_chunk_with_load.js&lt;/a&gt; in the &lt;a href=&quot;https://github.com/10gen/workloads&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;workloads&lt;/a&gt; repository.&lt;/p&gt;

&lt;p&gt;This test measures the throughput of multiple CRUD operations while a chunk continuously migrates from one shard to another, thereby highlighting the performance penalties introduced by the critical section of the migration protocol and measuring the overall impact on ordinary business operations.&lt;/p&gt;

&lt;p&gt;Some details on the test implementation follow:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Setup &quot;n&quot; shards, 1 sharded collection splitted into &quot;n + 1&quot; chunks each containing 25K document, and distribute 1 chunk per shard except the first one which hosts 2 chunks&lt;/li&gt;
	&lt;li&gt;Start &quot;n&quot; background threads that continuously&#160;execute &lt;tt&gt;insert&lt;/tt&gt;, &lt;tt&gt;remove&lt;/tt&gt;, &lt;tt&gt;update&lt;/tt&gt; and &lt;tt&gt;findOne&lt;/tt&gt; operations on the sharded collection&lt;/li&gt;
	&lt;li&gt;Move the first chunk (originally hosted by the fist shard) through the following shards&lt;/li&gt;
	&lt;li&gt;Record the throughput of CRUD and &lt;tt&gt;moveChunk&lt;/tt&gt; operations at different stages for a final report&lt;/li&gt;
	&lt;li&gt;Reiterate the tests with different thread numbers&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The logic seems more than sufficient for the purposes of this activity, however we could refine measurements and final reports by working on &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-58414&quot; title=&quot;Evaluate the performance of the new workload on master&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-58414&quot;&gt;&lt;del&gt;SERVER-58414&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="1814808">SERVER-58414</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1814811">SERVER-58415</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 14 Jul 2021 07:28:11 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 28 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2098</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 28 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>antonio.fuschetto@mongodb.com</customfieldvalue>
            <customfieldvalue>sergi.mateo-bellido@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzrkmf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hz9arb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4521">Sharding 2021-07-12</customfieldvalue>
    <customfieldvalue id="4981">Sharding EMEA 2021-07-26</customfieldvalue>
    <customfieldvalue id="5178">Sharding EMEA 2021-08-09</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzr6vj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>