<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:00:35 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-22503] movePrimary builds indexes in foreground on destination shard</title>
                <link>https://jira.mongodb.org/browse/SERVER-22503</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The movePrimary command is used to move primary database and unsharded collections from one shard to another. When movePrimary clones a collection from the donor to the recipient shard, it will create indexes using a MultiIndexBlock. It does not call MultiIndexBlock::allowBackgroundBuilding() before executing. Without this call all index builds will be performed in the foreground, regardless of whether {background: true} is set in a given index&apos;s specification.&lt;/p&gt;

&lt;p&gt;Performing foreground index builds on the recipient shard will block all database read/write activity for the duration of each index build. This can be a significant period of time depending on data size and hardware resources available. &lt;/p&gt;

&lt;p&gt;We should look to build indexes in the background during movePrimary. If not possible, we should at a minimum update the movePrimary documentation to reflect.&lt;/p&gt;

&lt;p&gt;This also impacts CloudManager shard removal via automation, in the case where the shard being removed is the primary database for unsharded collections.&lt;/p&gt;

&lt;p&gt;For movePrimary index build code see:&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.1/src/mongo/db/cloner.cpp#L358-L370&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.2.1/src/mongo/db/cloner.cpp#L358-L370&lt;/a&gt;&lt;/p&gt;


</description>
                <environment></environment>
        <key id="263719">SERVER-22503</key>
            <summary>movePrimary builds indexes in foreground on destination shard</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="backlog-server-sharding">[DO NOT USE] Backlog - Sharding Team</assignee>
                                    <reporter username="james.wahlin@mongodb.com">James Wahlin</reporter>
                        <labels>
                            <label>pm-1051-legacy-tickets</label>
                    </labels>
                <created>Sun, 7 Feb 2016 20:42:08 +0000</created>
                <updated>Tue, 6 Dec 2022 04:34:09 +0000</updated>
                            <resolved>Mon, 9 Dec 2019 15:54:24 +0000</resolved>
                                    <version>3.0.9</version>
                    <version>3.2.1</version>
                                    <fixVersion>4.2.0</fixVersion>
                                    <component>Index Maintenance</component>
                                        <votes>2</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="2593875" author="sheeri.cabral" created="Mon, 9 Dec 2019 15:54:24 +0000"  >&lt;p&gt;fixed in 4.1.8. Hybrid Index builds project fixed the negative impact.&lt;/p&gt;</comment>
                            <comment id="1920654" author="milkie" created="Thu, 14 Jun 2018 14:44:41 +0000"  >&lt;p&gt;It turns out that movePrimary clones all the data and then builds indexes, similar to initial sync.  So one way we could fix this is by building the indexes first and then cloning the data.&lt;br/&gt;
I have also linked this ticket to the Hybrid Index Build project, which will not lock the db while the index build is progressing.&lt;/p&gt;</comment>
                            <comment id="1898651" author="milkie" created="Tue, 22 May 2018 18:41:57 +0000"  >&lt;p&gt;Putting this ticket back in Needs Triage, as I don&apos;t believe this is a problem, at least in 4.0.  I would expect the movePrimary command to build indexes on empty collections, not ones with data in them.  &lt;br/&gt;
I also removed this ticket from the Simultaneous Indexing on All Nodes epic.&lt;/p&gt;</comment>
                            <comment id="1168478" author="spencer" created="Tue, 9 Feb 2016 21:22:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt;, you&apos;re right, it may be possible to make the indexes build in the background on the destination shard, but there are other problems with background index builds on replica sets, such as if there are multiple indexes on any of the collections being transferred replication to the secondaries will wind up blocked as they can&apos;t build more than one background index at a time.&lt;/p&gt;

&lt;p&gt;So to really make this process work without pain for the user requires a rethinking of how we manage indexes in a sharded cluster, as the workaround for single replica sets (building the index in a rolling fashion across replica set members) doesn&apos;t work for any index builds being driven by the system rather than by the user.&lt;/p&gt;</comment>
                            <comment id="1167923" author="james.wahlin@10gen.com" created="Tue, 9 Feb 2016 16:01:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; - calling MultiIndexBlock::allowBackgroundBuilding() would be an improvement, but I think we will a way to ensure background index builds (either via movePrimary or operationally) to make this safe for production environments that cannot tolerate downtime.&lt;/p&gt;</comment>
                            <comment id="1167885" author="kaloian.manassiev" created="Tue, 9 Feb 2016 15:39:01 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=james.wahlin&quot; class=&quot;user-hover&quot; rel=&quot;james.wahlin&quot;&gt;james.wahlin&lt;/a&gt;, the way I understand the problem is that during cloning of collections, because MultiIndexBlock::allowBackgroundBuilding is not called, even if all indexes on the source are background indexes, we will still build them as foreground. Is this correct?&lt;/p&gt;

&lt;p&gt;This should be easy to fix by just calling MultiIndexBlock::allowBackgroundBuilding. It won&apos;t help if you have mixed foreground and background indexes, but it would make it better in the case where all indexes are background.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=spencer&quot; class=&quot;user-hover&quot; rel=&quot;spencer&quot;&gt;spencer&lt;/a&gt;, what are your concerns with doing this?&lt;/p&gt;</comment>
                            <comment id="1166866" author="spencer" created="Mon, 8 Feb 2016 18:56:44 +0000"  >&lt;p&gt;This won&apos;t be easy to fix anytime soon, for now documenting it is probably the best we can do until we have a better overall story for index maintenance in a sharded cluster.  Filed &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-7112&quot; title=&quot;Add a warning that movePrimary may build indexes in the foreground&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-7112&quot;&gt;&lt;del&gt;DOCS-7112&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1166013" author="james.wahlin@10gen.com" created="Sun, 7 Feb 2016 21:20:53 +0000"  >&lt;p&gt;The &lt;a href=&quot;https://docs.mongodb.org/v3.0/reference/command/clone/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;clone command&lt;/a&gt; is also impacted by this. movePrimary uses the clone command to transfer data and build indexes.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="263904">DOCS-7112</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="333720">DOCS-9366</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25141"><![CDATA[Sharding]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000UaarlIAB]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 8 Feb 2016 18:56:44 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 9 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 9 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-sharding</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>james.wahlin@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>sheeri.cabral</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrki2v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrfzrz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsf9b3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>