<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:43:01 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-17014] foreground index build blocks database reads and writes</title>
                <link>https://jira.mongodb.org/browse/SERVER-17014</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Some discussion is at &lt;a href=&quot;https://groups.google.com/forum/#!topic/mongodb-dev/_1IrogzovEQ&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://groups.google.com/forum/#!topic/mongodb-dev/_1IrogzovEQ&lt;/a&gt;. When I create an index with background:false then many (all?) operations in the db are blocked even for engines like WiredTiger that don&apos;t require a per-db writer lock. The URL above shows thread stacks where background jobs (TTLMonitor, ClientCursorMonitor) get blocked on a per-db lock by a background:false index create. I assume bad things can happen when TTL enforcement doesn&apos;t run for too long.&lt;/p&gt;

&lt;p&gt;This creates other problems as &quot;show collections&quot;, db.$foo.getIndexes() and queries from other collections in the same database will be blocked for the duration of the index create. &lt;/p&gt;

&lt;p&gt;While background:true is the workaround background index creation can take more time.&lt;/p&gt;</description>
                <environment></environment>
        <key id="180278">SERVER-17014</key>
            <summary>foreground index build blocks database reads and writes</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="louis.williams@mongodb.com">Louis Williams</assignee>
                                    <reporter username="mdcallag">Mark Callaghan</reporter>
                        <labels>
                    </labels>
                <created>Thu, 22 Jan 2015 23:57:46 +0000</created>
                <updated>Thu, 17 Jan 2019 17:18:07 +0000</updated>
                            <resolved>Thu, 17 Jan 2019 17:18:07 +0000</resolved>
                                    <version>2.8.0-rc5</version>
                                                    <component>Index Maintenance</component>
                                        <votes>6</votes>
                                    <watches>20</watches>
                                                                                                                <comments>
                            <comment id="2118506" author="louis.williams" created="Thu, 17 Jan 2019 17:18:07 +0000"  >&lt;p&gt;As of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-37270&quot; title=&quot;Remove foreground index build functionality&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-37270&quot;&gt;&lt;del&gt;SERVER-37270&lt;/del&gt;&lt;/a&gt;, all indexes are built in the background. Index builds no longer block reads or writes for their entire duration. Instead there are very brief periods at completion where index builds block writes.&lt;/p&gt;</comment>
                            <comment id="1841483" author="charles.sarrazin" created="Thu, 22 Mar 2018 00:20:20 +0000"  >&lt;p&gt;I guess the only thing I see which could go wrong with using the oplog to permit writes on an ongoing index creation would be for unique indexes. Indeed, in this case you might reach a point where you might already have duplicate documents, preventing the index from being created.&lt;/p&gt;

&lt;p&gt;The problem here is that we might have an acknowledged write sent to the client, appearing in the oplog, but which would normally fail because of the unique constraint. In this case, we would either need to drop the index, or purge the oplog entries which would no longer be valid (which would actually be pretty bad for consistency, as the effect would be more or less be similar rollback).&lt;/p&gt;</comment>
                            <comment id="811661" author="milkie" created="Fri, 23 Jan 2015 14:36:10 +0000"  >&lt;p&gt;We could improve foreground index builds in the following manner:&lt;br/&gt;
   We could convert the DB X lock to Collection IX before doing the collection scan, and then convert back to DB X lock at the end of the build, similar to how it is already done for background index builds.  There is no need to hold a DB X lock while scanning the collection; making this change will permit reads to the collection during the index build, but writes will still block.&lt;br/&gt;
TTL could then be improved to use trylock with a timeout for each collection where deletes need to happen.&lt;/p&gt;

&lt;p&gt;Longer term, we could improve foreground index builds even more by making use of the oplog to permit writes while scanning the collection, similar to how initial sync works.  After the collection scan and btree build is complete, we catch up by applying all the index changes for the writes we see in the oplog.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="608277">SERVER-37270</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000bU13uIAC]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 23 Jan 2015 14:36:10 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 3 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-663</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 3 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>charles.sarrazin@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>mdcallag</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrldw7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu7vov:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>158290</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2709">Storage NYC 2019-01-28</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;1) start an index create that takes 10+ seconds for a collection in database &quot;a&quot;&lt;br/&gt;
2) wait a few seconds and then get thread stacks using PMP &amp;#8211; &lt;a href=&quot;http://poormansprofiler.org/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://poormansprofiler.org/&lt;/a&gt;&lt;/p&gt;
</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsgan3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>