<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:42:39 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-57737] The index builds code path can throw WCEs and invalidate an active cursor by calling abandonSnapshot without calling save/restore cursor.</title>
                <link>https://jira.mongodb.org/browse/SERVER-57737</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;TLDR; the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/catalog/multi_index_block.cpp#L578-L597&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MultiIndexBlock::_doCollectionScan runs a while-loop&lt;/a&gt;&#160;that iterates a collection cursor and inserts once document at a time into the index build bulk loaders. The write can throw a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/db7e21b3f2ff986a4e78b583dfed41dd83596f8a/src/mongo/db/concurrency/write_conflict_exception.h#L97-L104&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;WCE, which calls abandonSnapshot&lt;/a&gt;, resetting the active cursor. The cursor must be saved and restored around WCE handling. We hopefully can minimize save/restore to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/index/index_access_method.cpp#L593-L606&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;only when this code&lt;/a&gt;&#160;is run to handle constraint violations writes, where the WCE is occurring, to save performance. Should probably check that there aren&apos;t any other write paths in there that can throw.&lt;/p&gt;

&lt;p&gt;----------------------------------------------&lt;/p&gt;

&lt;p&gt;Copied from the test failure diagnosis:&lt;/p&gt;

&lt;p&gt;----------------------------------------------&lt;/p&gt;

&lt;p&gt;The invariant stack trace goes through the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/catalog/multi_index_block.cpp#L578&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MultiIndexBlock::_doCollectionScan() function that calls getNext here&lt;/a&gt;. The cursor is reading from the collection in a while-loop, and as each document is read it gets &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/catalog/multi_index_block.cpp#L597&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;inserted into the index builders tables&lt;/a&gt;. The inserts go through the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/catalog/multi_index_block.cpp#L634&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;bulk loader&lt;/a&gt;, through the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/index/index_access_method.cpp#L571&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;AbstractIndexAccessMethod ::BulkBuilderImpl::insert&lt;/a&gt;. The BulkBuilder has &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/index/index_access_method.cpp#L593-L606&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;special handling for documents that violate index constraints, writing them to a side table&lt;/a&gt;. The constraints violations side table does a write &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/index/index_access_method.cpp#L605&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; that can &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/index/skipped_record_tracker.cpp#L81-L92&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;throw a WCE and retry&lt;/a&gt;. Throwing the WCE &lt;a href=&quot;https://github.com/mongodb/mongo/blob/db7e21b3f2ff986a4e78b583dfed41dd83596f8a/src/mongo/db/concurrency/write_conflict_exception.h#L97-L104&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;causes abandonSnapshot() to be called&lt;/a&gt;. Lastly, Louis and I think that, since the cursor is not saved and restored around the insert, the abandonSnapshot() call must reset the cursor internally in WT.&lt;/p&gt;

&lt;p&gt;The hybrid index build logic doesn&apos;t do a save/restore cursor around the inserts, unlike the old index build code. This was a significant performance improvement, as saving and restoring the cursor is very costly. We clearly need some of the saves/restores back, but Louis suggested limiting those calls to only when writing to the constraints violations side table. It is not expected that there will be maybe violation constraints during an index build, except in pathological cases.&lt;/p&gt;

&lt;p&gt;We could pass the cursor down to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/index/index_access_method.cpp#L594-L606&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this logic&lt;/a&gt; that writes out to the side table &#8211; or somewhere lower, even. Louis suggested maybe passing down a lambda so as to avoid adding linking / code layer violation to the lower level code. Given the perf gains when the save/restore calls were removed with hybrid builds, we should probably sacrifice code niceties for performance. Other ideas might be passing flags down to recognize when the WCE+abandonSnapshot() has triggered, so we could save and use the high level record id &lt;a href=&quot;https://github.com/mongodb/mongo/blob/bf657d3db9cad295c9912ebc27aa786e68076751/src/mongo/db/catalog/multi_index_block.cpp#L597&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;before and after this call,&lt;/a&gt;, using the record id to forward the cursor back to where we need&#160;&#8211; that&apos;s what save/restore does, save the record ID on save and seek for the record ID on restore.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1784381">SERVER-57737</key>
            <summary>The index builds code path can throw WCEs and invalidate an active cursor by calling abandonSnapshot without calling save/restore cursor.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="dianna.hohensee@mongodb.com">Dianna Hohensee</reporter>
                        <labels>
                    </labels>
                <created>Wed, 16 Jun 2021 00:30:06 +0000</created>
                <updated>Sun, 29 Oct 2023 21:52:07 +0000</updated>
                            <resolved>Mon, 9 Aug 2021 20:25:46 +0000</resolved>
                                                    <fixVersion>5.0.3</fixVersion>
                    <fixVersion>5.1.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="4107423" author="JIRAUSER1259052" created="Wed, 6 Oct 2021 18:26:09 +0000"  >&lt;p&gt;Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it&#8217;s been triggered. For more active release information, please keep an eye on #server-release. Thank you!&lt;/p&gt;</comment>
                            <comment id="4012829" author="xgen-internal-githook" created="Fri, 20 Aug 2021 14:08:35 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;email&apos;: &apos;dianna.hohensee@mongodb.com&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-57737&quot; title=&quot;The index builds code path can throw WCEs and invalidate an active cursor by calling abandonSnapshot without calling save/restore cursor.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-57737&quot;&gt;&lt;del&gt;SERVER-57737&lt;/del&gt;&lt;/a&gt; Index builds must save and restore the collection scanning cursor around writes to the&lt;br/&gt;
violated index key constraints side table in case the write throws and unpositions the read cursor.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 05f055aad47b6afa3902ec4c11925c848f9cd534)&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/853ef0de7a7adfe2b7e36d96d1ae89791ef41832&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/853ef0de7a7adfe2b7e36d96d1ae89791ef41832&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3988962" author="xgen-internal-githook" created="Mon, 9 Aug 2021 16:58:15 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;email&apos;: &apos;dianna.hohensee@mongodb.com&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-57737&quot; title=&quot;The index builds code path can throw WCEs and invalidate an active cursor by calling abandonSnapshot without calling save/restore cursor.&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-57737&quot;&gt;&lt;del&gt;SERVER-57737&lt;/del&gt;&lt;/a&gt; Index builds must save and restore the collection scanning cursor around writes to the&lt;br/&gt;
violated index key constraints side table in case the write throws and unpositions the read cursor.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/05f055aad47b6afa3902ec4c11925c848f9cd534&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/05f055aad47b6afa3902ec4c11925c848f9cd534&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="1889529">SERVER-60451</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    <customfieldvalue key="18953"><![CDATA[v4.4]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 16 Jun 2021 22:34:09 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 18 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 18 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>118.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>vivian.ge@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzmeav:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hype2n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4881">Execution Team 2021-07-12</customfieldvalue>
    <customfieldvalue id="4882">Execution Team 2021-07-26</customfieldvalue>
    <customfieldvalue id="5116">Execution Team 2021-08-09</customfieldvalue>
    <customfieldvalue id="5117">Execution Team 2021-08-23</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzm0jz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>