<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:17:23 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-70879] Fix race where multiple threads are turning an index multikey concurrently</title>
                <link>https://jira.mongodb.org/browse/SERVER-70879</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In BF-26686 we see an inconsistent multikey state in the in-memory catalog.&lt;br/&gt;
From the comments under BF-26686 we witness two (presumably) concurrent inserts occur:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{ &#8220;_id&#8221; : ObjectId(&#8220;633a9b678c7aa637fbd079a6&quot;), &#8220;a&#8221; : [ { &#8220;b&#8221; : [ &lt;/span&gt;&lt;span style=&quot;color: #009900; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; ] } ] }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{ &#8220;_id&#8221; : ObjectId(&#8220;633a9b678c7aa637fbd079ac&#8221;), &#8220;a&#8221; : { &#8220;b&#8221; : [ -&lt;/span&gt;&lt;span style=&quot;color: #009900; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;, -&lt;/span&gt;&lt;span style=&quot;color: #009900; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2&lt;/span&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt; ] } }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;Notably, the first insert will set the multikey path for a and b while the second document will only set b. Ultimately, we only have b as a multikey path (on disk).&lt;/p&gt;

&lt;p&gt;We potentially have a situation where writers could be obtaining copies of the collection metadata before it is marked as multikey and the second writer could overwrite the other writer&#8217;s collection metadata write.&#160;&lt;/p&gt;

&lt;p&gt;This ticket would include writing a detailed c++ test to verify the findings and potential bug.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2167944">SERVER-70879</key>
            <summary>Fix race where multiple threads are turning an index multikey concurrently</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="henrik.edin@mongodb.com">Henrik Edin</assignee>
                                    <reporter username="fausto.leyva@mongodb.com">Fausto Leyva</reporter>
                        <labels>
                    </labels>
                <created>Wed, 26 Oct 2022 21:26:22 +0000</created>
                <updated>Sun, 29 Oct 2023 21:31:25 +0000</updated>
                            <resolved>Fri, 4 Nov 2022 18:43:55 +0000</resolved>
                                                    <fixVersion>6.1.1</fixVersion>
                    <fixVersion>5.0.14</fixVersion>
                    <fixVersion>6.0.3</fixVersion>
                    <fixVersion>6.2.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="4966955" author="xgen-internal-githook" created="Wed, 9 Nov 2022 14:47:13 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Henrik Edin&apos;, &apos;email&apos;: &apos;henrik.edin@mongodb.com&apos;, &apos;username&apos;: &apos;henrikedin&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70879&quot; title=&quot;Fix race where multiple threads are turning an index multikey concurrently&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70879&quot;&gt;&lt;del&gt;SERVER-70879&lt;/del&gt;&lt;/a&gt; Fix race where multiple threads are turning an index multikey concurrently&lt;/p&gt;

&lt;p&gt;Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer.&lt;/p&gt;

&lt;p&gt;To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 74bf9118e38ff4ee63db8bb17146dabe3ef07392)&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/defa7826e96929cabbaa204f91a39d6bd6e30239&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/defa7826e96929cabbaa204f91a39d6bd6e30239&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4965113" author="xgen-internal-githook" created="Tue, 8 Nov 2022 22:21:07 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Henrik Edin&apos;, &apos;email&apos;: &apos;henrik.edin@mongodb.com&apos;, &apos;username&apos;: &apos;henrikedin&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70879&quot; title=&quot;Fix race where multiple threads are turning an index multikey concurrently&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70879&quot;&gt;&lt;del&gt;SERVER-70879&lt;/del&gt;&lt;/a&gt; Fix race where multiple threads are turning an index multikey concurrently&lt;/p&gt;

&lt;p&gt;Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer.&lt;/p&gt;

&lt;p&gt;To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 74bf9118e38ff4ee63db8bb17146dabe3ef07392)&lt;br/&gt;
Branch: v6.1&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/7465f99eeb0a3c5ff05dc70a2e4e71de2f98ffd8&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/7465f99eeb0a3c5ff05dc70a2e4e71de2f98ffd8&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4965066" author="xgen-internal-githook" created="Tue, 8 Nov 2022 22:05:40 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Henrik Edin&apos;, &apos;email&apos;: &apos;henrik.edin@mongodb.com&apos;, &apos;username&apos;: &apos;henrikedin&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70879&quot; title=&quot;Fix race where multiple threads are turning an index multikey concurrently&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70879&quot;&gt;&lt;del&gt;SERVER-70879&lt;/del&gt;&lt;/a&gt; Fix race where multiple threads are turning an index multikey concurrently&lt;/p&gt;

&lt;p&gt;Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer.&lt;/p&gt;

&lt;p&gt;To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 74bf9118e38ff4ee63db8bb17146dabe3ef07392)&lt;br/&gt;
Branch: v6.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/ac5d0034debcc646ffe610f8d63302c25a3598b7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/ac5d0034debcc646ffe610f8d63302c25a3598b7&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4955958" author="xgen-internal-githook" created="Fri, 4 Nov 2022 18:08:55 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Henrik Edin&apos;, &apos;email&apos;: &apos;henrik.edin@mongodb.com&apos;, &apos;username&apos;: &apos;henrikedin&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70879&quot; title=&quot;Fix race where multiple threads are turning an index multikey concurrently&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70879&quot;&gt;&lt;del&gt;SERVER-70879&lt;/del&gt;&lt;/a&gt; Fix race where multiple threads are turning an index multikey concurrently&lt;/p&gt;

&lt;p&gt;Fix race where the writes to the durable catalog are serialized but the second writer is using a stale in-memory state due to the commit handler not being executed yet. This interleaving does not cause a write conflict as the writes are serialized but the second writer overwrote the multikey paths set by the first writer.&lt;/p&gt;

&lt;p&gt;To handle it we always read the latest state from the durable catalog before performing multikey writes. This guarantees that we always are operating on the latest state. Some care was needed to ensure index offsets remain stable.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/74bf9118e38ff4ee63db8bb17146dabe3ef07392&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/74bf9118e38ff4ee63db8bb17146dabe3ef07392&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="24444"><![CDATA[v6.1]]></customfieldvalue>
    <customfieldvalue key="23470"><![CDATA[v6.0]]></customfieldvalue>
    <customfieldvalue key="21777"><![CDATA[v5.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 31 Oct 2022 18:55:14 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 13 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 13 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>115.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>fausto.leyva@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>henrik.edin@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1fl1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0y8hc:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6492">Execution Team 2022-11-14</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1f77b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>