<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:31:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-32851] setFeatureCompatibilityVersion can race with createCollection such that FCV 3.6 is set and some collections do not have UUIDs</title>
                <link>https://jira.mongodb.org/browse/SERVER-32851</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Create collection &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a98d497c957dc2da7d29c37be9809ace992ef946/src/mongo/db/catalog/database_impl.cpp#L766-L782&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;takes a database exclusive lock and then checks the FCV&lt;/a&gt; to determine whether to create a collection with a UUID or not. The setFeatureCompatibilityVersion command, however, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a98d497c957dc2da7d29c37be9809ace992ef946/src/mongo/db/commands/set_feature_compatibility_version_command.cpp#L156&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;sets the FCV to upgrading&lt;/a&gt; &lt;b&gt;without taking any locks&lt;/b&gt;. Therefore, createCollection could get as far as checking the FCV, not seeing upgradingTo36 or fullyUpgradedTo36, then the setFeatureCompatibilityVersion can assign all the existing collections to have UUIDs, and finally the createCollection finishes &lt;b&gt;without assigning a UUID to the new collection&lt;/b&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="486188">SERVER-32851</key>
            <summary>setFeatureCompatibilityVersion can race with createCollection such that FCV 3.6 is set and some collections do not have UUIDs</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="dianna.hohensee@mongodb.com">Dianna Hohensee</reporter>
                        <labels>
                            <label>SWNA</label>
                    </labels>
                <created>Tue, 23 Jan 2018 15:38:10 +0000</created>
                <updated>Mon, 30 Oct 2023 23:09:05 +0000</updated>
                            <resolved>Tue, 30 Jan 2018 15:57:11 +0000</resolved>
                                    <version>3.6.0</version>
                    <version>3.7.1</version>
                                    <fixVersion>3.6.3</fixVersion>
                    <fixVersion>3.7.2</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="2991088" author="bigbourin@gmail.com" created="Sat, 21 Mar 2020 10:00:54 +0000"  >&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;We&apos;re having some trouble which this upgrade on our RS already reported to Enterprise support under ticket 00642150&lt;br/&gt;
I created a 3 minutes downtime when starting the migration because of this global S lock because as far as we saw it&apos;s locking every write operation not just create collections, fortunately after 3 minutes the server crashed which allowed the fail-over to secondary. When we looked at the number of collections with UUIDs we noticed only 140 of them have been converted (we have more than 8k) which means the migration should take around 3h locking the whole database in write. This is not really acceptable of course &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/wink.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;So I&apos;m wondering what has gone wrong in this change, you seem to say that the operation can take a long time:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;which could take a long time and does a lot of network calls&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;But on the other end you end up tacking a global S lock for the duration of the UUIDs update (and not just the flag set) which (as far as I know and we noticed in production) will block every write, because they need IX on all ancestors, right?&lt;/p&gt;

&lt;p&gt;I don&apos;t see any update to this code until 3.6.17 so did I misunderstood something or is this migration supposed to block everybody&apos;s database for any duration between a couple of seconds to several hours?&lt;/p&gt;

&lt;p&gt;Thanks for your help.&lt;/p&gt;</comment>
                            <comment id="1790752" author="xgen-internal-githook" created="Wed, 31 Jan 2018 17:03:34 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;dianna.hohensee@10gen.com&apos;, &apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32851&quot; title=&quot;setFeatureCompatibilityVersion can race with createCollection such that FCV 3.6 is set and some collections do not have UUIDs&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-32851&quot;&gt;&lt;del&gt;SERVER-32851&lt;/del&gt;&lt;/a&gt; Prevent setFCV and createCollection racing such that a collection has UUID in FCV 3.4 or doesn&apos;t in FCV 3.6&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 9081e227228f11744200fbe66a9683b0bd82e5b1)&lt;br/&gt;
Branch: v3.6&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/816defceefd0ca0d0a89cf9b747b41aa08617375&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/816defceefd0ca0d0a89cf9b747b41aa08617375&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1789123" author="xgen-internal-githook" created="Tue, 30 Jan 2018 15:47:49 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;dianna.hohensee@10gen.com&apos;, &apos;name&apos;: &apos;Dianna Hohensee&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32851&quot; title=&quot;setFeatureCompatibilityVersion can race with createCollection such that FCV 3.6 is set and some collections do not have UUIDs&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-32851&quot;&gt;&lt;del&gt;SERVER-32851&lt;/del&gt;&lt;/a&gt; Prevent setFCV and createCollection racing such that a collection has UUID in FCV 3.4 or doesn&apos;t in FCV 3.6&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/9081e227228f11744200fbe66a9683b0bd82e5b1&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/9081e227228f11744200fbe66a9683b0bd82e5b1&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1787889" author="schwerin" created="Mon, 29 Jan 2018 17:12:29 +0000"  >&lt;p&gt;I think the current approach is acceptable. Normally, I&apos;d prefer to delay the DDL ops in favor of unblocking the CRUD ops, but this is a really rare event, and the global S lock proposal is really easy to understand.&lt;/p&gt;</comment>
                            <comment id="1786531" author="spencer" created="Fri, 26 Jan 2018 22:22:57 +0000"  >&lt;p&gt;The fix in the CR works by taking and immediately releasing a global S lock after setting the FCV targetVersion.   This creates a barrier for createCollection such that the collection will either be created after setting the targetVersion, and thus createCollection will make it with a UUID, or the createCollection happens before, but is guaranteed to be completed by the time the FCV command starts assigning UUIDs to existing collections.&lt;/p&gt;

&lt;p&gt;An alternative approach that &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dianna.hohensee&quot; class=&quot;user-hover&quot; rel=&quot;dianna.hohensee&quot;&gt;dianna.hohensee&lt;/a&gt; and I just discussed is that we could instead take and hold global and database IX locks for the duration of where we list databases and collections and set UUIDs on them.&lt;/p&gt;

&lt;p&gt;The latter approach has the advantage that crud write ops are never blocked, but the downside that DDL ops are blocked for a longer window of time.  I think both approaches are valid, and I don&apos;t have a strong preference for which we do, I just thought it&apos;d be worth considering both.&lt;/p&gt;

&lt;p&gt;If no one else has a strong opinion then I&apos;m included to default to the one that&apos;s already written, using the global S lock barrier.&lt;/p&gt;</comment>
                            <comment id="1786333" author="dianna.hohensee" created="Fri, 26 Jan 2018 19:59:23 +0000"  >&lt;p&gt;Per offline discussion with Andy, going back to using a global lock, because adding too many miscellaneous locks outside of the global/database/collection hierarchy confuses lock taking order expectations and can easily lead to deadlocks.&lt;/p&gt;</comment>
                            <comment id="1784079" author="dianna.hohensee" created="Wed, 24 Jan 2018 21:34:47 +0000"  >&lt;p&gt;The proposed solution for this ticket is to create a new ResourceMutex lock. createCollection will take it in shared mode for the duration of its operations. setFCV will take it in exclusive mode only while doing the FCV document write. This will prevent the FCV document from being updated in the middle of createCollection operations, between determining whether an UUID is needed and actually creating the collection. &lt;/p&gt;

&lt;p&gt;So createCollection will be able to run in parallel with other createCollection calls (as long as different dbs because createCollection already takes a db X lock), and no global exclusive lock is needed.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;note&lt;/b&gt;: the existing fcvLock cannot be used because it would prevent createCollection from running for the duration of setFCV, which could take a long time and does a lot of network calls.&lt;/p&gt;</comment>
                            <comment id="1784043" author="dianna.hohensee" created="Wed, 24 Jan 2018 21:21:03 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=schwerin&quot; class=&quot;user-hover&quot; rel=&quot;schwerin&quot;&gt;schwerin&lt;/a&gt;, I&apos;m going to take advantage of the fcvLock instead of taking the global exclusive lock in setFCV. Updating the title now. Do you still want repl on the CR?&lt;/p&gt;

&lt;p&gt;&lt;b&gt;update&lt;/b&gt;: I&apos;ll actually make a new ResourceMutex lock, because the existing fcvLock is held too long.&lt;/p&gt;</comment>
                            <comment id="1782887" author="ian@10gen.com" created="Tue, 23 Jan 2018 22:40:38 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=dianna.hohensee&quot; class=&quot;user-hover&quot; rel=&quot;dianna.hohensee&quot;&gt;dianna.hohensee&lt;/a&gt; request from Andy to have someone from Repl also review this please.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15141"><![CDATA[v3.6]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 23 Jan 2018 22:40:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 46 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 46 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bigbourin@gmail.com</customfieldvalue>
            <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>ian@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htok33:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hteuo7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2094">Storage 2018-01-29</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hto67j:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>