<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:48:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38145] Extend upgradeCheckAllDBs() and/or upgradeCheck() to look for indexes not compliant with version 3.4 index definition rules</title>
                <link>https://jira.mongodb.org/browse/SERVER-38145</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Version 3.4 introduced &lt;a href=&quot;https://docs.mongodb.com/manual/release-notes/3.4-compatibility/#stricter-validation-of-index-specifications&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;stricter validation of index specifications&lt;/a&gt;.  &lt;/p&gt;

&lt;p&gt;However, the new code is only run when an index is recreated, meaning the following scenario is all too possible:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;In version 3.2, an index with key {&lt;tt&gt;cust_id:1, background:true&lt;/tt&gt;} is created.&lt;/li&gt;
	&lt;li&gt;The weaker syntax checking in that version does not catch the missing &lt;tt&gt;}, {&lt;/tt&gt; and  instead of a single field index on cust_id created in background, a compound index is created in the foreground with a non-existent second field called &quot;background&quot; appended to the key).&lt;/li&gt;
	&lt;li&gt;upgrade in place to 3.4 of PSA replica set goes through without error&lt;/li&gt;
	&lt;li&gt;more upgrades to 3.6 and 4.0 can also succeed.&lt;/li&gt;
	&lt;li&gt;Then when hardware failure necessitates the resyncing of a node, it fails on the recreate of this index due to the non-compliant definition.&lt;/li&gt;
	&lt;li&gt;likewise if the OM/CM headDB becomes stale and needs to be resynced, it will fail.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;To enable DBA&apos;s to detect and fix index definitions that do not comply with the stricter rules in version 3.4 and later, it would be nice if the upgradeCheck utilities (or some other new utility) existed that would examine all indexes for a collection/database/cluster and report on those that cannot be upgraded.  &lt;/p&gt;

&lt;p&gt;It would be better to discover and repair them in planned outage windows than in the midst of an already unplanned outage.  &lt;/p&gt;

&lt;p&gt;Many of these boolean fields in  version 3.2 index keys are called unique, sparse and background, meaning they were almost certainly intended to be index options and not part of the key, so the index is not created as intended either, and it is in the customer&apos;s interest to fix that too. &lt;/p&gt;</description>
                <environment></environment>
        <key id="634253">SERVER-38145</key>
            <summary>Extend upgradeCheckAllDBs() and/or upgradeCheck() to look for indexes not compliant with version 3.4 index definition rules</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="kelsey.schubert@mongodb.com">Kelsey Schubert</assignee>
                                    <reporter username="william.byrne@mongodb.com">William Byrne III</reporter>
                        <labels>
                    </labels>
                <created>Thu, 15 Nov 2018 03:05:08 +0000</created>
                <updated>Thu, 9 May 2019 18:02:51 +0000</updated>
                            <resolved>Thu, 9 May 2019 18:02:51 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="2240281" author="thomas.schubert" created="Thu, 9 May 2019 18:02:51 +0000"  >&lt;p&gt;Including this feature inside of MongoDB is expensive, and may require minimum floors before upgrading. Instead, we&apos;re looking into external tools that behave similar to the script William provided which would address the same user pain without concern about which particular version they are are on before upgrading.&lt;/p&gt;</comment>
                            <comment id="2106670" author="william.byrne" created="Mon, 7 Jan 2019 00:17:16 +0000"  >&lt;p&gt;The attached script has been replaced with one that has comments:  &lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/205566/205566_index34.js&quot; title=&quot;index34.js attached to SERVER-38145&quot;&gt;index34.js&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt; &lt;/p&gt;

&lt;p&gt;The side effects if the non-compliant key field comes from index options mistakenly added to the key is as follows:&lt;/p&gt;

&lt;div class=&apos;table-wrap&apos;&gt;
&lt;table class=&apos;confluenceTable&apos;&gt;&lt;tbody&gt;
&lt;tr&gt;
&lt;th class=&apos;confluenceTh&apos;&gt; Intended Index Option &lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt; Actual Key &lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt; Effect &lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; create in background &lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; {lsid:1, background:true} &lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; Index is created in foreground, blocking other jobs &lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; sparse index &lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; {lsid:1, sparse:true} &lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; Index is not sparse, and is being updated for every insert/delete &lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; unique index &lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; {lsid:1, unique:true} &lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt; Index does &lt;b&gt;not&lt;/b&gt; enforce uniqueness of lsid &lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;/div&gt;


&lt;p&gt;The impact of indexes with these mistakes in their definition is mostly small.  Once created, the index that was intended to be built in the background will probably be used by the Query Planner in exactly the way the index with the correct definition would be, and will return the correct results.  This is because the intended key is most commonly an &lt;span class=&quot;error&quot;&gt;&amp;#91;index prefix&amp;#93;&lt;/span&gt;(&lt;a href=&quot;https://docs.mongodb.com/v3.4/core/index-compound/#prefixes&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/v3.4/core/index-compound/#prefixes&lt;/a&gt;) of the actual key.  Similarly, the non-sparse index will assist queries correctly.  &lt;/p&gt;

&lt;p&gt;All three options specification errors will waste space, as the extra non-existent field name at the end of the key will be indexed as a null for every entry.  An index intended to be &lt;tt&gt;sparse&lt;/tt&gt; will waste more space as it will index all the documents with null key field values.&lt;/p&gt;

&lt;p&gt;The most serious potential impact is from uniqueness not being enforced as expected, as that means duplicate documents would be loaded without error.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="205566" name="index34.js" size="1677" author="william.byrne@mongodb.com" created="Sun, 6 Jan 2019 23:03:29 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000bzhSrIAI, 5002K00000dMfi2QAC]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 15 Nov 2018 06:13:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 39 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 39 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>william.byrne@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hucy9r:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu34hr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huckj3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>