<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:16:05 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-48083] Support rolling back a collMod oplog entry that modifies an index when using rollback via refetch</title>
                <link>https://jira.mongodb.org/browse/SERVER-48083</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently, attempting to roll back a collMod oplog entry that targets an &#8216;index&#8217; field will trigger an &lt;a href=&quot;https://github.com/mongodb/mongo/blob/f2aa1ffe05804aa3cc21ad5f980bca998dde09f3/src/mongo/db/repl/rs_rollback.cpp#L691-L694&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;fassert during rollback via refetch&lt;/a&gt;. This is true for collMod commands that modify an index&#8217;s hidden field as well as those which modify an index&#8217;s TTL option. Since changes to an index&#8217;s TTL option &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-30999&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;are not considered eligible for rollback via refetch&lt;/a&gt;, neither are changes to an index&#8217;s hidden option considered eligible at present. &lt;/p&gt;</description>
                <environment></environment>
        <key id="1346351">SERVER-48083</key>
            <summary>Support rolling back a collMod oplog entry that modifies an index when using rollback via refetch</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="mihai.andrei@mongodb.com">Mihai Andrei</assignee>
                                    <reporter username="mihai.andrei@mongodb.com">Mihai Andrei</reporter>
                        <labels>
                            <label>qexec-team</label>
                    </labels>
                <created>Mon, 11 May 2020 14:19:33 +0000</created>
                <updated>Mon, 22 Jun 2020 13:18:23 +0000</updated>
                            <resolved>Mon, 22 Jun 2020 13:18:23 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="3217123" author="mihai.andrei" created="Mon, 22 Jun 2020 13:18:23 +0000"  >&lt;p&gt;Filed &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-13722&quot; title=&quot;Investigate changes in SERVER-48083: Support rolling back a collMod oplog entry that modifies an index when using rollback via refetch&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-13722&quot;&gt;&lt;del&gt;DOCS-13722&lt;/del&gt;&lt;/a&gt; to track work to update documentation for &apos;enableMajorityReadConcern&apos;.&lt;/p&gt;</comment>
                            <comment id="3211351" author="mihai.andrei" created="Tue, 16 Jun 2020 21:51:05 +0000"  >&lt;p&gt;That sounds reasonable; I think the place to note this would be within &lt;a href=&quot;https://docs.mongodb.com/manual/reference/configuration-options/#replication.enableMajorityReadConcern&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the documentation for enableMajorityReadConcern&lt;/a&gt;. If it&apos;s alright with everyone, I&apos;ll close this ticket as won&apos;t fix and file a docs ticket to update the linked section in the docs.&lt;/p&gt;</comment>
                            <comment id="3210440" author="daniel.gottlieb@10gen.com" created="Tue, 16 Jun 2020 18:17:44 +0000"  >&lt;p&gt;If we choose to go ahead with advertising hidden indexes in our 4.4 docs, we should add a note to our docs that replication rollback fails to undo the &lt;tt&gt;collMod&lt;/tt&gt; operation and that the affected nodes can only recover with a resync.&lt;/p&gt;</comment>
                            <comment id="3210388" author="mihai.andrei" created="Tue, 16 Jun 2020 17:56:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; Yes, hidden indexes are documented for 4.4, per DOCSP-8660. As to whether there is a workaround for this that doesn&#8217;t involve resyncing, I&#8217;m not sure.&lt;/p&gt;</comment>
                            <comment id="3210344" author="daniel.gottlieb@10gen.com" created="Tue, 16 Jun 2020 17:32:56 +0000"  >&lt;p&gt;Are hidden indexes documented for 4.4?&lt;/p&gt;

&lt;p&gt;If a user hits the fassert trying to rollback the operation, is there a workaround to getting the node back into the replica set without requiring a resync?&lt;/p&gt;</comment>
                            <comment id="3210312" author="mihai.andrei" created="Tue, 16 Jun 2020 17:25:35 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=william.schultz&quot; class=&quot;user-hover&quot; rel=&quot;william.schultz&quot;&gt;william.schultz&lt;/a&gt; With regards to your interpretation of how hidden indexes is used, I think your intuition is correct: a user might configure an index&apos;s hidden field during some time window, after which they keep it hidden or disable it indefinitely. &lt;/p&gt;

&lt;p&gt;Unless anyone feels strongly about doing this work or it comes to pass that eMRC=false won&apos;t be removed in 4.6, I&apos;m going to close this ticket as &apos;Won&apos;t fix&apos; .&lt;/p&gt;</comment>
                            <comment id="3204914" author="william.schultz" created="Thu, 11 Jun 2020 13:14:44 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mihai.andrei&quot; class=&quot;user-hover&quot; rel=&quot;mihai.andrei&quot;&gt;mihai.andrei&lt;/a&gt; With regards to implementation, it turns out we actually added initial support for making collMod oplog entries &quot;reversible&quot; in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-28205&quot; title=&quot;Recoverable Rollback: Extend TTL-related collMod oplog entry to include previous expiration&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-28205&quot;&gt;&lt;del&gt;SERVER-28205&lt;/del&gt;&lt;/a&gt; (e.g. see the &quot;collectionOptions_old&quot; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6836555e1d9734e21a681f3495d8febbfb4621ee/src/mongo/db/op_observer_impl.cpp#L670&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;field&lt;/a&gt;). We never utilized the extra information in rollbackViaRefetch, though. This would probably make it easier to add support for rolling back collMod oplog entries that change collection options, though. Interestingly, it looks like we also added the value of the &quot;old&quot; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6836555e1d9734e21a681f3495d8febbfb4621ee/src/mongo/db/op_observer_impl.cpp#L678-L679&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;hidden field&lt;/a&gt; to the collMod oplog entry as a part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-9306&quot; title=&quot;Ability to temporarily forbid query optimizer from using index (&amp;quot;invisible index&amp;quot;)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-9306&quot;&gt;&lt;del&gt;SERVER-9306&lt;/del&gt;&lt;/a&gt;, which might further ease the implementation.&lt;/p&gt;

&lt;p&gt;To address your questions about priority/value of implementing this feature: it seems likely that enableMajorityReadConcern:false will be going away in 4.6 (PM-1769), but I don&apos;t think it has been firmly decided yet. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; might have the better understanding of that decision. So, we would then only be worried about supporting rollback of these hidden index changes on 4.4 clusters that use enableMajorityReadConcern:false. All things considered, I would predict that roll back of a hidden index change operation on a EMRC=false node is relatively rare, but determining whether we need/want to support that might be more of a question for some product people, if they, for example, have a better sense of how many users will also be using the hidden indexes feature. With my current understanding, I would probably think that it is reasonable to not add support for it in rollbackViaRefetch. On the other hand, if the implementation is relatively simple (and doesn&apos;t require refetching any extra documents from the sync source, for example), then it might be worth doing.&#160;&lt;/p&gt;

&lt;p&gt;To also address &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;&apos;s point:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;presumably because adding/removing the hidden field is going to be more common than changing an index&apos;s TTL time&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;If I understand correctly, a hidden index might be built but not enabled yet, so that users can optionally &quot;turn it on&quot; to see how it affects performance. This doesn&apos;t seem like a very frequent operation i.e. you might turn it on and test the index performance for a bit, and then possibly make a decision to turn it off or leave it on permanently. I don&apos;t really see why users would be continually turning it off and on. I suppose if they have many indexes and/or many collections they might be testing out various combinations of hidden/non-hidden indexes, which could contribute to a higher rate of hidden index modifications over a certain period of time. My intuition, however, would be that these changes would be relatively localized in time i.e. there might be some experimentation window where users are learning about the perf characteristics of certain indexes, and then end up leaving the hidden settings fixed. Of course, this intuition could be misaligned with how users will actually use the feature. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mihai.andrei&quot; class=&quot;user-hover&quot; rel=&quot;mihai.andrei&quot;&gt;mihai.andrei&lt;/a&gt; you might be able to make a better judgement on it than me.&lt;/p&gt;</comment>
                            <comment id="3204245" author="mihai.andrei" created="Wed, 10 Jun 2020 23:34:14 +0000"  >&lt;p&gt;If I understand correctly, rollback via refetch only gets used when enableMajorityReadConcern is set to false, forceRollbackViaRefetch is set to true, or the ephemeralForTest storage engine is used. Given this, the decision to add support for rolling back collMod entries which modify an index&#8217;s &#8216;hidden&#8217; field (and not the TTL option, since this isn&apos;t supported as of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-30999&quot; title=&quot;Fix roll back of collMod TTL changes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-30999&quot;&gt;&lt;del&gt;SERVER-30999&lt;/del&gt;&lt;/a&gt;) depends on how likely it is that users can run into this issue. In particular:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;How often are clusters deployed with majority read concern explicitly disabled?&lt;/li&gt;
	&lt;li&gt;Will enableMajorityReadConcern=false be removed in 4.6?&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;For what it&#8217;s worth, I don&#8217;t think this fix is terribly complicated, since, if I understand rollback via refetch correctly, undoing a collMod on an index&apos;s hidden field consists of inverting its current value (as well as doing any necessary work to remove redundant operations like flipping the hidden value of an index that is going to be dropped, for example). That said, I would like to get a sense of how important it is to support this before committing to implementing it. Intuitively, I would think that if we don&apos;t support rolling back a collMod for TTL options, we wouldn&apos;t do so for any index options (including hidden), but &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; makes a good point that modifying an index&apos;s TTL option would probably happen less frequently than modifying its hidden option.&lt;/p&gt;

&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=william.schultz&quot; class=&quot;user-hover&quot; rel=&quot;william.schultz&quot;&gt;william.schultz&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3081963" author="milkie" created="Tue, 12 May 2020 20:59:47 +0000"  >&lt;p&gt;The problem that Mihai is describing is related to the code that was added as part of the Hidden Indexes project.  We never implemented rollback for index TTL values, so the new code added by Hidden Indexes didn&apos;t get that support either.  This ticket is to consider adding that support, presumably because adding/removing the hidden field is going to be more common than changing an index&apos;s TTL time, and thus the danger of hitting this fassert is greater.&lt;/p&gt;</comment>
                            <comment id="3080960" author="craig.homa" created="Tue, 12 May 2020 15:44:35 +0000"  >&lt;p&gt;It isn&apos;t clear to Query that this falls under their responsibilities and we are not sure why it is currently not feasible to rollback a collMod oplog entry under these circumstances. Could you shed some light on what work would be needed to permit this?&lt;/p&gt;

&lt;p&gt;CC &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="1385713">DOCS-13722</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 12 May 2020 15:44:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 33 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>mihai.andrei@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 33 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>craig.homa@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>mihai.andrei@mongodb.com</customfieldvalue>
            <customfieldvalue>william.schultz@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxkm8n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hx7wk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="3608">Query 2020-06-29</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hxk8hz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>