<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:36:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-34440] Secondary reads from internal (non-network) clients should read at the last applied timestamp</title>
                <link>https://jira.mongodb.org/browse/SERVER-34440</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When checking whether or not to read from the local snapshot (last applied timestamp) on a secondary, we restrict to clients where &lt;tt&gt;isFromUserConnection&lt;/tt&gt; is true. This is to prevent a bug where rolled back index builds are still visible (similar to the invariant in BF-8258). &lt;a href=&quot;https://evergreen.mongodb.com/task/mongodb_mongo_master_linux_64_debug_replica_sets_patch_40cad604c1401f1ccfe66bcee011d824b2cac27e_5acf8e5a2fbabe797bc3a1e3_18_04_12_16_52_06##%257B%2522compare%2522%253A%255B%257B%2522hash%2522%253A%252240cad604c1401f1ccfe66bcee011d824b2cac27e%2522%257D%255D%257D&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;See this patch build failure &lt;/a&gt;where the check for &lt;tt&gt;isFromUserConnection&lt;/tt&gt; is removed:&lt;/p&gt;

&lt;p&gt;The most likely cause is that rollback does not correctly rewind the last applied snapshot timestamp. We should evaluate how replication rolls back its timestamps and ensure that the local snapshot timestamp is correctly updated.&lt;/p&gt;

&lt;p&gt;The solution should be for users of &lt;tt&gt;ShouldNotConflictWithSecondaryBatchApplication&lt;/tt&gt; who opt-out of taking the PBWM lock should be able to read without a timestamp. This applies to the threads listed in the comments, with the addition of &lt;tt&gt;rsBackgroundSync&lt;/tt&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="526144">SERVER-34440</key>
            <summary>Secondary reads from internal (non-network) clients should read at the last applied timestamp</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="louis.williams@mongodb.com">Louis Williams</assignee>
                                    <reporter username="louis.williams@mongodb.com">Louis Williams</reporter>
                        <labels>
                            <label>nyc</label>
                    </labels>
                <created>Thu, 12 Apr 2018 18:19:08 +0000</created>
                <updated>Sun, 29 Oct 2023 22:32:46 +0000</updated>
                            <resolved>Mon, 14 May 2018 20:44:42 +0000</resolved>
                                                    <fixVersion>4.0.0-rc0</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="1890997" author="xgen-internal-githook" created="Mon, 14 May 2018 20:43:52 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Louis Williams&apos;, &apos;email&apos;: &apos;louis.williams@mongodb.com&apos;, &apos;username&apos;: &apos;louiswilliams&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34440&quot; title=&quot;Secondary reads from internal (non-network) clients should read at the last applied timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34440&quot;&gt;&lt;del&gt;SERVER-34440&lt;/del&gt;&lt;/a&gt; Internal readers who opt-out of the PBWM lock when using AutoGetCollectionForRead should not read at the last-applied timestamp.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/4e5d23881a130d854bf0b546ef7e4a6902c3c5b9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/4e5d23881a130d854bf0b546ef7e4a6902c3c5b9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1886171" author="louis.williams" created="Tue, 8 May 2018 18:30:10 +0000"  >&lt;p&gt;Discussed with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=spencer&quot; class=&quot;user-hover&quot; rel=&quot;spencer&quot;&gt;spencer&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;judah@mongodb.com&quot;&gt;judah@mongodb.com&lt;/a&gt;. We believe there is no reason that any of the internal readers listed above, nor any readers that opt-out of taking the PBWM lock using &lt;tt&gt;ShouldNotConflictWithSecondaryBatchApplicationBlock&lt;/tt&gt; should read at the last applied timestamp. These readers would already have be exposed to reading inconsistent data, if any. For oplog reads by FTDC or repl, we do not advance the oplog &quot;all-committed&quot; time until the end of batches, so there is no additional risk posed to readers. Rollback needs to read without a timestamp when reloading the catalog, or the bug described above can occur. In all known cases, reading without a timestamp from internal readers is acceptable behavior.&lt;/p&gt;

&lt;p&gt;&lt;del&gt;No further work is is required for this ticket.&lt;/del&gt;&lt;/p&gt;</comment>
                            <comment id="1878549" author="louis.williams" created="Mon, 30 Apr 2018 22:01:49 +0000"  >&lt;p&gt;I have identified at least one failure with rollback related to this change. Consider the following scenario:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;A rollback occurs. A dropIndex command is one of the rolled-back operations.&lt;/li&gt;
	&lt;li&gt;The last applied timestamp on the rolled-back secondary is reset to time T1 and the oplog is truncated. The primary is at time T2, the current clusterTime.&lt;/li&gt;
	&lt;li&gt;&lt;em&gt;The index re-build on the secondary is timestamped at time T2, which is ahead of the last applied time, T1&lt;/em&gt;&lt;/li&gt;
	&lt;li&gt;The collection catalog is refreshed with a read timestamp at T1, when the index is not ready yet. The in-memory catalog information about the index is that it is unfinished.&lt;/li&gt;
	&lt;li&gt;Replication continues normally until time T3 when the secondary is up-to-date.&lt;/li&gt;
	&lt;li&gt;A reader on the secondary reads at timestamp T3. The on-disk catalog shows the index as complete, but because the in-memory catalog was loaded at timestamp T1 when the index was incomplete, an &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c68c5f493b042350f61dff7aaa32aa2e3e6d651e/src/mongo/db/catalog/index_catalog_impl.cpp#L1133&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;invariant is triggered&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Possible solutions:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Don&apos;t timestamp index builds at the current clusterTime. This is similar to &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34343&quot; title=&quot;Initial sync should not timestamp secondary indexes at the current &amp;#39;clusterTime&amp;#39;&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34343&quot;&gt;&lt;del&gt;SERVER-34343&lt;/del&gt;&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;Don&apos;t refresh (or refresh again) the catalog once time T2 is the current last applied timestamp on the secondary&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1878148" author="louis.williams" created="Mon, 30 Apr 2018 18:00:30 +0000"  >&lt;p&gt;The following internal threads use AutoGetCollectionForRead, and would be affected by reading from the last applied timestamp:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&quot;ftdc&quot;: Reads from the oplog&lt;/li&gt;
	&lt;li&gt;&quot;rsBackgroundSync&quot;: Reads from both oplog and replicated collections&lt;/li&gt;
	&lt;li&gt;&quot;ReplBatcher&quot;: Reads from the oplog&lt;/li&gt;
	&lt;li&gt;&quot;monitoring keys for HMAC&quot;: Reads from admin.system.keys&lt;/li&gt;
&lt;/ul&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="522695">SERVER-34343</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 8 May 2018 15:57:11 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 39 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-644</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 39 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htv2yf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htnotb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2320">Storage NYC 2018-05-21</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htup6n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>