<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:37:04 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-34556] Iterating through indexes when reading from a timestamp behind the minimumVisibleSnapshot should not cause an invariant</title>
                <link>https://jira.mongodb.org/browse/SERVER-34556</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;An &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a4108de09a0703f550ec4f15764f8ad75e756271/src/mongo/db/catalog/index_catalog_impl.cpp?utf8=%E2%9C%93#L1203&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;invariant is hit&lt;/a&gt; when an index exists in memory (because it has completed), but it is not yet visible on the storage engine when reading at an earlier timestamp.&lt;/p&gt;

&lt;p&gt;A query that reads from a timestamp is allowed to yield. When its locks are reacquired and it reads again, say it reads from a newer timestamp, T1. If an index build occurs during the yield at T2, and the read timestamp on the query is before the index build timestamp, iterating through the list of indexes on a collection can trigger this invariant.&lt;/p&gt;

&lt;p&gt;Rather than always triggering an invariant, we should just not show the index as ready. This will only be the case when the current operation is reading from a timestamp and the index is marked ready in memory but not visible on the storage engine.&lt;/p&gt;</description>
                <environment></environment>
        <key id="530639">SERVER-34556</key>
            <summary>Iterating through indexes when reading from a timestamp behind the minimumVisibleSnapshot should not cause an invariant</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="louis.williams@mongodb.com">Louis Williams</reporter>
                        <labels>
                    </labels>
                <created>Wed, 18 Apr 2018 22:11:51 +0000</created>
                <updated>Sun, 29 Oct 2023 22:32:35 +0000</updated>
                            <resolved>Thu, 17 May 2018 15:35:52 +0000</resolved>
                                    <version>3.7.5</version>
                                    <fixVersion>4.0.0-rc0</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1894025" author="xgen-internal-githook" created="Thu, 17 May 2018 15:34:16 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;dianna.hohensee@10gen.com&apos;, &apos;username&apos;: &apos;DiannaHohensee&apos;, &apos;name&apos;: &apos;Dianna Hohensee&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34556&quot; title=&quot;Iterating through indexes when reading from a timestamp behind the minimumVisibleSnapshot should not cause an invariant&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34556&quot;&gt;&lt;del&gt;SERVER-34556&lt;/del&gt;&lt;/a&gt; Iterating through indexes for a last applied timestamp transaction should use that timestamp&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/04fc6cc223b1599131d912c7e3654001f561b07b&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/04fc6cc223b1599131d912c7e3654001f561b07b&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1893745" author="dianna.hohensee" created="Thu, 17 May 2018 12:58:58 +0000"  >&lt;p&gt;Thanks for the pointers, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;!&lt;/p&gt;

&lt;p&gt;In conclusion, it appears for kLastApplied reads that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6d2de545a7cfcf4ab23dcf73426a1d50896d6d0c/src/mongo/db/catalog/index_catalog_impl.cpp#L1203&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this check would&lt;/a&gt; return false and the code would continue on to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6d2de545a7cfcf4ab23dcf73426a1d50896d6d0c/src/mongo/db/catalog/index_catalog_impl.cpp#L1211&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;hit the invariant&lt;/a&gt;. This is because &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6d2de545a7cfcf4ab23dcf73426a1d50896d6d0c/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L328&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;getPointInTimeReadTimestamp returns boost::none for kLastApplied&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Th invariant is hit because IndexCatalogEntryImpl is unaware of timestamps and depends upon IndexCatalogImpl::IndexIteratorImpl to manage timestamps such that it (IndexCatalogEntryImpl) never must. In this case&#160;IndexCatalogImpl::IndexIteratorImpl did not manage it correctly.&lt;/p&gt;</comment>
                            <comment id="1891990" author="louis.williams" created="Tue, 15 May 2018 18:18:30 +0000"  >&lt;p&gt;After &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34443&quot; title=&quot;Refactor how WiredTigerRecoveryUnit and WiredTigerSnapshotManager begin transactions at points in time&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34443&quot;&gt;&lt;del&gt;SERVER-34443&lt;/del&gt;&lt;/a&gt;, this fix is much more straightforward:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;when reading from &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6d2de545a7cfcf4ab23dcf73426a1d50896d6d0c/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L362&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;kLastApplied&lt;/a&gt;, set the &lt;tt&gt;_readAtTimestamp&lt;/tt&gt; with the return value from &lt;tt&gt;beginTransactionOnLocalSnapshot()&lt;/tt&gt;&lt;/li&gt;
	&lt;li&gt;change &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6d2de545a7cfcf4ab23dcf73426a1d50896d6d0c/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L317-L318&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this condition in getPointInTimeReadTimestamp()&lt;/a&gt;&#160;to include &lt;tt&gt;kLastApplied&lt;/tt&gt; to allow returning the value of &lt;tt&gt;_readAtTimestamp&lt;/tt&gt;.&lt;/li&gt;
	&lt;li&gt;Using getPointInTimeReadTimestamp() can now return the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/6d2de545a7cfcf4ab23dcf73426a1d50896d6d0c/src/mongo/db/catalog/index_catalog_impl.cpp#L1203&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;last applied timestamp here&lt;/a&gt;, and allow checking the read timestamp against the minVisibleSnapshot on the collection&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1890966" author="daniel.gottlieb@10gen.com" created="Mon, 14 May 2018 20:21:11 +0000"  >&lt;p&gt;Perhaps an alternative perspective, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2edb91e2a51f0bdcf89f51b1a3281443c13e52c5/src/mongo/db/catalog/index_catalog_impl.cpp#L1203-L1208&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this condition&lt;/a&gt; to prevent from reading behind the &lt;tt&gt;`minSnapshot`&lt;/tt&gt; is ineffective because the recovery unit &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2edb91e2a51f0bdcf89f51b1a3281443c13e52c5/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L317-L318&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;does not acknowledge it is reading from a point in time when the read concern level is local&lt;/a&gt; (which is the default). Moreover, (in the case of reading from last applied) the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2edb91e2a51f0bdcf89f51b1a3281443c13e52c5/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L320-L326&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_readAtTimestamp/_majorityCommittedSnapshot&lt;/a&gt; are the null timestamp.&lt;/p&gt;

&lt;p&gt;It seems the problem could be resolved, if instead the iterating code could reliably know the timestamp that is being read at. In general it seems like a much more useful contract to transparently return the read time.&lt;/p&gt;</comment>
                            <comment id="1876560" author="dianna.hohensee" created="Fri, 27 Apr 2018 19:21:37 +0000"  >&lt;p&gt;Talked with Louis and he believes that &lt;a href=&quot;https://github.com/mongodb/mongo/commit/2deac715b6345a8103acc1da6187f77e65843019&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;a recent commit of his&lt;/a&gt; incidentally fixed this bug.&lt;/p&gt;

&lt;p&gt;The new code now &lt;a href=&quot;https://github.com/mongodb/mongo/commit/2deac715b6345a8103acc1da6187f77e65843019#diff-9632acb32146b264c3c8ee47a852de9aR430&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;advances the minVisibleSnapshot on the in-memory catalog entry&lt;/a&gt; when the index is committed to the storage engine. This should avoid the invariant check performed in &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a4108de09a0703f550ec4f15764f8ad75e756271/src/mongo/db/catalog/index_catalog_impl.cpp?utf8=%E2%9C%93#L1203&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this call&lt;/a&gt; because the code should now &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a4108de09a0703f550ec4f15764f8ad75e756271/src/mongo/db/catalog/index_catalog_impl.cpp?utf8=%E2%9C%93#L1196&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;return true here&lt;/a&gt; and never the invariant. The minVisibleSnapshot of the index is now greater than the reader&apos;s timestamp, and the index is ignored correctly.&lt;/p&gt;

&lt;p&gt;The invariant was originally getting tripped because &lt;a href=&quot;https://github.com/mongodb/mongo/blob/2deac715b6345a8103acc1da6187f77e65843019/src/mongo/db/catalog/index_catalog_entry_impl.cpp?utf8=%E2%9C%93#L166&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this code&lt;/a&gt; ensures in-memory and storage engine both believe either that the index is ready or not ready. With a read started on a timestamp predating the index build, the storage engine would obey that timestamp and say that the index does not exist; whereas the in-memory answer to whether the index is ready is not versioned, and reports yes to ready.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 27 Apr 2018 19:21:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 38 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 38 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>16.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htvuhj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr9ckn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2252">Storage NYC 2018-05-07</customfieldvalue>
    <customfieldvalue id="2320">Storage NYC 2018-05-21</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htvgpr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>