<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:39:23 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-35282] Reading from unreplicated collections in a transaction has undefined behavior</title>
                <link>https://jira.mongodb.org/browse/SERVER-35282</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Consider a replica set primary with &lt;tt&gt;top of oplog = logical clock = commit point = T&lt;/tt&gt;. A writer that creates an &lt;b&gt;unreplicated&lt;/b&gt; collection (in a KVStorageEngine such as WT) takes the following steps:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Acquire locks&lt;/li&gt;
	&lt;li&gt;Create collection in the &lt;tt&gt;_mdb_catalog&lt;/tt&gt;&lt;/li&gt;
	&lt;li&gt;Commits the transaction&lt;/li&gt;
	&lt;li&gt;Runs onCommit handlers, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e573d7f2f908f3fbe96716851cd1b1e3d65fe7c9/src/mongo/db/catalog/index_catalog_impl.cpp#L491-L497&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;setting the min visible on the collection to T&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Releases locks&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;A reader starting a new transaction and acquiring a collection takes the following steps:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Open a transaction with read timestamp of &lt;tt&gt;T&lt;/tt&gt;.&lt;/li&gt;
	&lt;li&gt;Acquire locks (I believe these are quickly interrupted if there&apos;s lock contention).&lt;/li&gt;
	&lt;li&gt;Note the minVisible on the collection = T which &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e573d7f2f908f3fbe96716851cd1b1e3d65fe7c9/src/mongo/db/db_raii.cpp#L236-L247&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;is satisfied&lt;/a&gt; via the transaction read timestamp of T.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;If the reader opens the transaction (R1) before the writer commits (W3) and acquires locks (R2) after the writer releases (W4), the reader will observe the in-memory catalog believing the collection exists, but the storage engine will not (due to the concurrent transactions). In debug builds, this typically manifests as a crash when trying to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e573d7f2f908f3fbe96716851cd1b1e3d65fe7c9/src/mongo/db/storage/bson_collection_catalog_entry.cpp#L197&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;access any index&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;It&apos;s unclear how this can manifest with arbitrary queries/sorts/aggregations being performed with the query code consuming this state where the in-memory catalog is inconsistent with the storage engine&apos;s recovery unit snapshot.&lt;/p&gt;</description>
                <environment></environment>
        <key id="551244">SERVER-35282</key>
            <summary>Reading from unreplicated collections in a transaction has undefined behavior</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="spencer@mongodb.com">Spencer Brody</assignee>
                                    <reporter username="daniel.gottlieb@mongodb.com">Daniel Gottlieb</reporter>
                        <labels>
                    </labels>
                <created>Wed, 30 May 2018 16:12:52 +0000</created>
                <updated>Fri, 27 Oct 2023 20:43:24 +0000</updated>
                            <resolved>Thu, 7 Jun 2018 20:55:39 +0000</resolved>
                                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1914357" author="spencer" created="Thu, 7 Jun 2018 20:55:12 +0000"  >&lt;p&gt;While this issue theoretically still exists for any unreplicated collection read in a transaction, we believe that there are no unreplicated collections remaining that it is possible to read in a transaction (since we disallow accessing system collections or collections in the admin, config, and local databases), now that &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35365&quot; title=&quot;MapReduce temporary inc collections should be written to the local database&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35365&quot;&gt;&lt;del&gt;SERVER-35365&lt;/del&gt;&lt;/a&gt; is complete.  So effectively this issue has gone away, as there is no longer a way to actually trigger it.&lt;/p&gt;</comment>
                            <comment id="1907591" author="spencer" created="Thu, 31 May 2018 23:19:05 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt; and I just went through all uses of UnreplicatedWritesBlock, and I believe the only collections that could actually be affected by this issue (and aren&apos;t already disallowed in transactions by virtue of being system collections or living in the config or local dbs) is mapReduce temp collections.  MapReduce writes to collection names that are valid for users to write to and would otherwise be replicated, but all the writes to that collection happen in an UnreplicatedWritesBlock.  I&apos;m not sure if there&apos;s really any way for the transaction system to know that these collections are special, since neither their name nor anything in the catalog about them are special, it&apos;s just the way the system writes to them that is unique.  I think the only way to fix this might be to change the collection name that mapReduce writes to to either start with &apos;system&apos; or be in the local database.&lt;/p&gt;

&lt;p&gt;Doing that would also have the advantage of fixing &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27147&quot; title=&quot;Do not initial sync data written with setReplicatedWrites(false)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27147&quot;&gt;&lt;del&gt;SERVER-27147&lt;/del&gt;&lt;/a&gt; while we&apos;re at it.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1907074" author="max.hirschhorn@10gen.com" created="Thu, 31 May 2018 18:00:07 +0000"  >&lt;p&gt;I don&apos;t think &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r4.1.0/src/mongo/db/repl/replication_coordinator.cpp#L67-L97&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the current implementation of &lt;tt&gt;ReplicationCoordinator::isOplogDisabledFor()&lt;/tt&gt;&lt;/a&gt; is a complete answer to the question of &quot;is this collection replicated?&quot; as the reads and writes inside of a transaction would also be in an entirely separate &lt;tt&gt;OperationContext&lt;/tt&gt; and therefore wouldn&apos;t know about &lt;tt&gt;UnreplicatedWriteBlock&lt;/tt&gt; previously being used for that namespace.&lt;/p&gt;</comment>
                            <comment id="1905129" author="spencer" created="Wed, 30 May 2018 19:53:22 +0000"  >&lt;p&gt;Yeah, I think Kyle&apos;s right.  We should forbid all operations, reads or writes, on unreplicated collections inside transactions.&lt;/p&gt;</comment>
                            <comment id="1905031" author="kyle.suarez" created="Wed, 30 May 2018 19:00:48 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-33115&quot; title=&quot;Prevent writes to unreplicated collections from within transactions&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-33115&quot;&gt;&lt;del&gt;SERVER-33115&lt;/del&gt;&lt;/a&gt; added a mechanism to prevent writes to unreplicated collections; perhaps it should be moved to a more general place so that reads are banned, too.&lt;/p&gt;</comment>
                            <comment id="1905012" author="milkie" created="Wed, 30 May 2018 18:48:31 +0000"  >&lt;p&gt;I wonder if we should simply prohibit reads to such collections.  In the BF, a reader was attempting to read from a temporary unreplicated map/reduce collection; such reads have no real-world use case.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="553260">SERVER-35365</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="492125">SERVER-33115</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="15640"><![CDATA[v4.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 30 May 2018 18:48:31 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 35 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-35365'>SERVER-35365</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-718</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 35 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>kyle.suarez@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htzarz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htn6n3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2297">Repl 2018-06-18</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htyx1b:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>