<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:02:12 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-43076] What is the correct way to access the OperationContext in the commit method of a RecoveryUnit::Change implementation? </title>
                <link>https://jira.mongodb.org/browse/SERVER-43076</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The opCtx we were under when the handler was created is not valid during commit. What is the prescribed way to access the current opCtx? The approach we are &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_chunk_cloner_source_legacy.cpp#L167&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;currently using&lt;/a&gt;&#160;is to retrieve the opCtx from the current client:&lt;/p&gt;

&lt;p&gt;auto opCtx = cc().getOperationContext();&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="911207">SERVER-43076</key>
            <summary>What is the correct way to access the OperationContext in the commit method of a RecoveryUnit::Change implementation? </summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="henrik.edin@mongodb.com">Henrik Edin</assignee>
                                    <reporter username="alex.taskov@mongodb.com">Alexander Taskov</reporter>
                        <labels>
                    </labels>
                <created>Thu, 29 Aug 2019 13:34:16 +0000</created>
                <updated>Sun, 29 Oct 2023 22:17:36 +0000</updated>
                            <resolved>Tue, 20 Sep 2022 20:03:51 +0000</resolved>
                                                    <fixVersion>6.2.0-rc0</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="4841951" author="xgen-internal-githook" created="Tue, 20 Sep 2022 14:47:58 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Henrik Edin&apos;, &apos;email&apos;: &apos;henrik.edin@mongodb.com&apos;, &apos;username&apos;: &apos;henrikedin&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-43076&quot; title=&quot;What is the correct way to access the OperationContext in the commit method of a RecoveryUnit::Change implementation? &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-43076&quot;&gt;&lt;del&gt;SERVER-43076&lt;/del&gt;&lt;/a&gt; Pass OperationContext through commit and rollback handlers&lt;/p&gt;

&lt;p&gt;Supports multi-document transactions where the OperationContext instance may change.&lt;br/&gt;
The RecoveryUnit contains a back pointer to its currently owning OperationContext.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/48fd6095abaf8d688c1e3e216eadc96262025e17&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/48fd6095abaf8d688c1e3e216eadc96262025e17&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4819108" author="kaloian.manassiev" created="Mon, 12 Sep 2022 15:02:32 +0000"  >&lt;p&gt;I think it is more like that onCommit/onRollback shouldn&apos;t be allowed to access the transaction context, because it is too late for them to be making any changes to it or even to introspect it. We just need the &quot;OS&quot; part of the context, which is about taking locks.&lt;/p&gt;</comment>
                            <comment id="4818909" author="henrik.edin" created="Mon, 12 Sep 2022 14:25:33 +0000"  >&lt;p&gt;I think it would be a nice cleanup to separate the OperationContext into a context for the whole transaction and one for the individual operations within that transaction. But with such a change code would still need to access both as meaningful state exist on both sides. I can buy an argument that in such a world commit/rollback handlers should only need the transaction context part (as they operate on transactions).&lt;/p&gt;

&lt;p&gt;We currently have lots of decorations living on the OperationContext that should probably be moved to the &quot;TransactionContext&quot;. It is going to be some work to fix all this and I think the first step is to simply make it possible to access the correct OperationContext pointer in commit/rollback handlers. Over time, when we have created a &quot;TransactionContext&quot; we can move over to that system.&lt;/p&gt;</comment>
                            <comment id="4818500" author="kaloian.manassiev" created="Mon, 12 Sep 2022 13:00:15 +0000"  >&lt;p&gt;Just related to the previous comment (doesn&apos;t solve the overall problem of this ticket): The OperationContext is a fundamental part of our &quot;tread management&quot; infrastructure and we should not have blocking operations not taking one, both for diagnostics/tracking purposes and for interruptability. If there are places taking ResourceLocks that are not safe to be interrupted, they should be explicitly marked as such.&lt;/p&gt;</comment>
                            <comment id="4818341" author="louis.williams" created="Mon, 12 Sep 2022 12:02:24 +0000"  >&lt;p&gt;I feel that we need to separate OperationContext into the parts that represent a single operation running on a single thread and the parts that represent its longer-lived transactional state that can inhabit multiple OperationContexts.&lt;/p&gt;

&lt;p&gt;Things like Locker and RecoveryUnit can be used on multiple OperationContexts and should probably have their own separate state.&lt;/p&gt;

&lt;p&gt;Part of the reason why we were capturing OpCtx in the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/50b526fecf6e13b2ac8541b69564f368ac17a8ac/src/mongo/db/s/migration_chunk_cloner_source_legacy.cpp#L173&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;LogOpForShardingHandler&lt;/a&gt; was to acquire a ResourceLock, which required an OperationContext argument.&lt;/p&gt;</comment>
                            <comment id="4808413" author="henrik.edin" created="Wed, 7 Sep 2022 15:23:24 +0000"  >&lt;p&gt;Reopening this and proposing that we implement an idiomatic way to get access to the &lt;tt&gt;OperationContext&lt;/tt&gt; in commit/rollback handlers.&lt;/p&gt;

&lt;p&gt;It is already common that we capture the &lt;tt&gt;OperationContext&lt;/tt&gt; in commit handlers. The &lt;tt&gt;OperationContext&lt;/tt&gt; is used to access various decorations or validate some lock state. We also require the &lt;tt&gt;OperationContext&lt;/tt&gt; to be able to perform writes to the &lt;tt&gt;CollectionCatalog&lt;/tt&gt; which can be required in commit/rollback handlers.&lt;/p&gt;

&lt;p&gt;It seems fragile that this pattern works except in the context of multi-document transactions. When more and more support is added within multi-document transactions new code paths that capture an &lt;tt&gt;OperationContext&lt;/tt&gt; may execute in a way that is currently not safe.&lt;/p&gt;</comment>
                            <comment id="2409393" author="daniel.gottlieb@10gen.com" created="Thu, 5 Sep 2019 18:00:57 +0000"  >&lt;p&gt;There&apos;s not currently a better way to do this. Looking at the provided code snippet, I see the operation context is only being used to acquire a lock. One alternative is to instead grab that lock prior to adding a &lt;tt&gt;_stmt&lt;/tt&gt; to the &lt;tt&gt;LogTransactionOperationsForShardingHandler&lt;/tt&gt;. Assuming that lock participates in two phase locking (2PL), that will guarantee the lock is held for the entirety of the transaction. The obvious trade-offs here:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;I suspect the lifetime of a &lt;tt&gt;LogTransactionOperationsForShardingHandler&lt;/tt&gt; can be multiple network hops and holding a lock for those would have a severe negative impact/create possible deadlocks.&lt;/li&gt;
	&lt;li&gt;There&apos;s less granularity for interruption when an operations that needs a competing handle on the &lt;tt&gt;CollectionShardingRuntime::CSRLock&lt;/tt&gt; comes in (which also implies the lock does not participate in 2PL)&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;I&apos;m speculating quite a bit at this point; I really know nothing about these structures. If the CSRLock is for protecting state on the &lt;tt&gt;MigrationSourceManager&lt;/tt&gt;, perhaps that state can versioned such that a lock is not required (or it can be turned into a mutex)? If the version meaningfully changed since the &lt;tt&gt;LogTransactionOperationsForShardingHandler&lt;/tt&gt; was constructed, that would presumably end up in this &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1d5d6a2fa25ebf81e96052c32425816f6d64d2e1/src/mongo/db/s/migration_chunk_cloner_source_legacy.cpp#L171-L173&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;no-op path&lt;/a&gt;?&lt;/p&gt;

&lt;p&gt;Just brainstorming here, but an API change that&apos;s theoretically possible; we could add a &quot;&lt;tt&gt;RecoveryUnit::preCommitHook&lt;/tt&gt;&quot; that does take in an OperationContext and if any of those hooks fail, the storage transaction can safely abort. Would this in essence solve the problem? We&apos;d obviously have to think whether that&apos;s something we&apos;d actually want to add.&lt;/p&gt;</comment>
                            <comment id="2408889" author="alex.taskov" created="Thu, 5 Sep 2019 14:32:41 +0000"  >&lt;p&gt;The code that brought up this question is here:&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_chunk_cloner_source_legacy.cpp#L167&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_chunk_cloner_source_legacy.cpp#L167&lt;/a&gt;&#160;&lt;/p&gt;

&lt;p&gt;Basically, we&apos;re in the commit method of the Change handler that was created &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/op_observer_sharding_impl.cpp#L173&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. The opCtx that was valid during the creation is not the same one in effect when commit is called so we can&apos;t cache it then. We are currently just pulling the opCtx from the current client but we&apos;re wondering if there is a better way to do this.&lt;/p&gt;
</comment>
                            <comment id="2408804" author="daniel.gottlieb@10gen.com" created="Thu, 5 Sep 2019 13:59:51 +0000"  >&lt;p&gt;Poke &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alex.taskov&quot; class=&quot;user-hover&quot; rel=&quot;alex.taskov&quot;&gt;alex.taskov&lt;/a&gt; if you have any details worth sharing.&lt;/p&gt;</comment>
                            <comment id="2401677" author="daniel.gottlieb@10gen.com" created="Fri, 30 Aug 2019 14:54:57 +0000"  >&lt;p&gt;I agree that having an OperationContext would be useful, but it&apos;s a bit dangerous to expose. onCommit handlers must not fail and the main uses of an OperationContext (from my perspective) is to acquire locks or to write to storage. Both of which are expected to fail. Do you mind sharing the intended use-case? &lt;/p&gt;</comment>
                            <comment id="2400896" author="kaloian.manassiev" created="Fri, 30 Aug 2019 07:54:53 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;, I suggested that Alex files this ticket, but didn&apos;t mean it to be a question - rather for the storage team to provide a way for accessing the committing OpContext from within the commit/abort methods of the RecoveryUnit.&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;Since the OperationContext outlives its RecoveryUnit, it should always be valid to capture the OperationContext pointer in the Change lambda.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;With the introduction of transactions, the recovery unit can now actually span multiple operation contexts and because of this, it is no longer safe to store a pointer to the parent OpContext. This is why we brought up this question.&lt;/p&gt;</comment>
                            <comment id="2400464" author="louis.williams" created="Thu, 29 Aug 2019 20:29:04 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alex.taskov&quot; class=&quot;user-hover&quot; rel=&quot;alex.taskov&quot;&gt;alex.taskov&lt;/a&gt;, the RecoveryUnit::Change handlers are executed by the RecoveryUnit &lt;a href=&quot;https://github.com/mongodb/mongo/blob/eb47871d19c81d08827f83df0554ce2c83cd8487/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L196&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;when a WriteUnitOfWork is committed&lt;/a&gt;. Additionally, an OperationContext &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c27c4a843018761c32013810e39fd3838aa229b8/src/mongo/db/operation_context.h#L462&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;owns a RecoveryUnit&lt;/a&gt;, so it should always outlive the RecoveryUnit.&lt;/p&gt;

&lt;p&gt;Since the OperationContext outlives its RecoveryUnit, it should always be valid to capture the OperationContext pointer in the Change lambda. See &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c27c4a843018761c32013810e39fd3838aa229b8/src/mongo/db/transaction_participant.cpp#L2085&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this example&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The SERVER project is only for reporting bugs or feature requests. Do you have a link to code or a specific bug where you are experiencing this problem?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="2098522">SERVER-68265</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 29 Aug 2019 20:29:04 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 20 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 20 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alex.taskov@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>henrik.edin@mongodb.com</customfieldvalue>
            <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvnkyn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hvcce7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6330">Execution Team 2022-09-19</customfieldvalue>
    <customfieldvalue id="6331">Execution Team 2022-10-03</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hvn77z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>