<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:47:35 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-81850] Use a more aggressive IDHACK for find/update/remove by _id</title>
                <link>https://jira.mongodb.org/browse/SERVER-81850</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Right now we have IDHACK as a stage, but it still involves going through the planner and working with a &lt;tt&gt;PlanExecutor&lt;/tt&gt;, &lt;tt&gt;WorkingSet&lt;/tt&gt; and similar. There is at least a 10% win from making the optimization more aggressive and cutting over to dedicated C++ code that directly uses the &lt;tt&gt;SortedDataInterface&lt;/tt&gt; and &lt;tt&gt;RecordStore&lt;/tt&gt; APIs, and calling that rather than directly. You can see my POC patch for find in PERF-4696, but I don&apos;t think that is exactly the right path to take. I think it is worth making functions for find/update/remove by _id and using them throughout the codebase when those operations are needed. &lt;/p&gt;

&lt;p&gt;I also prototyped a similar change for update in the update command logic. It didn&apos;t show a measurable impact on sys-perf because that is running with &lt;tt&gt;w:majority, j:1&lt;/tt&gt; which adds a ton of extra overhead and noise (some of which I&apos;m filing other tickets about). However local testing with &lt;tt&gt;w:1, j:0&lt;/tt&gt; shows that it also experiences a 10% end-to-end improvement from using dedicated code for update by _id. The improvement should be even larger for internal update-by-id codepaths such as &lt;a href=&quot;https://github.com/mongodb/mongo/blob/42b0d920a6ffe5a7eeea5c53e3fa145d9468f422/src/mongo/db/repl/oplog.cpp#L1795-L2014&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;oplog application&lt;/a&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/42b0d920a6ffe5a7eeea5c53e3fa145d9468f422/src/mongo/db/repl/storage_interface_impl.cpp#L1055&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;StorageInterface::upsertById&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I did not prototype remove, but I assume it would benefit from a similar treatment.&lt;/p&gt;

&lt;p&gt;A productionized version would have a few enhancements:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Improving the detection of &quot;simple &lt;tt&gt;_id&lt;/tt&gt; query&quot; to include things like {&lt;tt&gt;_id: {$eq: 7&lt;/tt&gt;}}&lt;/li&gt;
	&lt;li&gt;Splitting {&lt;tt&gt;_id: 1, a: 2, b: 3&lt;/tt&gt;} into separate matches on &lt;tt&gt;_id&lt;/tt&gt; (which will be handled by the index) and everything else and apply the remaining filter (if any) to the resulting doc before deciding to return it.
	&lt;ul&gt;
		&lt;li&gt;I believe this is necessary in order to use the fast path for &lt;b&gt;all&lt;/b&gt; oplog updates. We may be able to do &lt;em&gt;most&lt;/em&gt; updates without this enhancement, but it will need to fallback to the slow path if there is a residual query.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;This should also work for clustered collections where it can skip the _id index and just go directly to the record store.&lt;/li&gt;
	&lt;li&gt;In an ideal world, the &lt;tt&gt;RecordStore::Cursor&lt;/tt&gt; type (or a subclass) would be modified to support update operations rather than having them as methods on &lt;tt&gt;RecordStore&lt;/tt&gt; itself since that better matches the WiredTiger APIs. We could then use a single WT/RecordStore cursor both to fetch the document and to apply the update, which will save some lookup cost.
	&lt;ul&gt;
		&lt;li&gt;This should probably also happen in the &lt;tt&gt;UpdateStage&lt;/tt&gt; logic, but it may be more complicated because we would need to remove calls to &lt;tt&gt;WorkingSet::fetch&lt;/tt&gt; that naturally won&apos;t exist in the dedicated IDHACK code since it will just work directly with RecordIds.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="2459061">SERVER-81850</key>
            <summary>Use a more aggressive IDHACK for find/update/remove by _id</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="3" iconUrl="https://jira.mongodb.org/images/icons/statuses/inprogress.png" description="This issue is being actively worked on at the moment by the assignee.">In Progress</status>
                    <statusCategory id="4" key="indeterminate" colorName="inprogress"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="colin.stolley@mongodb.com">Colin Stolley</assignee>
                                    <reporter username="mathias@mongodb.com">Mathias Stearn</reporter>
                        <labels>
                            <label>perf-8.0</label>
                            <label>perf-tiger</label>
                            <label>perf-tiger-handoff</label>
                            <label>perf-tiger-poc</label>
                            <label>perf-tiger-q4</label>
                            <label>query-director-triage</label>
                            <label>query-perf-q4</label>
                            <label>risk</label>
                    </labels>
                <created>Wed, 4 Oct 2023 13:32:01 +0000</created>
                <updated>Fri, 2 Feb 2024 23:19:58 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>19</watches>
                                                                                                                                                            <comments>
                            <comment id="5764328" author="JIRAUSER1265607" created="Mon, 9 Oct 2023 14:46:52 +0000"  >&lt;p&gt;FYI &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kyle.suarez%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;kyle.suarez@mongodb.com&quot;&gt;kyle.suarez@mongodb.com&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=bernard.gorman%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;bernard.gorman@mongodb.com&quot;&gt;bernard.gorman@mongodb.com&lt;/a&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="5751843" author="redbeard0531" created="Wed, 4 Oct 2023 13:32:43 +0000"  >&lt;p&gt;Let me know if you want me to split this into separate tickets for find/update/remove.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2492619">SERVER-82865</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                            <subtask id="2513310">SERVER-83758</subtask>
                            <subtask id="2513312">SERVER-83759</subtask>
                            <subtask id="2513313">SERVER-83760</subtask>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 9 Oct 2023 14:46:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        17 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3386</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>brenda.rodriguez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            17 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>colin.stolley@mongodb.com</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
            <customfieldvalue>xiaochen.wu@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2tfjr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2bcjg:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7259">QE 2023-11-27</customfieldvalue>
    <customfieldvalue id="7261">QE 2023-12-11</customfieldvalue>
    <customfieldvalue id="7264">QE 2023-12-25</customfieldvalue>
    <customfieldvalue id="8043">QE 2024-01-08</customfieldvalue>
    <customfieldvalue id="8060">QE 2024-01-22</customfieldvalue>
    <customfieldvalue id="8061">QE 2024-02-05</customfieldvalue>
    <customfieldvalue id="8062">QE 2024-02-19</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2t1p3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>