<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:56:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-84972] Investigate if pipeline-style update performance got better for IncFewLargeDocLongFields</title>
                <link>https://jira.mongodb.org/browse/SERVER-84972</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;See some comments and previous description for context, we&apos;re wondering if the new strategy for logging these updates will help performance on this workload.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;The focus for this ticket will be profiling of the&#160;IncFewLargeDocLongFields&#160;perf test. There was a 2x-5x slowdown for this test as compared to the baseline across all variants.&lt;/p&gt;</description>
                <environment></environment>
        <key id="791806">SERVER-84972</key>
            <summary>Investigate if pipeline-style update performance got better for IncFewLargeDocLongFields</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="ruoxin.xu@mongodb.com">Ruoxin Xu</assignee>
                                    <reporter username="james.wahlin@mongodb.com">James Wahlin</reporter>
                        <labels>
                            <label>qexec-team</label>
                    </labels>
                <created>Thu, 6 Jun 2019 18:02:22 +0000</created>
                <updated>Fri, 12 Jan 2024 02:58:21 +0000</updated>
                            <resolved>Thu, 17 Sep 2020 10:20:53 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="3397744" author="ian.boros" created="Wed, 16 Sep 2020 19:42:24 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ruoxin.xu&quot; class=&quot;user-hover&quot; rel=&quot;ruoxin.xu&quot;&gt;ruoxin.xu&lt;/a&gt;&#160;I&apos;d say you&apos;re good to close this.&lt;/p&gt;</comment>
                            <comment id="3397613" author="ruoxin.xu" created="Wed, 16 Sep 2020 18:46:00 +0000"  >&lt;p&gt;This microbenchmark was running an incorrect query. It&apos;s now been modified and renamed to &apos;IncrementFewKeysLargeDocLongFields&apos; after PERF-2028. Generating $v:2 oplog entries didn&apos;t seem to have obvious regression on the new microbenchmark. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ian.boros&quot; class=&quot;user-hover&quot; rel=&quot;ian.boros&quot;&gt;ian.boros&lt;/a&gt; Do you think we need other further investigation on this ticket?&lt;/p&gt;</comment>
                            <comment id="3375795" author="ian.boros" created="Thu, 3 Sep 2020 22:41:59 +0000"  >&lt;p&gt;This workload has a bug caused by a typo in the update pipeline. Instead of incrementing fields, the update is making the document one level deeper each time. Eventually, the updates start failing with a &quot;depth exceeded&quot; error and taking a slow uasserted()/exception code path. I don&apos;t think we should read too much into changes in performance until this problem is fixed. See&#160;PERF-2028.&lt;/p&gt;</comment>
                            <comment id="2644550" author="charlie.swanson" created="Wed, 18 Dec 2019 22:43:57 +0000"  >&lt;p&gt;Re-titled this ticket to better reflect current reality after discussing with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=ian.boros&quot; class=&quot;user-hover&quot; rel=&quot;ian.boros&quot;&gt;ian.boros&lt;/a&gt;. Throwing this back on the backlog for now.&lt;/p&gt;</comment>
                            <comment id="2497253" author="james.wahlin@10gen.com" created="Wed, 23 Oct 2019 12:49:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt; - it is definitely possible that &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41114&quot; title=&quot;Replace mutablebson with Document/Value in UpdateStage and UpdateDriver&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41114&quot;&gt;SERVER-41114&lt;/a&gt; would improve performance and is worth a try. &lt;/p&gt;

&lt;p&gt;This ticket was filed to address to explore slowness in pipeline-based updates when compared to non-pipeline-based updates. The IncFewLargeDocLongFields microbenchmark was implemented for both and is an example of a scenario that pipeline-based updates will not be performant on since we piggy-back on the replacement update mechanism. This is not actually a regression since we have not replaced the &lt;tt&gt;$inc&lt;/tt&gt; update operator with pipeline updates. I will retitle this ticket to reflect.&lt;/p&gt;</comment>
                            <comment id="2496407" author="david.storch" created="Tue, 22 Oct 2019 20:52:47 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=james.wahlin&quot; class=&quot;user-hover&quot; rel=&quot;james.wahlin&quot;&gt;james.wahlin&lt;/a&gt;, is it possible that &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41114&quot; title=&quot;Replace mutablebson with Document/Value in UpdateStage and UpdateDriver&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41114&quot;&gt;SERVER-41114&lt;/a&gt; would improve performance? If that&apos;s at least plausible, then we could try running a perf patch build with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=justin.seyster&quot; class=&quot;user-hover&quot; rel=&quot;justin.seyster&quot;&gt;justin.seyster&lt;/a&gt;&apos;s draft changes for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-41114&quot; title=&quot;Replace mutablebson with Document/Value in UpdateStage and UpdateDriver&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-41114&quot;&gt;SERVER-41114&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Can you also clarify whether there was a performance regression? Or were you making an observation about how pipeline-based updates are slow compared to the baseline set by regular old-fashioned non-pipeline-based updates?&lt;/p&gt;</comment>
                            <comment id="2492321" author="james.wahlin@10gen.com" created="Mon, 21 Oct 2019 12:36:02 +0000"  >&lt;p&gt;I profiled this and found a significant amount of time going through a&#160;BSON -&amp;gt; Document/Value -&amp;gt; mutablebson -&amp;gt; BSON transformation for pipeline updates. I was curious whether the work being done for Document/Value at the time help make this faster.&lt;/p&gt;</comment>
                            <comment id="2490932" author="david.storch" created="Sat, 19 Oct 2019 00:11:51 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=james.wahlin&quot; class=&quot;user-hover&quot; rel=&quot;james.wahlin&quot;&gt;james.wahlin&lt;/a&gt;, I&apos;m not aware of Martin ever looking into this. Can you elaborate on your understanding of the source of the slowness?&lt;/p&gt;</comment>
                            <comment id="2489901" author="james.wahlin@10gen.com" created="Fri, 18 Oct 2019 15:15:12 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=david.storch&quot; class=&quot;user-hover&quot; rel=&quot;david.storch&quot;&gt;david.storch&lt;/a&gt; - if &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=martin.neupauer&quot; class=&quot;user-hover&quot; rel=&quot;martin.neupauer&quot;&gt;martin.neupauer&lt;/a&gt; did not find anything Document/Value related that would improve document transformation performance (outside of longer term CQF work) then I think we can close this ticket. I investigated earlier and did not find any quick wins.&lt;/p&gt;</comment>
                            <comment id="2489888" author="david.storch" created="Fri, 18 Oct 2019 15:08:44 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=james.wahlin&quot; class=&quot;user-hover&quot; rel=&quot;james.wahlin&quot;&gt;james.wahlin&lt;/a&gt; do you have time to investigate during this sprint?&lt;/p&gt;</comment>
                            <comment id="2309666" author="james.wahlin@10gen.com" created="Tue, 2 Jul 2019 15:31:19 +0000"  >&lt;p&gt;We plan to address the slowness caused by document transformation between mutablebson, BSONObj and Document/Value as part of the Common Query Framework roadmap. The only step remaining here is to confirm whether the Document/Value project can provide any nearer-term improvements. &lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=martin.neupauer&quot; class=&quot;user-hover&quot; rel=&quot;martin.neupauer&quot;&gt;martin.neupauer&lt;/a&gt; - I am assigning this ticket to you to wrap up as I will be OOO next week. If there are no quick-wins to be had via Document/Value then feel free to close this ticket.&lt;/p&gt;</comment>
                            <comment id="2296844" author="james.wahlin@10gen.com" created="Tue, 25 Jun 2019 16:34:06 +0000"  >&lt;p&gt;Reviewing the perf data confirms that the majority of time spent is in creating and destroying document elements. Pipeline updates pay a substantial cost in that they go through a BSON -&amp;gt; Document/Value -&amp;gt; mutablebson -&amp;gt; BSON transformation. &lt;/p&gt;

&lt;p&gt;I suspect that the way forward here will be to replace mutablebson with Document/Value across the update system. As part of this we can consider replacing use of the ObjectReplaceExecutor with a mechanism specific for pipeline update.&lt;/p&gt;</comment>
                            <comment id="2294709" author="james.wahlin@10gen.com" created="Mon, 24 Jun 2019 14:31:32 +0000"  >&lt;p&gt;Initial profiling efforts show that CPU time is split across transforming  Document to BSONObj, performing the pipeline transformation and applying the replacement style update to the provided mutablebson document. Applying the changes from the Document/Value project did not improve performance.&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;221676_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/221676/221676_Screen+Shot+2019-06-17+at+2.20.56+PM.png&quot; title=&quot;Screen Shot 2019-06-17 at 2.20.56 PM.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;221676&quot; file-preview-title=&quot;Screen Shot 2019-06-17 at 2.20.56 PM.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/221676/_thumb_221676.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt; &lt;/p&gt;

&lt;p&gt;Next steps are:&lt;br/&gt;
1) Martin is going to take a look to see if the Document/Value project can improve performance for this use case.&lt;br/&gt;
2) I am going to dig deeper into where time is being spent to see how we could improve.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="221676" name="Screen Shot 2019-06-17 at 2.20.56 PM.png" size="154925" author="james.wahlin@mongodb.com" created="Mon, 24 Jun 2019 14:27:01 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>13.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 13 Jun 2019 16:33:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 21 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1460</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>kelly.lewis@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 21 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>charlie.swanson@mongodb.com</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>ian.boros@mongodb.com</customfieldvalue>
            <customfieldvalue>james.wahlin@mongodb.com</customfieldvalue>
            <customfieldvalue>ruoxin.xu@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hv3gnr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hxg6in:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2844">Query 2019-06-17</customfieldvalue>
    <customfieldvalue id="2845">Query 2019-07-01</customfieldvalue>
    <customfieldvalue id="2881">Query 2019-07-29</customfieldvalue>
    <customfieldvalue id="2882">Query 2019-08-12</customfieldvalue>
    <customfieldvalue id="2883">Query 2019-08-26</customfieldvalue>
    <customfieldvalue id="2884">Query 2019-09-09</customfieldvalue>
    <customfieldvalue id="2886">Query 2019-10-07</customfieldvalue>
    <customfieldvalue id="3287">Query 2019-12-16</customfieldvalue>
    <customfieldvalue id="3615">Query 2020-09-21</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hv32x3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>