<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:56:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-41048] Address regression in transaction performance with storage stats collection</title>
                <link>https://jira.mongodb.org/browse/SERVER-41048</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;There is appx 15% regression in transaction performance due to the collection of storage statistics after each individual operation under the transaction. This ticket tracks work the work required to resolve this issue.&lt;/p&gt;</description>
                <environment></environment>
        <key id="760427">SERVER-41048</key>
            <summary>Address regression in transaction performance with storage stats collection</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="sulabh.mahajan@mongodb.com">Sulabh Mahajan</reporter>
                        <labels>
                    </labels>
                <created>Wed, 8 May 2019 07:51:58 +0000</created>
                <updated>Tue, 6 Dec 2022 02:59:58 +0000</updated>
                            <resolved>Wed, 18 Aug 2021 22:18:49 +0000</resolved>
                                                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="4008156" author="geert.bosch" created="Wed, 18 Aug 2021 22:18:50 +0000"  >&lt;p&gt;Closing this, as we&apos;d address this in &lt;a href=&quot;https://jira.mongodb.org/browse/WT-4813&quot; title=&quot;Enable cursor caching for statistics cursors&quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-4813&quot;&gt;WT-4813&lt;/a&gt; if at all.&lt;/p&gt;</comment>
                            <comment id="3751114" author="alexander.gorrod" created="Mon, 3 May 2021 00:38:20 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt; the Storage Engines knowledge about this regression has timed out now, I think. I&apos;m going to assign this to the execution backlog, and let you triage it again.&lt;/p&gt;

&lt;p&gt;Gathering statistics has an overhead - so I&apos;m inclined to say this is an expected consequence of adding more functionality for users but didn&apos;t want to make that choice for you.&lt;/p&gt;</comment>
                            <comment id="2257472" author="sulabh.mahajan" created="Thu, 23 May 2019 07:50:03 +0000"  >&lt;p&gt;Attached test files. The way I was running test:&lt;/p&gt;

&lt;p&gt;1. setup 1-node or 3-node replset using &lt;tt&gt;mongod.conf&lt;/tt&gt; attached. &lt;tt&gt;restart_mongo.sh&lt;/tt&gt; will restart mongod(s) and put them in a replset&lt;br/&gt;
2. run the python script to execute test 10 times and get an average of those runtimes.&lt;/p&gt;</comment>
                            <comment id="2257443" author="sulabh.mahajan" created="Thu, 23 May 2019 07:33:23 +0000"  >&lt;p&gt;I reran test several times with different portions of the code disabled.&lt;br/&gt;
I confirm that most of the regression comes from the opening and closing of the statistics cursor itself.&lt;/p&gt;

&lt;p&gt;The linked ticket is to enable cursor caching for statistics cursor. This should help mitigate tracked regression.&lt;/p&gt;</comment>
                            <comment id="2255467" author="alexander.gorrod" created="Wed, 22 May 2019 08:31:33 +0000"  >&lt;p&gt;Thanks for the results and write up Sulabh. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=brian.lane&quot; class=&quot;user-hover&quot; rel=&quot;brian.lane&quot;&gt;brian.lane&lt;/a&gt; could we discuss how to deal with the performance overhead here?&lt;/p&gt;</comment>
                            <comment id="2255446" author="sulabh.mahajan" created="Wed, 22 May 2019 07:51:11 +0000"  >&lt;p&gt;I ran tests again, N updates per transaction, 10 transactions per thread and 5 threads in total.&lt;br/&gt;
(Run times in milliseconds averaged over 10 runs):&lt;/p&gt;

&lt;div class=&apos;table-wrap&apos;&gt;
&lt;table class=&apos;confluenceTable&apos;&gt;&lt;tbody&gt;
&lt;tr&gt;
&lt;th class=&apos;confluenceTh&apos;&gt;replset-config&lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt;read/write concern&lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt; N &lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt; runtime-with-stats&lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt;runtime-without-stats&lt;/th&gt;
&lt;th class=&apos;confluenceTh&apos;&gt;regression %&lt;/th&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;default&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3000&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;~18000&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;~16000&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;12.5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;default&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;300&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1960&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1710&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;14.7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;snapshot,majority&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;300&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1986&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1900&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;4.3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;default&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3000&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;18980&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;17711&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;7.1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;snapshot,majority&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3000&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;19222&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;18182&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;5.7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;default&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;300&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;2003&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;1908&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;5&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;3-node&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;snapshot,majority&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;300&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;2068&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;2019&lt;/td&gt;
&lt;td class=&apos;confluenceTd&apos;&gt;2.4&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;/div&gt;


&lt;blockquote&gt;
&lt;p&gt;I see in your script that you configure --setParameter wiredTigerCursorCacheSize=0 - does that make a difference to the performance difference?&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;Sorry for the confusion, it is commented out, remnants of what I based my script on.&lt;/p&gt;</comment>
                            <comment id="2238393" author="alexander.gorrod" created="Wed, 8 May 2019 13:22:10 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sulabh.mahajan&quot; class=&quot;user-hover&quot; rel=&quot;sulabh.mahajan&quot;&gt;sulabh.mahajan&lt;/a&gt; I&apos;d like to understand more about this performance cost:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;Is there a regression if running in a replica set configuration?&lt;/li&gt;
	&lt;li&gt;What is the scale of the regression if the transaction does less updates? i.e: if you do 300 updates is there still a noticeable change in performance?&lt;/li&gt;
	&lt;li&gt;I see in your script that you configure &lt;tt&gt;--setParameter wiredTigerCursorCacheSize=0&lt;/tt&gt; - does that make a difference to the performance difference?&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="2238231" author="sulabh.mahajan" created="Wed, 8 May 2019 07:59:21 +0000"  >&lt;p&gt;Attached script updates 3000 documents in each transaction, 10 transactions per thread, 5 threads simultaneously and reports the time taken.&lt;br/&gt;
I am roughly getting the following runtime:&lt;/p&gt;

&lt;p&gt;Without stats collection: 16 seconds&lt;br/&gt;
With stats collection: 18 seconds&lt;/p&gt;

&lt;p&gt;That amounts to a regression of 12.5%.&lt;/p&gt;

&lt;p&gt;I further investigated and looks like most of the regression comes from opening a &lt;tt&gt;session:statistics&lt;/tt&gt; cursor and iterating through its contents.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="775057">WT-4813</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="217793" name="mongod.conf" size="529" author="sulabh.mahajan@mongodb.com" created="Thu, 23 May 2019 07:47:53 +0000"/>
                            <attachment id="217794" name="restart_mongo.sh" size="521" author="sulabh.mahajan@mongodb.com" created="Thu, 23 May 2019 07:47:53 +0000"/>
                            <attachment id="217795" name="runtest.py" size="1868" author="sulabh.mahajan@mongodb.com" created="Thu, 23 May 2019 07:47:53 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 8 May 2019 13:22:10 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 25 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 25 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alexander.gorrod@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>geert.bosch@mongodb.com</customfieldvalue>
            <customfieldvalue>sulabh.mahajan@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huy8xr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr2r2f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huxv73:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>