<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:10:12 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-5905] Add data collection and command to get histogram of query response times</title>
                <link>https://jira.mongodb.org/browse/SERVER-5905</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We should have instrumentation to characterize the overall workload and response time of a mongod or mongos server.  A histogram with buckets with log base 2 microsecond resolution  would be a nice start.  Here&apos;s a straw man proposal:&lt;/p&gt;

&lt;p&gt;1)  For every request from a client, log the time it was received using the least expensive high resolution method.  On Windows, this would be QueryPerformanceCounter().&lt;br/&gt;
2)  When the response is complete, compute the elapsed time in microseconds.  On Windows, this would be another call to QueryPerformanceCounter() and division by a precomputed conversion factor.&lt;br/&gt;
3)  Add 1 to the bucket associated with this time interval.  Bucket 0 gets all times below 1 microsecond, bucket 1 gets times above 1 microsecond but below 2 microseconds, bucket 2 gets times from 2 to 4 microseconds, then 4 to 8, etc.  31 buckets would cover times up to 2147 seconds and anything taking longer than 2147 seconds would go in the last bucket, so 32 buckets would cover the time periods we are most interested in.&lt;br/&gt;
4)  Every 10 seconds, add the histogram to a &quot;since started&quot; histogram, write it to a capped collection sized for one week of data, save a snapshot copy and then zero it.&lt;br/&gt;
5)  Provide $cmd commands to fetch the most recent snapshot and the &quot;since started&quot; histogram.&lt;br/&gt;
6)  Give MMS the ability to show the most recent snapshot and the &quot;since started&quot; snapshot.&lt;br/&gt;
7)  For extra credit, MMS could show a contour plot or some other 3D display of response time history, showing the changing shape of the curve.&lt;/p&gt;

&lt;p&gt;Once the baseline functionality is working, we could consider doing this by database, by collection, by request type or by some other criterion.  These would be additional instances of the same feature.&lt;/p&gt;

&lt;p&gt;There are a lot of things that we could learn by having this information:&lt;br/&gt;
1)  If a query was slow at one time but not at another, was there a difference in the number of requests it was competing with in the two cases?&lt;br/&gt;
2)  Is a workload doing mostly very fast stuff with a little slow stuff, or is everything slow?&lt;br/&gt;
3)  Does a change to something in the system change the mix of response times?&lt;br/&gt;
4)  Do response times follow a recognizable pattern, like a bell curve with a visible center, or a skew towards fast responses, or a curve with multiple peaks?&lt;br/&gt;
5)  Is anything really fast, or is the minimum response time in the millisecond and above range?&lt;br/&gt;
6)  Do we have periods with little visible activity followed by periods when many slow requests complete?&lt;br/&gt;
7)  Does the addition of a new application, or a new shard, or a new mongos change the response time pattern?&lt;/p&gt;

&lt;p&gt;The better we can characterize workloads and our response to them, the better we can diagnose problems and propose solutions.  All to the good.&lt;/p&gt;</description>
                <environment></environment>
        <key id="39410">SERVER-5905</key>
            <summary>Add data collection and command to get histogram of query response times</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kevin.albertson@mongodb.com">Kevin Albertson</assignee>
                                    <reporter username="tad">Tad Marshall</reporter>
                        <labels>
                            <label>neweng</label>
                            <label>performance</label>
                    </labels>
                <created>Wed, 23 May 2012 03:08:16 +0000</created>
                <updated>Wed, 22 Mar 2017 15:27:48 +0000</updated>
                            <resolved>Fri, 24 Jun 2016 21:53:59 +0000</resolved>
                                                    <fixVersion>3.3.9</fixVersion>
                                    <component>Diagnostics</component>
                    <component>Performance</component>
                                        <votes>5</votes>
                                    <watches>16</watches>
                                                                                                                <comments>
                            <comment id="1306381" author="xgen-internal-githook" created="Fri, 24 Jun 2016 21:51:26 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;kevinAlbs&apos;, u&apos;name&apos;: u&apos;Kevin Albertson&apos;, u&apos;email&apos;: u&apos;kevin.albertson@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5905&quot; title=&quot;Add data collection and command to get histogram of query response times&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-5905&quot;&gt;&lt;del&gt;SERVER-5905&lt;/del&gt;&lt;/a&gt; Add collStats aggregation stage&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6f0af04446b6dcd682ca844757f023f7f5c900cc&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6f0af04446b6dcd682ca844757f023f7f5c900cc&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1306380" author="xgen-internal-githook" created="Fri, 24 Jun 2016 21:51:25 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;kevinAlbs&apos;, u&apos;name&apos;: u&apos;Kevin Albertson&apos;, u&apos;email&apos;: u&apos;kevin.albertson@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5905&quot; title=&quot;Add data collection and command to get histogram of query response times&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-5905&quot;&gt;&lt;del&gt;SERVER-5905&lt;/del&gt;&lt;/a&gt; Add operation latency histogram&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6c755905c31ac284d88077500ebba021d20b3626&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6c755905c31ac284d88077500ebba021d20b3626&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1305883" author="xgen-internal-githook" created="Fri, 24 Jun 2016 17:00:49 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;RedBeard0531&apos;, u&apos;name&apos;: u&apos;Mathias Stearn&apos;, u&apos;email&apos;: u&apos;mathias@10gen.com&apos;}
&lt;p&gt;Message: Revert &quot;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5905&quot; title=&quot;Add data collection and command to get histogram of query response times&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-5905&quot;&gt;&lt;del&gt;SERVER-5905&lt;/del&gt;&lt;/a&gt; Add operation latency histogram&quot;&lt;/p&gt;

&lt;p&gt;This reverts commit c7794350b056cdea85e1c6185a7dda4579936179.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/7084de1f754ffaf94ead1a5e1bd8475e3a115b76&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/7084de1f754ffaf94ead1a5e1bd8475e3a115b76&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1305882" author="xgen-internal-githook" created="Fri, 24 Jun 2016 17:00:48 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;RedBeard0531&apos;, u&apos;name&apos;: u&apos;Mathias Stearn&apos;, u&apos;email&apos;: u&apos;mathias@10gen.com&apos;}
&lt;p&gt;Message: Revert &quot;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5905&quot; title=&quot;Add data collection and command to get histogram of query response times&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-5905&quot;&gt;&lt;del&gt;SERVER-5905&lt;/del&gt;&lt;/a&gt; Add collStats aggregation stage&quot;&lt;/p&gt;

&lt;p&gt;This reverts commit a22d2843ccab7f0333434d1124358c5c182427f6.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a5e8fb2029c4d96daa0d07f287bf07a68a3ec1fa&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a5e8fb2029c4d96daa0d07f287bf07a68a3ec1fa&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1304907" author="kevin.albertson" created="Thu, 23 Jun 2016 21:51:19 +0000"  >&lt;p&gt;We need to add a documentation page describing the operation latency histogram and its usage.&lt;/p&gt;</comment>
                            <comment id="1304902" author="xgen-internal-githook" created="Thu, 23 Jun 2016 21:48:24 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;kevinAlbs&apos;, u&apos;name&apos;: u&apos;Kevin Albertson&apos;, u&apos;email&apos;: u&apos;kevin.albertson@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5905&quot; title=&quot;Add data collection and command to get histogram of query response times&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-5905&quot;&gt;&lt;del&gt;SERVER-5905&lt;/del&gt;&lt;/a&gt; Add collStats aggregation stage&lt;/p&gt;

&lt;p&gt;Signed-off-by: Kyle Suarez &amp;lt;kyle.suarez@mongodb.com&amp;gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a22d2843ccab7f0333434d1124358c5c182427f6&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a22d2843ccab7f0333434d1124358c5c182427f6&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1304901" author="xgen-internal-githook" created="Thu, 23 Jun 2016 21:48:22 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;kevinAlbs&apos;, u&apos;name&apos;: u&apos;Kevin Albertson&apos;, u&apos;email&apos;: u&apos;kevin.albertson@10gen.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-5905&quot; title=&quot;Add data collection and command to get histogram of query response times&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-5905&quot;&gt;&lt;del&gt;SERVER-5905&lt;/del&gt;&lt;/a&gt; Add operation latency histogram&lt;/p&gt;

&lt;p&gt;Signed-off-by: Kyle Suarez &amp;lt;kyle.suarez@mongodb.com&amp;gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/c7794350b056cdea85e1c6185a7dda4579936179&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/c7794350b056cdea85e1c6185a7dda4579936179&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                                                <inwardlinks description="is documented by">
                                        <issuelink>
            <issuekey id="319553">DOCS-8914</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="296321">DOCS-8165</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="57630">SERVER-7774</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="38493">SERVER-5828</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 13 Jun 2016 20:26:50 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 33 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-503</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>emily.hall</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 33 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>kevin.albertson@mongodb.com</customfieldvalue>
            <customfieldvalue>tad</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hro273:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrcuiv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5070</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1097">Integrate+Tuning 16 (06/24/16)</customfieldvalue>
    <customfieldvalue id="1139">Integration 17 (07/15/16)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hriw73:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>