<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:47:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37782] Would like server stats in mongos on scatter-gather queries</title>
                <link>https://jira.mongodb.org/browse/SERVER-37782</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;While monitoring a sharded cluster it would be helpful to know how seriously we are being affected by scatter-gather queries - how many operations are being targeted to multiple shards, how many require sorting and merging results on a shardsvr, and so forth.&lt;/p&gt;

&lt;p&gt;While some of this data can be gathered by analyzing queries as they go in, it would be nice to be able to monitor a simple counter or two on grafana, CloudWatch, or Ops Manager to tell if some new code release or some other event has triggered a spate of scatter-gather queries that are negatively affecting performance.&lt;/p&gt;</description>
                <environment></environment>
        <key id="625690">SERVER-37782</key>
            <summary>Would like server stats in mongos on scatter-gather queries</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="3">Duplicate</resolution>
                                        <assignee username="backlog-server-query">Backlog - Query Team</assignee>
                                    <reporter username="mzraly">Mike Zraly</reporter>
                        <labels>
                            <label>SWDI</label>
                            <label>qopt-team</label>
                    </labels>
                <created>Fri, 26 Oct 2018 18:06:52 +0000</created>
                <updated>Tue, 6 Dec 2022 03:14:45 +0000</updated>
                            <resolved>Tue, 10 Dec 2019 16:18:33 +0000</resolved>
                                                                    <component>Diagnostics</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="2594012" author="asya" created="Mon, 9 Dec 2019 16:34:38 +0000"  >&lt;p&gt;This appears to be a duplicate of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-43617&quot; title=&quot;Add metrics on the mongos to indicate the number of shards targeted for the commands (find, aggregate, etc)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-43617&quot;&gt;&lt;del&gt;SERVER-43617&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2062149" author="mzraly" created="Wed, 14 Nov 2018 19:18:23 +0000"  >&lt;p&gt;I am looking for an external monitoring method for sharding overhead that does not require a lot of the monitor.&lt;/p&gt;

&lt;p&gt;Parsing log files is out of course, since that ignores queries that execute faster than the slow query threshold.&lt;/p&gt;

&lt;p&gt;Parsing db.currentOp() is better.&#160; But when I run db.currentOp() via mongos 3.4.17 I get output for each shard - at least for the primary replica set member.&#160; That&apos;s showing current operations from the perspective of the individual shards.&#160; Even with an nShards entry in the output I would have to work backwards to avoid over-counting, because a query addressing N shards can appear up to N times in the output.&#160; Besides, this still requires the monitor to parse a variable-length structure that gets larger precisely when we are most in need of timely reporting.&#160; And it only samples the queries being processed by mongos, it won&apos;t capture queries that execute and complete in between calls from the monitor.&lt;/p&gt;

&lt;p&gt;I do think that a few counters, one for queries processed, another for shards targeted, perhaps a third counting queries that require sorting, would provide a quickly processed unambiguous measure of sharding fan-out and overhead.&#160;&#160;Perhaps these counters could be located in a new subsection of the db.serverStatus() shardingStatistics object, as a peer to&#160;catalogCache.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2062009" author="asya" created="Wed, 14 Nov 2018 17:44:45 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=mzraly&quot; class=&quot;user-hover&quot; rel=&quot;mzraly&quot;&gt;mzraly&lt;/a&gt; there was a project for 4.0 which added slow query logging for mongos which might have what you are looking for.  It was &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-14900&quot; title=&quot;log slow queries on mongos&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-14900&quot;&gt;&lt;del&gt;SERVER-14900&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Here is an example of logging of query which was scatter gather (my cluster has two shards):&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;2018-11-14T12:32:05.723-0500 I COMMAND  [conn12] command logtest.coll appName: &quot;MongoDB Shell&quot; command: find { find: &quot;coll&quot;, filter: { _id: { $in: [ 3.0, 13.0, 23.0, 33.0 ] } }, $clusterTime: { clusterTime: Timestamp(1542216711, 1), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $db: &quot;logtest&quot; } nShards:2 cursorExhausted:1 numYields:0 nreturned:4 reslen:354 protocol:op_msg 2ms&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;You can see &lt;tt&gt;nShards:2&lt;/tt&gt; in the logs (as well as in profiler output and currentOp.  When the logged query is sent to one shard, it&apos;s logged with &lt;tt&gt;nShards:1&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;Would this be what you are looking for?  Technically it doesn&apos;t say if query uses shard key or not, just how many shards it was sent to.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="939410">SERVER-43617</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25143"><![CDATA[Query]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 9 Nov 2018 15:42:14 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 9 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 9 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-query</customfieldvalue>
            <customfieldvalue>mzraly</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hubhqn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr6ra7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hub3zz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>