<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:19:40 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-29006] Query failed with error code 6 with YCSB workload</title>
                <link>https://jira.mongodb.org/browse/SERVER-29006</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hi ,&lt;/p&gt;

&lt;p&gt;I have the following setup  4 nodes, 1 client&lt;br/&gt;
Client  -&amp;gt;    node1 - node2 - node3 - node4&lt;br/&gt;
   node1 and node2 config server is running&lt;br/&gt;
   node&lt;span class=&quot;error&quot;&gt;&amp;#91;1-4&amp;#93;&lt;/span&gt;  are also acting as shard server , Sharding is enabled for YCSB database and for collection usertable.&lt;/p&gt;

&lt;p&gt;Mongos is running on client. &lt;/p&gt;

&lt;p&gt;YCSB: &lt;a href=&quot;https://github.com/brianfrankcooper/YCSB/tree/master/mongodb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/brianfrankcooper/YCSB/tree/master/mongodb&lt;/a&gt;&lt;br/&gt;
Workload: I insert 10M doc , it works fine, When i run workloade issue started to happen&lt;/p&gt;

&lt;p&gt;With Version 3.4 Enterprise it crashed. Issue looks similar &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-27701&quot; title=&quot;Race in AsyncResultsMerger error handling may trigger mongos invariant&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-27701&quot;&gt;&lt;del&gt;SERVER-27701&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;With version 3.5.6 Lot of queries throwing following  Error  &lt;br/&gt;
com.mongodb.MongoQueryException: Query failed with error code 6 and error message&lt;/p&gt;
</description>
                <environment></environment>
        <key id="378338">SERVER-29006</key>
            <summary>Query failed with error code 6 with YCSB workload</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13203">Gone away</resolution>
                                        <assignee username="mark.agarunov">Mark Agarunov</assignee>
                                    <reporter username="sannandi@indiana.edu">sandip</reporter>
                        <labels>
                    </labels>
                <created>Thu, 27 Apr 2017 20:34:58 +0000</created>
                <updated>Fri, 27 Oct 2023 20:44:06 +0000</updated>
                            <resolved>Fri, 23 Jun 2017 17:50:27 +0000</resolved>
                                                                    <component>Querying</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="1605454" author="mark.agarunov" created="Fri, 23 Jun 2017 17:50:05 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sannandi%40indiana.edu&quot; class=&quot;user-hover&quot; rel=&quot;sannandi@indiana.edu&quot;&gt;sannandi@indiana.edu&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for the information, I am glad this is now working for you. As this is no longer an issue, I&apos;ve closed this ticket.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1604735" author="sannandi@indiana.edu" created="Thu, 22 Jun 2017 22:22:00 +0000"  >&lt;p&gt;Hi ,&lt;/p&gt;

&lt;p&gt;I  figure it out and make it work. It was a problem related to multiple issues.&lt;/p&gt;

&lt;p&gt;1. The YCSB repo uses old mongo JDK, the implementation does some leak.  if I use latest one 3.4.2 it works good.&lt;br/&gt;
2. I need to set up ulimit -n 32000 and sysctl net.core.somaxconn = 4096 , the failures are due to system limit. &lt;/p&gt;

&lt;p&gt;Thanks,&lt;/p&gt;</comment>
                            <comment id="1604669" author="mark.agarunov" created="Thu, 22 Jun 2017 21:04:23 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sannandi%40indiana.edu&quot; class=&quot;user-hover&quot; rel=&quot;sannandi@indiana.edu&quot;&gt;sannandi@indiana.edu&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We still need additional information to diagnose the problem. If this is still an issue for you, would you please provide the complete log files from both mongod and mongos when this issue is present?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1593477" author="mark.agarunov" created="Fri, 9 Jun 2017 21:03:11 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sannandi%40indiana.edu&quot; class=&quot;user-hover&quot; rel=&quot;sannandi@indiana.edu&quot;&gt;sannandi@indiana.edu&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for providing the script. Looking into this further, the error code you are seeing, 6, is &quot;HostUnreachable&quot;. As this can be caused by a few different paths in the code, I would like to request the complete log files from both &lt;tt&gt;mongod&lt;/tt&gt; and &lt;tt&gt;mongos&lt;/tt&gt; when this issue is present so that we can get a better idea of what the issue may be. I&apos;ve generated a &lt;a href=&quot;https://10gen-httpsupload.s3.amazonaws.com/upload_forms/7e3b8502-e9c8-4b90-ac28-3e25c6e24b84.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;secure upload portal&lt;/a&gt; so that you can send us these files privately.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1585720" author="talha" created="Fri, 2 Jun 2017 08:27:00 +0000"  >&lt;p&gt;Hello&lt;/p&gt;

&lt;p&gt;i&apos;m facing same issue on my mongodb cluster. I have a collection in which sharding is not enabled. When i run single find query it seems ok. But during concurrent query it started showing exception &quot;com.mongodb.MongoQueryException: Query failed with error code 6 and error message &apos;Operation aborted.&apos; on server&quot; . And on &quot;Exit Codes and Statuses&quot;  of mongodb documentation there are nothing about error code 6&lt;/p&gt;</comment>
                            <comment id="1563048" author="sannandi@indiana.edu" created="Tue, 2 May 2017 23:25:25 +0000"  >&lt;p&gt;The configuration is here:&lt;br/&gt;
Client -&amp;gt; node1 - node2 - node3 - node4&lt;br/&gt;
node1 and node2 config server is running&lt;br/&gt;
node&lt;span class=&quot;error&quot;&gt;&amp;#91;1-4&amp;#93;&lt;/span&gt; are also acting as shard server , Sharding is enabled for YCSB database and for collection usertable.&lt;br/&gt;
Mongos is running on client.&lt;/p&gt;

&lt;p&gt;Only a Single instance of mongos running on client&lt;/p&gt;

&lt;p&gt;Commands used to run:&lt;br/&gt;
Attached the script &lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/155139/155139_mongodb.py.txt&quot; title=&quot;mongodb.py.txt attached to SERVER-29006&quot;&gt;mongodb.py.txt&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;What is the configuration you are using ?&lt;/p&gt;
</comment>
                            <comment id="1562961" author="mark.agarunov" created="Tue, 2 May 2017 21:25:26 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sannandi%40indiana.edu&quot; class=&quot;user-hover&quot; rel=&quot;sannandi@indiana.edu&quot;&gt;sannandi@indiana.edu&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I have not yet been able to reproduce the error that you&apos;ve described, and was hoping to get some more information. If possible, please provide the following:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;The complete output of YCSB&lt;/li&gt;
	&lt;li&gt;The complete logs from &lt;tt&gt;mongos&lt;/tt&gt;&lt;/li&gt;
	&lt;li&gt;The complete logs from all &lt;tt&gt;mongod&lt;/tt&gt; nodes&lt;/li&gt;
	&lt;li&gt;The configuration you are using for &lt;tt&gt;mongod&lt;/tt&gt; and &lt;tt&gt;mongos&lt;/tt&gt;&lt;/li&gt;
	&lt;li&gt;The exact command you are using to execute YCSB&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;The should help shed some light on what may be causing this behavior.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Mark&lt;/p&gt;</comment>
                            <comment id="1559786" author="jeff.yemin" created="Thu, 27 Apr 2017 22:20:47 +0000"  >&lt;p&gt;I don&apos;t think this is a networking issue between client and mongos.  Running against MongoDB &amp;gt;= 3.2, the driver will only throw a MongoQueryException in response to a non-ok response from the server to either the find or getMore command (in this case probably a find since all YCSB queries are for single documents).  The error code of 6 and the error message of &quot;&apos;End of file&apos; on server x.x.x.x:27017.&quot; is what&apos;s being returned from mongos in the command response, e.g.:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  ok: 0, &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  code : 6, &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  errmsg : &quot;&apos;End of file&apos; on server x.x.x.x:27017.&quot;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                            <comment id="1559777" author="dan@10gen.com" created="Thu, 27 Apr 2017 22:05:53 +0000"  >&lt;p&gt;that&apos;s the error message given by the java driver back to the client.  It looks like a networking issue between the client and the mongos.&lt;/p&gt;</comment>
                            <comment id="1559770" author="sannandi@indiana.edu" created="Thu, 27 Apr 2017 21:58:33 +0000"  >&lt;p&gt;Only error message i can see is  &lt;br/&gt;
com.mongodb.MongoQueryException: Query failed with error code 6 and error message &apos;End of file&apos; on server x.x.x.x:27017.&lt;/p&gt;

&lt;p&gt;All the clusters are up while the test is on.  Is there anything wrong with the setup ? As I have mentioned if i use only write query it works fine.&lt;/p&gt;</comment>
                            <comment id="1559754" author="dan@10gen.com" created="Thu, 27 Apr 2017 21:50:15 +0000"  >&lt;p&gt;What is the error on the mongos?  &lt;br/&gt;
Are all the nodes in the cluster up and running at the time of the error message?&lt;/p&gt;</comment>
                            <comment id="1559668" author="sannandi@indiana.edu" created="Thu, 27 Apr 2017 20:38:42 +0000"  >&lt;p&gt;The YCSB workload e is 95% scan and 5% insert. If I change the workload to 100% insert it works fine,  When I do scan operation it is throwing error.  The earlier version was crashing. With version 3.5.6 crash issue on mongos has fixed but exception starts to happen.&lt;/p&gt;

&lt;p&gt;I am using 4 instances of YCSB and each one doing 30 threads.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="155139" name="mongodb.py.txt" size="2648" author="sannandi@indiana.edu" created="Tue, 2 May 2017 23:24:10 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 27 Apr 2017 21:50:15 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 33 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 33 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>jeff.yemin@mongodb.com</customfieldvalue>
            <customfieldvalue>talha</customfieldvalue>
            <customfieldvalue>mark.agarunov</customfieldvalue>
            <customfieldvalue>sannandi@indiana.edu</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht6jf3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsyrwn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Mentioned in the description.&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs49pr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>