<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Wed Feb 07 21:36:30 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[CSHARP-331] System.IO.IOException: Unable to read data from the transport connection</title>
                <link>https://jira.mongodb.org/browse/CSHARP-331</link>
                <project id="10041" key="CSHARP">C# Driver</project>
                    <description>&lt;p&gt;We multiple web crawlers, dumping data into single node of mongod running on windows 2008. All crawlers are .net 3.5 on windows 2008 machines as well.  I routinely get this error:&lt;/p&gt;

&lt;p&gt;System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.&lt;/p&gt;

&lt;p&gt;Initially when the db size was smaller, it happened less frequently, but now was the size of the db grows (currently only at ~13gb), it is happening at a rate of once every 3-4 seconds and it slows everything down. This happens even if I turn off ALL crawlers, and only leave 1 instance running.&lt;/p&gt;

&lt;p&gt;I should note that the crawler is multithreaded. The thread pool on the crawler is usually kept between 32-64 and alot of data are flushed into mongod in batches of 200-500 items at a time, every 10-30 seconds (depending on collections).&lt;/p&gt;

&lt;p&gt;Attached are the full mongod log file, along with 1 of the service error logs (so you can see the full errors).  I&apos;ve also attached the mongo connection wrapper that we are using and a few cs file  of interest so you can see how data are dumped into mongo.&lt;/p&gt;

&lt;p&gt;We also notice when used with driver v1.1, the error is a little less frequent than in v1.2.  For the driver, we are setting the time out at 10s for requests.&lt;/p&gt;</description>
                <environment>Mongod 2.0 on windows 2008 (single node). C#.NET 3.5 driver 1.1/1.2 on windows 2008 service</environment>
        <key id="22886">CSHARP-331</key>
            <summary>System.IO.IOException: Unable to read data from the transport connection</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="robert@mongodb.com">Robert Stam</assignee>
                                    <reporter username="huyn@docstoc.com">Huy Nguyen</reporter>
                        <labels>
                            <label>concurrency</label>
                            <label>connection</label>
                            <label>driver</label>
                            <label>performance</label>
                    </labels>
                <created>Tue, 27 Sep 2011 18:20:49 +0000</created>
                <updated>Thu, 2 Apr 2015 18:28:18 +0000</updated>
                            <resolved>Fri, 14 Oct 2011 17:38:05 +0000</resolved>
                                    <version>1.2</version>
                                    <fixVersion>1.3</fixVersion>
                                                        <votes>0</votes>
                                    <watches>0</watches>
                                                                                                                <comments>
                            <comment id="60519" author="rstam" created="Fri, 14 Oct 2011 17:37:54 +0000"  >&lt;p&gt;Glad to hear that everything is working great now! I will go ahead and close this ticket now. Feel free to follow up as needed.&lt;/p&gt;</comment>
                            <comment id="60518" author="huyn@docstoc.com" created="Fri, 14 Oct 2011 17:35:51 +0000"  >&lt;p&gt;Robert. Sorry for the late reply. I&apos;ve been swamp. I was able to go back to my code and figure out the issues base on your comments &amp;amp; suggestions. You were right, I had server.Connect(), ReConnect(), &amp;amp; Disconnect() calls in places. Taking these out completely eliminated the high connection count.&lt;/p&gt;

&lt;p&gt;Going back to the slow query log, I was able to identify the queries that required indexes. After doing these 2 things, we got stellar performance from Mongo 2.0 &amp;amp; the new 1.2 c# driver, which is great news.  We will scale this project up on Mongo &amp;amp; switch to the new driver on our other production systems. &lt;/p&gt;

&lt;p&gt;Thank you so much for your help.&lt;/p&gt;</comment>
                            <comment id="59224" author="rstam" created="Fri, 7 Oct 2011 22:18:52 +0000"  >&lt;p&gt;Any updates on this? I tried looking at your code but got lost in the multiple levels of nested lambdas (the lambdas also make the stack traces have weird names in them).&lt;/p&gt;

&lt;p&gt;The server log shows lots of operations that are taking multiple seconds. If you are really setting a 10 second timeout (I couldn&apos;t find where you were doing that) you may just be getting lots of timeouts because your timeout is too low.&lt;/p&gt;

&lt;p&gt;The other thing that looked odd in the server log was that the connection numbers were really high, which seems to indicate that the connections are being closed and reopened a lot. Are you calling Disconnect or Reconnect anywhere?&lt;/p&gt;

&lt;p&gt;Let me know if you have any more information.&lt;/p&gt;</comment>
                            <comment id="57575" author="rstam" created="Thu, 29 Sep 2011 14:47:00 +0000"  >&lt;p&gt;These exceptions are thrown by the C# driver when the server is not responding within the timeout period (default 30 seconds). So it&apos;s not really correct to classify this as a C# driver bug. Although if you routinely expect operations to take a long time you can configure a longer timeout period. But what you really need to do is find out why the server operations are slow. Often it&apos;s just a matter of adding the right index.&lt;/p&gt;</comment>
                            <comment id="57420" author="huyn@docstoc.com" created="Wed, 28 Sep 2011 22:42:24 +0000"  >&lt;p&gt;I also notice that most (if not all) of the error thrown are caused by FindAndModify or FindAndRemove methods.  Most of everything else are flushed to the DB using un-safe writes so there are no errors for those.  Even for the batch inserts or upserts with safemodes, they are responding ok.  &lt;/p&gt;

&lt;p&gt;I should note that FindAndModify or FindAndRemove methods sometimes takes upward of 400 seconds (as logged on the mongod server). However, it is not showing up very often when comparing to the amount of error being thrown.&lt;/p&gt;</comment>
                            <comment id="57327" author="huyn@docstoc.com" created="Wed, 28 Sep 2011 18:24:11 +0000"  >&lt;p&gt;Attached (Debug &amp;amp; Error Driver v1.2.zip) is a sample run log with version 1.2 driver since this version throws slightly different error than version 1.1. &lt;/p&gt;

&lt;p&gt;in 1.1, it throws typically IOException, in v1.2, it mainly throws TimeoutExceptions.&lt;/p&gt;

&lt;p&gt;In the zip is both the full debug log &amp;amp; the error only log.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="13319" name="App.config" size="10080" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                            <attachment id="13318" name="CONST.cs" size="4107" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                            <attachment id="13317" name="CrawlRequestDB.cs" size="14463" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                            <attachment id="13330" name="Debug &amp; Error Driver v1.2.zip" size="10579" author="huyn@docstoc.com" created="Wed, 28 Sep 2011 18:24:11 +0000"/>
                            <attachment id="13313" name="Error.zip" size="1167030" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                            <attachment id="13316" name="FlushableStore.cs" size="4053" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                            <attachment id="13315" name="MongoAccess.cs" size="18245" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                            <attachment id="13314" name="mongo.zip" size="2557192" author="huyn@docstoc.com" created="Tue, 27 Sep 2011 18:20:50 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10012"><![CDATA[Major Change]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrh8cn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14152</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>