<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 02:59:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-2377] Mongos sharding/failover behaves strangely</title>
                <link>https://jira.mongodb.org/browse/SERVER-2377</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Summary:&lt;br/&gt;
mongos fails temporarily when replica set primary member goes down (&quot;dbclient error communicating with server: &amp;lt;host&amp;gt;:&amp;lt;port&amp;gt;&quot;), then fails semi-permanently until all replica set are up (&quot;mongos connectionpool: connect failed &amp;lt;replicaSet&amp;gt;/&amp;lt;host&amp;gt;:&amp;lt;port&amp;gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;,&amp;lt;host&amp;gt;:&amp;lt;port&amp;gt;...&amp;#93;&lt;/span&gt;&quot;) and (&quot;not master and slaveok=false&quot;)&lt;br/&gt;
I wonder if mongos should:&lt;br/&gt;
1.	Do auto-retry/auto-reconnect at least for read operations&lt;br/&gt;
2.	Do not fail permanently until replica set has all servers running again&lt;/p&gt;

&lt;p&gt;Configuration:&lt;br/&gt;
A set of 3 machines hosting replica set (named testRS), config servers and an instance of mongos. Sharding is enabled, no actual collections are sharded.&lt;br/&gt;
celestine-1: config1, rs1, mongos&lt;br/&gt;
celestine-2: config2, rs2&lt;br/&gt;
celestine-3: config3, rs3&lt;br/&gt;
One user database &quot;test1&quot;, having one collection &quot;items&quot; with two documents (see session below).&lt;/p&gt;

&lt;p&gt;Versions:&lt;br/&gt;
mongos 1.7.5 nightly (2011-01-18), used it because mongos 1.6.5/1.6.6 causes mongo shell to fail with assertion (ERROR: MessagingPort::call() wrong id got:XXX expect:YYY)&lt;br/&gt;
mongod 1.7.5 nightly (2011-01-18)&lt;br/&gt;
mongo shell 1.7.5 nightly (2011-01-18)&lt;/p&gt;

&lt;p&gt;Mongos session:&lt;br/&gt;
&amp;gt; db.items.find()&lt;/p&gt;
{ &quot;_id&quot; : ObjectId(&quot;4d35bea8ba15dc15b0d3878e&quot;), &quot;value&quot; : 123456789012345 }
{ &quot;_id&quot; : ObjectId(&quot;4d35c228abfec6a6bac2d04b&quot;), &quot;value&quot; : 98 }

&lt;p&gt;&amp;#8212; bring down primary member of replica set (celestine-2 ATM) here &amp;#8212;&lt;/p&gt;

&lt;p&gt;&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;dbclient error communicating with server: celestine-2:27100&quot;,&lt;br/&gt;
        &quot;code&quot; : 10278&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;dbclient error communicating with server: celestine-2:27100&quot;,&lt;br/&gt;
        &quot;code&quot; : 10278&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&quot;,&lt;br/&gt;
        &quot;code&quot; : 11002&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: &lt;/p&gt;
{ &quot;$err&quot; : &quot;not master and slaveok=false&quot;, &quot;code&quot; : 13435 }

&lt;p&gt;Mongos log:&lt;br/&gt;
Wed Jan 19 12:31:46 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; ~ScopedDbConnection: _conn != null&lt;br/&gt;
Wed Jan 19 12:31:46 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; caught exception while doing balance: DBClientBase::findOne: transport error: celestine-1:27100 query: &lt;/p&gt;
{ features: 1 }
&lt;p&gt;Wed Jan 19 12:32:36 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; ~ScopedDbConnection: _conn != null&lt;br/&gt;
Wed Jan 19 12:32:36 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; caught exception while doing balance: mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&lt;br/&gt;
Wed Jan 19 12:33:06 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; ~ScopedDbConnection: _conn != null&lt;br/&gt;
Wed Jan 19 12:33:06 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; caught exception while doing balance: mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&lt;/p&gt;
</description>
                <environment>Linux x86/x86_64</environment>
        <key id="14250">SERVER-2377</key>
            <summary>Mongos sharding/failover behaves strangely</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kristina">Kristina Chodorow</assignee>
                                    <reporter username="onyxmaster">Aristarkh Zagorodnikov</reporter>
                        <labels>
                    </labels>
                <created>Wed, 19 Jan 2011 09:48:04 +0000</created>
                <updated>Tue, 12 Jul 2016 00:17:06 +0000</updated>
                            <resolved>Mon, 24 Jan 2011 14:47:52 +0000</resolved>
                                    <version>1.7.5</version>
                                    <fixVersion>1.7.5</fixVersion>
                                    <component>Replication</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>0</watches>
                                                                                                                <comments>
                            <comment id="22618" author="kristina" created="Mon, 24 Jan 2011 14:47:52 +0000"  >&lt;p&gt;Great!&lt;/p&gt;</comment>
                            <comment id="22617" author="onyxmaster" created="Mon, 24 Jan 2011 09:16:22 +0000"  >&lt;p&gt;Checked this today, works as you described.&lt;/p&gt;</comment>
                            <comment id="22606" author="onyxmaster" created="Sat, 22 Jan 2011 17:31:39 +0000"  >&lt;p&gt;Will check this on Monday, thanks&lt;/p&gt;</comment>
                            <comment id="22587" author="kristina" created="Fri, 21 Jan 2011 19:01:01 +0000"  >&lt;p&gt;Can you give it a try?  It&apos;ll take a few seconds but will fail over:&lt;/p&gt;

&lt;p&gt;&amp;gt; db.items.find()&lt;/p&gt;
{ &quot;_id&quot; : ObjectId(&quot;4d35bea8ba15dc15b0d3878e&quot;), &quot;value&quot; : 123456789012345 }
{ &quot;_id&quot; : ObjectId(&quot;4d35c228abfec6a6bac2d04b&quot;), &quot;value&quot; : 98 }
&lt;p&gt;&amp;gt; // killed servers&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: &lt;/p&gt;
{ &quot;$err&quot; : &quot;ReplicaSetMonitor no master found for set: foo&quot;, &quot;code&quot; : 10009 }
&lt;p&gt;&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;mongos connectionpool: connect failed foo/ubuntu:27017,ubuntu:27019,ubuntu:27018 : connect failed to set foo/ubuntu:27017,ubuntu:27019,ubuntu:27018&quot;,&lt;br/&gt;
        &quot;code&quot; : 11002&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: &lt;/p&gt;
{
        &quot;$err&quot; : &quot;mongos connectionpool: connect failed foo/ubuntu:27017,ubuntu:27019,ubuntu:27018 : connect failed to set foo/ubuntu:27017,ubuntu:27019,ubuntu:27018&quot;,
        &quot;code&quot; : 11002
}
&amp;gt; db.items.find()
{ &quot;_id&quot; : ObjectId(&quot;4d35bea8ba15dc15b0d3878e&quot;), &quot;value&quot; : 123456789012345 }
{ &quot;_id&quot; : ObjectId(&quot;4d35c228abfec6a6bac2d04b&quot;), &quot;value&quot; : 98 }</comment>
                            <comment id="22586" author="auto" created="Fri, 21 Jan 2011 18:59:27 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;login&apos;: u&apos;kchodorow&apos;, u&apos;name&apos;: u&apos;Kristina&apos;, u&apos;email&apos;: u&apos;kristina@10gen.com&apos;}
&lt;p&gt;Message: rs throws out master on connection exception &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-2377&quot; title=&quot;Mongos sharding/failover behaves strangely&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-2377&quot;&gt;&lt;del&gt;SERVER-2377&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/f0eb24f2245d438a47b677a3c33d3c02c73f7097&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/f0eb24f2245d438a47b677a3c33d3c02c73f7097&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="22585" author="kristina" created="Fri, 21 Jan 2011 18:57:04 +0000"  >&lt;p&gt;You are correct.  Will fix in a sec... (there&apos;s a simpler fix)&lt;/p&gt;</comment>
                            <comment id="22578" author="onyxmaster" created="Fri, 21 Jan 2011 15:45:52 +0000"  >&lt;p&gt;Below is the version of the patch with the proposed loop (please take note that slave case is completely untested and I removed the &quot;if ( h != _slaveHost )&quot; check in the slave version.&lt;br/&gt;
This leads to the following session after killing the primary:&lt;br/&gt;
With this patch, after killing primary I get the following on the mongos client: &lt;br/&gt;
&amp;gt; db.items.find() &lt;br/&gt;
error: &lt;/p&gt;
{ 
        &quot;$err&quot; : &quot;dbclient error communicating with server: celestine-3:27100&quot;, 
        &quot;code&quot; : 10278 
} 
&amp;gt; db.items.find() 
{ &quot;_id&quot; : ObjectId(&quot;4d384a2061befa37946181af&quot;), &quot;v&quot; : 19959 }
&lt;p&gt; &lt;br/&gt;
... &lt;br/&gt;
has more &lt;/p&gt;


&lt;p&gt;&amp;#8212; client/dbclient_rs.cpp.orig	2011-01-21 18:06:33.478345002 +0300&lt;br/&gt;
+++ client/dbclient_rs.cpp	2011-01-21 18:40:54.338344995 +0300&lt;br/&gt;
@@ -365,9 +365,26 @@&lt;br/&gt;
             _monitor-&amp;gt;notifyFailure( _masterHost );&lt;br/&gt;
         }&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;_masterHost = _monitor-&amp;gt;getMaster();&lt;/li&gt;
	&lt;li&gt;_master.reset( new DBClientConnection( true ) );&lt;/li&gt;
	&lt;li&gt;_master-&amp;gt;connect( _masterHost );&lt;br/&gt;
+        size_t attempts = _monitor-&amp;gt;getNodesCount();&lt;br/&gt;
+        while (true) {&lt;br/&gt;
+    	    _masterHost = _monitor-&amp;gt;getMaster();&lt;br/&gt;
+	        _master.reset( new DBClientConnection( true ) );&lt;br/&gt;
+	        try&lt;br/&gt;
+    	    
{
+	        	_master-&amp;gt;connect( _masterHost );
+	        }
&lt;p&gt;+    	    catch (ConnectException&amp;amp;)&lt;br/&gt;
+	        &lt;/p&gt;
&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: {+    	    	_monitor-&amp;gt;notifyFailure( _masterHost );+    	    	if (attempts &amp;gt; 0)+    	    	{
+    	    		--attempts;
+	    	    	continue;
+	    	    }+	        	throw;+	        }&lt;/span&gt; &lt;/div&gt;
&lt;p&gt;+	        break;&lt;br/&gt;
+        }&lt;br/&gt;
         _auth( _master.get() );&lt;br/&gt;
         return _master.get();&lt;br/&gt;
     }&lt;br/&gt;
@@ -379,13 +396,27 @@&lt;br/&gt;
             _monitor-&amp;gt;notifySlaveFailure( _slaveHost );&lt;br/&gt;
         }&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;HostAndPort h = _monitor-&amp;gt;getSlave();&lt;/li&gt;
	&lt;li&gt;if ( h != _slaveHost ) {&lt;/li&gt;
	&lt;li&gt;_slaveHost = h;&lt;br/&gt;
+        size_t attempts = _monitor-&amp;gt;getNodesCount();&lt;br/&gt;
+        while (true) 
{
+            _slaveHost = _monitor-&amp;gt;getSlave();
             _slave.reset( new DBClientConnection( true ) );
-            _slave-&amp;gt;connect( _slaveHost );
-            _auth( _slave.get() );
-        }
&lt;p&gt;+        	try&lt;br/&gt;
+    	    &lt;/p&gt;
{
+	            _slave-&amp;gt;connect( _slaveHost );
+        	}
&lt;p&gt;+        	catch (ConnectException&amp;amp;)&lt;br/&gt;
+        	&lt;/p&gt;
&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: {+    	    	_monitor-&amp;gt;notifySlaveFailure( _slaveHost );+   	    		if (attempts &amp;gt; 0)+   	    		{
+   		    		--attempts;
+	    	    	continue;
+	    	    }+        		throw;+	        }&lt;/span&gt; &lt;/div&gt;
&lt;p&gt;+	        break;&lt;br/&gt;
+	    }&lt;br/&gt;
+		_auth( _slave.get() );&lt;br/&gt;
         return _slave.get();&lt;br/&gt;
     }&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&amp;#8212; client/dbclient_rs.h.orig	2011-01-21 18:27:11.490345008 +0300&lt;br/&gt;
+++ client/dbclient_rs.h	2011-01-21 18:27:46.702345006 +0300&lt;br/&gt;
@@ -83,6 +83,8 @@&lt;/p&gt;

&lt;p&gt;         string getServerAddress() const;&lt;/p&gt;

&lt;p&gt;+        size_t getNodesCount() const &lt;/p&gt;
{ return _nodes.size(); }
&lt;p&gt;+&lt;br/&gt;
     private:&lt;br/&gt;
         /**&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;This populates a list of hosts from the list of seeds (discarding the&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="22577" author="onyxmaster" created="Fri, 21 Jan 2011 15:15:24 +0000"  >&lt;p&gt;I explored client/dbclient_rs.cpp a bit and found out that _nodes&lt;span class=&quot;error&quot;&gt;&amp;#91;..&amp;#93;&lt;/span&gt;.ok won&apos;t get set to false in my scenario (connection errors aren&apos;t handled at all and ReplicaSetMonitorWatcher job executes only once for each replica set).&lt;br/&gt;
I concluded a small patch (mostly sure that it is a wrong way to solve this problem, but at least it appears to work, also I don&apos;t know if slaves should be handled in this way).&lt;br/&gt;
With this patch, after killing primary I get the following on the mongos client:&lt;br/&gt;
&amp;gt; db.items.find() &lt;br/&gt;
error: { &lt;br/&gt;
        &quot;$err&quot; : &quot;dbclient error communicating with server: celestine-2:27100&quot;, &lt;br/&gt;
        &quot;code&quot; : 10278 &lt;br/&gt;
} &lt;br/&gt;
&amp;gt; db.items.find() &lt;br/&gt;
error: &lt;/p&gt;
{ 
        &quot;$err&quot; : &quot;mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&quot;, 
        &quot;code&quot; : 11002 
} 
&amp;gt; db.items.find() 
{ &quot;_id&quot; : ObjectId(&quot;4d384a2061befa37946181af&quot;), &quot;v&quot; : 19959 }
&lt;p&gt; &lt;br/&gt;
... &lt;br/&gt;
has more &lt;/p&gt;

&lt;p&gt;So, looks like error does not become permanent after failure. I&apos;m sure this can be improved, so there is at most one error per server failure (adding some kind of a loop in DBClientReplicaSet::checkMaster() that gives up after N attempts, where N is ReplicaSetMonitor._nodes.size(), might be feasible).&lt;/p&gt;

&lt;p&gt;&amp;#8212; client/dbclient_rs.cpp.orig	2011-01-21 18:06:33.478345002 +0300&lt;br/&gt;
+++ client/dbclient_rs.cpp	2011-01-21 17:58:03.874345000 +0300&lt;br/&gt;
@@ -367,7 +367,15 @@&lt;/p&gt;

&lt;p&gt;         _masterHost = _monitor-&amp;gt;getMaster();&lt;br/&gt;
         _master.reset( new DBClientConnection( true ) );&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;_master-&amp;gt;connect( _masterHost );&lt;br/&gt;
+        try&lt;br/&gt;
+        
{
+        	_master-&amp;gt;connect( _masterHost );
+        }
&lt;p&gt;+        catch (ConnectException&amp;amp;)&lt;br/&gt;
+        &lt;/p&gt;
{
+        	_monitor-&amp;gt;notifyFailure( _masterHost );
+        	throw;
+        }
&lt;p&gt;         _auth( _master.get() );&lt;br/&gt;
         return _master.get();&lt;br/&gt;
     }&lt;br/&gt;
@@ -383,7 +391,15 @@&lt;br/&gt;
         if ( h != _slaveHost ) {&lt;br/&gt;
             _slaveHost = h;&lt;br/&gt;
             _slave.reset( new DBClientConnection( true ) );&lt;/p&gt;&lt;/li&gt;
	&lt;li&gt;_slave-&amp;gt;connect( _slaveHost );&lt;br/&gt;
+        	try&lt;br/&gt;
+    	    
{
+	            _slave-&amp;gt;connect( _slaveHost );
+        	}
&lt;p&gt;+        	catch (ConnectException&amp;amp;)&lt;br/&gt;
+        	&lt;/p&gt;
{
+    	    	_monitor-&amp;gt;notifySlaveFailure( _slaveHost );
+        		throw;
+	        }
&lt;p&gt;             _auth( _slave.get() );&lt;br/&gt;
         }&lt;br/&gt;
         return _slave.get();&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="22572" author="onyxmaster" created="Fri, 21 Jan 2011 11:24:25 +0000"  >&lt;p&gt;Looks like original problem isn&apos;t solved.&lt;br/&gt;
Built from git master (c31c7f322943034bd294993e832f51383799ee5f) mongos still behaves in the same way:&lt;/p&gt;

&lt;p&gt;xm@celestine-1:~$ ./db/bin/mongo celestine-1/test1&lt;br/&gt;
MongoDB shell version: 1.7.5-pre-&lt;br/&gt;
connecting to: celestine-1/test1&lt;br/&gt;
&amp;gt; db.items.find()&lt;/p&gt;
{ &quot;_id&quot; : ObjectId(&quot;4d384a2061befa37946181af&quot;), &quot;v&quot; : 19959 }
&lt;p&gt;...&lt;br/&gt;
has more&lt;/p&gt;

&lt;p&gt;&amp;#8212; kill primary here &amp;#8212;&lt;/p&gt;

&lt;p&gt;&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;dbclient error communicating with server: celestine-2:27100&quot;,&lt;br/&gt;
        &quot;code&quot; : 10278&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&quot;,&lt;br/&gt;
        &quot;code&quot; : 11002&lt;br/&gt;
}&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: {&lt;br/&gt;
        &quot;$err&quot; : &quot;mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&quot;,&lt;br/&gt;
        &quot;code&quot; : 11002&lt;br/&gt;
}&lt;/p&gt;

&lt;p&gt;Mongos log:&lt;br/&gt;
Fri Jan 21 14:21:20 /home/xm/db/bin/mongos db version v1.7.5-pre-, pdfile version 4.5 starting (--help for usage)&lt;br/&gt;
Fri Jan 21 14:21:20 git version: nogitversion&lt;br/&gt;
Fri Jan 21 14:21:20 sys info: Linux celestine-1 2.6.35-22-generic-pae #34~lucid1-Ubuntu SMP Mon Oct 11 14:53:39 UTC 2010 i686 BOOST_LIB_VERSION=1_40&lt;br/&gt;
Fri Jan 21 14:21:20 SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-1:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-2:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-3:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;mongosMain&amp;#93;&lt;/span&gt; waiting for connections on port 27017&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; about to contact config servers and shards&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;websvr&amp;#93;&lt;/span&gt; web admin interface listening on port 28017&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; updated set (testRS) to: testRS/celestine-1:27100,celestine-3:27100&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; updated set (testRS) to: testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; config servers and shards contacted successfully&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; balancer id: celestine-1:27017 started at Jan 21 14:21:20&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;LockPinger&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-1:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;LockPinger&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-2:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-1:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;LockPinger&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-3:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-2:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-3:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:21:28 &lt;span class=&quot;error&quot;&gt;&amp;#91;mongosMain&amp;#93;&lt;/span&gt; connection accepted from 192.168.7.241:35334 #1&lt;br/&gt;
Fri Jan 21 14:21:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn1&amp;#93;&lt;/span&gt; creating WriteBackListener for: testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100,celestine-3:27100,celestine-2:27100&lt;br/&gt;
Fri Jan 21 14:21:40 checking replica set: testRS&lt;br/&gt;
Fri Jan 21 14:21:50 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; ~ScopedDbConnection: _conn != null&lt;br/&gt;
Fri Jan 21 14:21:50 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; caught exception while doing balance: mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&lt;br/&gt;
Fri Jan 21 14:22:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; ~ScopedDbConnection: _conn != null&lt;br/&gt;
Fri Jan 21 14:22:20 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; caught exception while doing balance: mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&lt;br/&gt;
Fri Jan 21 14:22:50 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; ~ScopedDbConnection: _conn != null&lt;br/&gt;
Fri Jan 21 14:22:50 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; caught exception while doing balance: mongos connectionpool: connect failed testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100 : connect failed to set testRS/celestine-1:27100,celestine-3:27100,celestine-2:27100&lt;br/&gt;
Fri Jan 21 14:22:51 &lt;span class=&quot;error&quot;&gt;&amp;#91;LockPinger&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-1:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:22:51 &lt;span class=&quot;error&quot;&gt;&amp;#91;LockPinger&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-2:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:22:51 &lt;span class=&quot;error&quot;&gt;&amp;#91;LockPinger&amp;#93;&lt;/span&gt; SyncClusterConnection connecting to &lt;span class=&quot;error&quot;&gt;&amp;#91;celestine-3:27200&amp;#93;&lt;/span&gt;&lt;br/&gt;
Fri Jan 21 14:23:00 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn1&amp;#93;&lt;/span&gt; end connection 192.168.7.241:35334&lt;/p&gt;</comment>
                            <comment id="22556" author="kristina" created="Fri, 21 Jan 2011 00:37:35 +0000"  >&lt;p&gt;It should now find the new master.&lt;/p&gt;</comment>
                            <comment id="22555" author="auto" created="Fri, 21 Jan 2011 00:36:52 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;login&apos;: u&apos;kchodorow&apos;, u&apos;name&apos;: u&apos;Kristina&apos;, u&apos;email&apos;: u&apos;kristina@10gen.com&apos;}
&lt;p&gt;Message: make c++ client check that rs master is ok before returning it &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-2377&quot; title=&quot;Mongos sharding/failover behaves strangely&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-2377&quot;&gt;&lt;del&gt;SERVER-2377&lt;/del&gt;&lt;/a&gt;&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a756cf2375574732d77479fe3a4ed1c4138feb4e&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a756cf2375574732d77479fe3a4ed1c4138feb4e&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="22533" author="onyxmaster" created="Thu, 20 Jan 2011 20:48:21 +0000"  >&lt;p&gt;My fault for the setSlaveOk() part, yes. I thought (don&apos;t know why, docs aren&apos;t saying that) that mongos defaults to reading from slaves.&lt;/p&gt;</comment>
                            <comment id="22531" author="kristina" created="Thu, 20 Jan 2011 18:57:44 +0000"  >&lt;p&gt;Only part of the problem, but &lt;/p&gt;
{ &quot;$err&quot; : &quot;not master and slaveok=false&quot;, &quot;code&quot; : 13435 }
&lt;p&gt; just means you&apos;re trying to query a slave without saying db.getMongo().setSlaveOk() first.  Try:&lt;/p&gt;

&lt;p&gt;&amp;gt; db.getMongo().setSlaveOk()&lt;br/&gt;
&amp;gt; db.items.find() &lt;/p&gt;

&lt;p&gt;once all of the servers are up again.&lt;/p&gt;</comment>
                            <comment id="22527" author="onyxmaster" created="Thu, 20 Jan 2011 08:29:00 +0000"  >&lt;p&gt;~$ ./db/bin/mongo celestine-1/test1&lt;br/&gt;
MongoDB shell version: 1.7.5-pre-&lt;br/&gt;
connecting to: localhost/test1&lt;br/&gt;
&amp;gt; db.items.find()&lt;br/&gt;
error: &lt;/p&gt;
{ &quot;$err&quot; : &quot;not master and slaveok=false&quot;, &quot;code&quot; : 13435 }

&lt;p&gt;~$ ./db/bin/mongo celestine-1:27100&lt;br/&gt;
MongoDB shell version: 1.7.5-pre-&lt;br/&gt;
connecting to: celestine-1:27100/test&lt;br/&gt;
testRS:PRIMARY&amp;gt; rs.status()&lt;br/&gt;
{&lt;br/&gt;
        &quot;set&quot; : &quot;testRS&quot;,&lt;br/&gt;
        &quot;date&quot; : ISODate(&quot;2011-01-20T08:27:40Z&quot;),&lt;br/&gt;
        &quot;myState&quot; : 1,&lt;br/&gt;
        &quot;members&quot; : [&lt;br/&gt;
                {&lt;br/&gt;
                        &quot;_id&quot; : 0,&lt;br/&gt;
                        &quot;name&quot; : &quot;celestine-1:27100&quot;,&lt;br/&gt;
                        &quot;health&quot; : 1,&lt;br/&gt;
                        &quot;state&quot; : 1,&lt;br/&gt;
                        &quot;stateStr&quot; : &quot;PRIMARY&quot;,&lt;br/&gt;
                        &quot;optime&quot; : &lt;/p&gt;
{
                                &quot;t&quot; : 1295443467000,
                                &quot;i&quot; : 1
                        }
&lt;p&gt;,&lt;br/&gt;
                        &quot;optimeDate&quot; : ISODate(&quot;2011-01-19T13:24:27Z&quot;),&lt;br/&gt;
                        &quot;self&quot; : true&lt;br/&gt;
                },&lt;br/&gt;
                {&lt;br/&gt;
                        &quot;_id&quot; : 1,&lt;br/&gt;
                        &quot;name&quot; : &quot;celestine-2:27100&quot;,&lt;br/&gt;
                        &quot;health&quot; : 1,&lt;br/&gt;
                        &quot;state&quot; : 2,&lt;br/&gt;
                        &quot;stateStr&quot; : &quot;SECONDARY&quot;,&lt;br/&gt;
                        &quot;uptime&quot; : 58665,&lt;br/&gt;
                        &quot;optime&quot; : &lt;/p&gt;
{
                                &quot;t&quot; : 1295443467000,
                                &quot;i&quot; : 1
                        }
&lt;p&gt;,&lt;br/&gt;
                        &quot;optimeDate&quot; : ISODate(&quot;2011-01-19T13:24:27Z&quot;),&lt;br/&gt;
                        &quot;lastHeartbeat&quot; : ISODate(&quot;2011-01-20T08:27:39Z&quot;)&lt;br/&gt;
                },&lt;br/&gt;
                {&lt;br/&gt;
                        &quot;_id&quot; : 2,&lt;br/&gt;
                        &quot;name&quot; : &quot;celestine-3:27100&quot;,&lt;br/&gt;
                        &quot;health&quot; : 1,&lt;br/&gt;
                        &quot;state&quot; : 2,&lt;br/&gt;
                        &quot;stateStr&quot; : &quot;SECONDARY&quot;,&lt;br/&gt;
                        &quot;uptime&quot; : 58668,&lt;br/&gt;
                        &quot;optime&quot; : &lt;/p&gt;
{
                                &quot;t&quot; : 1295443467000,
                                &quot;i&quot; : 1
                        }
&lt;p&gt;,&lt;br/&gt;
                        &quot;optimeDate&quot; : ISODate(&quot;2011-01-19T13:24:27Z&quot;),&lt;br/&gt;
                        &quot;lastHeartbeat&quot; : ISODate(&quot;2011-01-20T08:27:39Z&quot;)&lt;br/&gt;
                }&lt;br/&gt;
        ],&lt;br/&gt;
        &quot;ok&quot; : 1&lt;br/&gt;
}&lt;/p&gt;</comment>
                            <comment id="22524" author="eliot" created="Thu, 20 Jan 2011 06:08:38 +0000"  >&lt;p&gt;I&apos;m a little confused about what state things were in at various error messages.&lt;/p&gt;

&lt;p&gt;When you got:&lt;br/&gt;
error: &lt;/p&gt;
{ &quot;$err&quot; : &quot;not master and slaveok=false&quot;, &quot;code&quot; : 13435 }

&lt;p&gt;What state was the replica set in?&lt;/p&gt;
</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 20 Jan 2011 06:08:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        13 years, 4 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            13 years, 4 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>onyxmaster</customfieldvalue>
            <customfieldvalue>auto</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>kristina</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrp8bb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hricm7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>20695</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrlm6v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>