[SERVER-8756] 2.2.2 RS with SSL, upgrade one node to 2.4 with --sslCAFile - either 1. should work or 2. message misleading Created: 27/Feb/13  Updated: 09/Jul/16  Resolved: 27/Feb/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Gregor Macadam Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Operating System: ALL
Participants:

 Description   

2 micro instances, one instance running 2.2.2

./bin/mongod --dbpath ./data --sslOnNormalPorts --sslPEMKeyFile ../gregorFreeBSD.pem --sslPEMKeyPassword mongo --replSet rs1

the other instance running 2.2.2 node and arbiter.

 ./bin/mongod --dbpath ./data/ --sslOnNormalPorts --sslPEMKeyFile ../gregorFreeBSD.pem --sslPEMKeyPassword mongo --replSet rs1 --smallfiles

./bin/mongod --dbpath ./data/arb/ --port 30000 --replSet rs1 --sslOnNormalPorts --sslPEMKeyFile ../gregorFreeBSD.pem  --sslPEMKeyPassword mongo --smallfiles

rs1:PRIMARY> rs.status()
{
	"set" : "rs1",
	"date" : ISODate("2013-02-27T12:24:09Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "ip-10-227-21-162:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 2399,
			"optime" : Timestamp(1361966564000, 1),
			"optimeDate" : ISODate("2013-02-27T12:02:44Z"),
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "ip-10-48-238-94:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 1634,
			"optime" : Timestamp(1361966564000, 1),
			"optimeDate" : ISODate("2013-02-27T12:02:44Z"),
			"lastHeartbeat" : ISODate("2013-02-27T12:24:08Z"),
			"pingMs" : 0
		},
		{
			"_id" : 2,
			"name" : "ip-10-48-238-94:30000",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 1285,
			"lastHeartbeat" : ISODate("2013-02-27T12:24:08Z"),
			"pingMs" : 0
		}
	],
	"ok" : 1
}

Take node from second instance offline and upgrade to 2.4.0-rc1 with --sslCAFile.

./bin/mongod --dbpath ../mongodb-linux-x86_64-subscription-ubuntu1104-2.2.2/data/ --sslOnNormalPorts --sslPEMKeyFile ../gregorFreeBSD.pem --sslCAFile ../cacert.pem --replSet rs1 --smallfiles

Repeated error message on this node upon restart is

Wed Feb 27 12:28:42.472 [initandlisten] connection accepted from 10.48.238.94:41645 #195 (1 connection now open)
Wed Feb 27 12:28:42.474 [conn195] ERROR: no SSL certificate provided by peer; connection rejected
Wed Feb 27 12:28:42.475 [conn195] SocketException handling request, closing client connection: 9001 socket exception [6] 
Wed Feb 27 12:28:43.680 [initandlisten] connection accepted from 10.227.21.162:56636 #196 (1 connection now open)
Wed Feb 27 12:28:43.683 [conn196] ERROR: no SSL certificate provided by peer; connection rejected
Wed Feb 27 12:28:43.683 [conn196] SocketException handling request, closing client connection: 9001 socket exception [6] 
Wed Feb 27 12:28:44.476 [initandlisten] connection accepted from 10.48.238.94:41647 #197 (1 connection now open)
Wed Feb 27 12:28:44.478 [conn197] ERROR: no SSL certificate provided by peer; connection rejected
Wed Feb 27 12:28:44.478 [conn197] SocketException handling request, closing client connection: 9001 socket exception [6] 

This is incorrect - other nodes are using a CA signed cert so surely this upgrade should work? Or if not then the error message

no SSL certificate provided by peer

is incorrect - they are supplying a cert, but it's not being accepted.



 Comments   
Comment by Eric Milkie [ 27/Feb/13 ]

You can achieve the same result if you make your network topology asymmetric, by allowing connections outbound from your 2.4 node but disallowing incoming connections.

Comment by Gregor Macadam [ 27/Feb/13 ]

Strangely, the node that now has --sslCAFile is still a secondary
ip-10-48-238-94 is the upgraded node using --sslCAFile

rs1:SECONDARY> rs.status()
{
	"set" : "rs1",
	"date" : ISODate("2013-02-27T16:39:56Z"),
	"myState" : 2,
	"syncingTo" : "ip-10-227-21-162:27017",
	"members" : [
		{
			"_id" : 0,
			"name" : "ip-10-227-21-162:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 108,
			"optime" : {
				"t" : 1361966564000,
				"i" : 1
			},
			"optimeDate" : ISODate("2013-02-27T12:02:44Z"),
			"lastHeartbeat" : ISODate("2013-02-27T16:39:54Z"),
			"lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
			"pingMs" : 0
		},
		{
			"_id" : 1,
			"name" : "ip-10-48-238-94:27017",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 108,
			"optime" : {
				"t" : 1361966564000,
				"i" : 1
			},
			"optimeDate" : ISODate("2013-02-27T12:02:44Z"),
			"errmsg" : "syncing to: ip-10-227-21-162:27017",
			"self" : true
		},
		{
			"_id" : 2,
			"name" : "ip-10-48-238-94:30000",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 108,
			"lastHeartbeat" : ISODate("2013-02-27T16:39:56Z"),
			"lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
			"pingMs" : 0
		}
	],
	"ok" : 1
}
rs1:SECONDARY> 

but from the PRIMARY it looks like node down

rs1:PRIMARY> rs.status()
{
	"set" : "rs1",
	"date" : ISODate("2013-02-27T16:38:36Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "ip-10-227-21-162:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 14649,
			"optime" : {
				"t" : 1361966564000,
				"i" : 1
			},
			"optimeDate" : ISODate("2013-02-27T12:02:44Z"),
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "ip-10-48-238-94:27017",
			"health" : 0,
			"state" : 8,
			"stateStr" : "(not reachable/healthy)",
			"uptime" : 0,
			"optime" : {
				"t" : 1361966564000,
				"i" : 1
			},
			"optimeDate" : ISODate("2013-02-27T12:02:44Z"),
			"lastHeartbeat" : ISODate("2013-02-27T15:51:04Z"),
			"pingMs" : 0,
			"errmsg" : "DBClientBase::findN: transport error: ip-10-48-238-94:27017 ns: admin.$cmd query: { replSetHeartbeat: \"rs1\", v: 3, pv: 1, checkEmpty: false, from: \"ip-10-227-21-162:27017\", $auth: {} }"
		},
		{
			"_id" : 2,
			"name" : "ip-10-48-238-94:30000",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 2855,
			"lastHeartbeat" : ISODate("2013-02-27T16:38:36Z"),
			"pingMs" : 0
		}
	],
	"ok" : 1
}
rs1:PRIMARY> ^C

Comment by Eric Milkie [ 27/Feb/13 ]

In version 2.2, the C++ driver will never present a client certificate when it connects with SSL. Internal connections between mongod's can be thought of as using the C++ driver to make outgoing connections, so these internal connections will never present a client certificate in version 2.2.

This is the purpose of --sslWeakCertificateValidation, to allow this type of upgrade.
The message isn't incorrect, but it would be hard to succinctly add anything to it to help in this corner case. I think if we add more documentation about the use case of --sslWeakCertificateValidation this should help to clear things up.

Generated at Thu Feb 08 03:18:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.