Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8461

mongod running with GSSAPI cannot be part of a replica set without MONGO-CR enabled

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.4.0-rc1
    • Affects Version/s: 2.4.0-rc0
    • Component/s: Security
    • Labels:
      None
    • Environment:
      MongoDB 2.4.0-rc0 running on RHEL6.3

      See below for issues with rs.add. The traffic (containing the request for 10.0.5.120 to be part of the replica set where 10.0.5.110 is primary) is seen leaving 10.0.5.110. however, 10.0.5.120 doesn't respond to the request (nonce failure I think).

      Two servers:

      kserver1a.realm5.10gen.me - 10.0.5.110

      realm5:PRIMARY> rs.add('kserver1b.realm5.10gen.me:27017')
      {
      	"errmsg" : "exception: need most members up to reconfigure, not ok : kserver1b.realm5.10gen.me:27017",
      	"code" : 13144,
      	"ok" : 0
      }
      realm5:PRIMARY> db.hostInfo()
      {
      	"system" : {
      		"currentTime" : ISODate("2013-02-06T15:41:55.226Z"),
      		"hostname" : "kserver1a.realm5.10gen.me",
      .......
      .......
      .......
      realm5:PRIMARY> rs.status()
      {
      	"set" : "realm5",
      	"date" : ISODate("2013-02-06T15:42:28Z"),
      	"myState" : 1,
      	"members" : [
      		{
      			"_id" : 0,
      			"name" : "kserver1a.realm5.10gen.me:27017",
      			"health" : 1,
      			"state" : 1,
      			"stateStr" : "PRIMARY",
      			"uptime" : 383,
      			"optime" : {
      				"t" : 1360162789000,
      				"i" : 1
      			},
      			"optimeDate" : ISODate("2013-02-06T14:59:49Z"),
      			"self" : true
      		}
      	],
      	"ok" : 1
      }
      realm5:PRIMARY> rs.conf()
      {
      	"_id" : "realm5",
      	"version" : 1,
      	"members" : [
      		{
      			"_id" : 0,
      			"host" : "kserver1a.realm5.10gen.me:27017"
      		}
      	]
      }
      

      kserver1b.realm5.10gen.me - 10.0.5.120

      [root@kserver1b ~]# hostname -f
      kserver1b.realm5.10gen.me
      [root@kserver1b ~]# psm
      root     10121  0.3  2.1 732184 35640 ?        Sl   10:37   0:00 /usr/local/bin/mongodb/bin/mongod --auth --setParameter authenticationMechanisms=GSSAPI --dbpath /data/db --fork --logpath /var/tmp/mongod_auth.log --smallfiles --nojournal --replSet realm5 --keyFile /etc/keyfile
      [root@kserver1b ~]# tcpdump -nnpi eth0 port 27017
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
      10:40:38.208945 IP 10.0.5.110.34177 > 10.0.5.120.27017: Flags [S], seq 1196150602, win 14600, options [mss 1460,sackOK,TS val 177359655 ecr 0,nop,wscale 7], length 0
      10:40:38.209021 IP 10.0.5.120.27017 > 10.0.5.110.34177: Flags [S.], seq 101929863, ack 1196150603, win 14480, options [mss 1460,sackOK,TS val 177258701 ecr 177359655,nop,wscale 7], length 0
      10:40:38.209547 IP 10.0.5.110.34177 > 10.0.5.120.27017: Flags [.], ack 1, win 115, options [nop,nop,TS val 177359656 ecr 177258701], length 0
      10:40:38.209664 IP 10.0.5.110.34177 > 10.0.5.120.27017: Flags [P.], seq 1:59, ack 1, win 115, options [nop,nop,TS val 177359656 ecr 177258701], length 58
      10:40:38.209682 IP 10.0.5.120.27017 > 10.0.5.110.34177: Flags [.], ack 59, win 114, options [nop,nop,TS val 177258702 ecr 177359656], length 0
      10:40:38.212447 IP 10.0.5.120.27017 > 10.0.5.110.34177: Flags [P.], seq 1:154, ack 59, win 114, options [nop,nop,TS val 177258704 ecr 177359656], length 153
      10:40:38.212898 IP 10.0.5.110.34177 > 10.0.5.120.27017: Flags [.], ack 154, win 123, options [nop,nop,TS val 177359659 ecr 177258704], length 0
      10:40:38.212952 IP 10.0.5.110.34177 > 10.0.5.120.27017: Flags [F.], seq 59, ack 154, win 123, options [nop,nop,TS val 177359659 ecr 177258704], length 0
      10:40:38.213134 IP 10.0.5.120.27017 > 10.0.5.110.34177: Flags [F.], seq 154, ack 60, win 114, options [nop,nop,TS val 177258705 ecr 177359659], length 0
      10:40:38.213561 IP 10.0.5.110.34177 > 10.0.5.120.27017: Flags [.], ack 155, win 123, options [nop,nop,TS val 177359660 ecr 177258705], length 0
      10:40:38.215190 IP 10.0.5.110.34178 > 10.0.5.120.27017: Flags [S], seq 3118371312, win 14600, options [mss 1460,sackOK,TS val 177359661 ecr 0,nop,wscale 7], length 0
      10:40:38.215217 IP 10.0.5.120.27017 > 10.0.5.110.34178: Flags [S.], seq 1189064659, ack 3118371313, win 14480, options [mss 1460,sackOK,TS val 177258707 ecr 177359661,nop,wscale 7], length 0
      10:40:38.239796 IP 10.0.5.110.34178 > 10.0.5.120.27017: Flags [.], ack 1, win 115, options [nop,nop,TS val 177359686 ecr 177258707], length 0
      10:40:38.239859 IP 10.0.5.110.34178 > 10.0.5.120.27017: Flags [P.], seq 1:59, ack 1, win 115, options [nop,nop,TS val 177359686 ecr 177258707], length 58
      10:40:38.239877 IP 10.0.5.120.27017 > 10.0.5.110.34178: Flags [.], ack 59, win 114, options [nop,nop,TS val 177258732 ecr 177359686], length 0
      10:40:38.240312 IP 10.0.5.120.27017 > 10.0.5.110.34178: Flags [P.], seq 1:154, ack 59, win 114, options [nop,nop,TS val 177258732 ecr 177359686], length 153
      10:40:38.240836 IP 10.0.5.110.34178 > 10.0.5.120.27017: Flags [.], ack 154, win 123, options [nop,nop,TS val 177359687 ecr 177258732], length 0
      10:40:38.240878 IP 10.0.5.110.34178 > 10.0.5.120.27017: Flags [F.], seq 59, ack 154, win 123, options [nop,nop,TS val 177359687 ecr 177258732], length 0
      10:40:38.241019 IP 10.0.5.120.27017 > 10.0.5.110.34178: Flags [F.], seq 154, ack 60, win 114, options [nop,nop,TS val 177258733 ecr 177359687], length 0
      10:40:38.241419 IP 10.0.5.110.34178 > 10.0.5.120.27017: Flags [.], ack 155, win 123, options [nop,nop,TS val 177359688 ecr 177258733], length 0
      10:40:38.242978 IP 10.0.5.110.34179 > 10.0.5.120.27017: Flags [S], seq 3997841901, win 14600, options [mss 1460,sackOK,TS val 177359689 ecr 0,nop,wscale 7], length 0
      10:40:38.243013 IP 10.0.5.120.27017 > 10.0.5.110.34179: Flags [S.], seq 1060265148, ack 3997841902, win 14480, options [mss 1460,sackOK,TS val 177258735 ecr 177359689,nop,wscale 7], length 0
      10:40:38.243430 IP 10.0.5.110.34179 > 10.0.5.120.27017: Flags [.], ack 1, win 115, options [nop,nop,TS val 177359690 ecr 177258735], length 0
      10:40:38.243533 IP 10.0.5.110.34179 > 10.0.5.120.27017: Flags [P.], seq 1:59, ack 1, win 115, options [nop,nop,TS val 177359690 ecr 177258735], length 58
      10:40:38.243548 IP 10.0.5.120.27017 > 10.0.5.110.34179: Flags [.], ack 59, win 114, options [nop,nop,TS val 177258736 ecr 177359690], length 0
      10:40:38.243828 IP 10.0.5.120.27017 > 10.0.5.110.34179: Flags [P.], seq 1:154, ack 59, win 114, options [nop,nop,TS val 177258736 ecr 177359690], length 153
      10:40:38.244260 IP 10.0.5.110.34179 > 10.0.5.120.27017: Flags [.], ack 154, win 123, options [nop,nop,TS val 177359690 ecr 177258736], length 0
      10:40:38.244274 IP 10.0.5.110.34179 > 10.0.5.120.27017: Flags [F.], seq 59, ack 154, win 123, options [nop,nop,TS val 177359690 ecr 177258736], length 0
      10:40:38.244338 IP 10.0.5.110.34176 > 10.0.5.120.27017: Flags [P.], seq 2614329454:2614329566, ack 652427820, win 123, options [nop,nop,TS val 177359691 ecr 177109785], length 112
      10:40:38.244482 IP 10.0.5.120.27017 > 10.0.5.110.34176: Flags [P.], seq 1:79, ack 112, win 114, options [nop,nop,TS val 177258736 ecr 177359691], length 78
      10:40:38.244711 IP 10.0.5.120.27017 > 10.0.5.110.34179: Flags [F.], seq 154, ack 60, win 114, options [nop,nop,TS val 177258737 ecr 177359690], length 0
      10:40:38.244936 IP 10.0.5.110.34176 > 10.0.5.120.27017: Flags [.], ack 79, win 123, options [nop,nop,TS val 177359691 ecr 177258736], length 0
      10:40:38.245134 IP 10.0.5.110.34179 > 10.0.5.120.27017: Flags [.], ack 155, win 123, options [nop,nop,TS val 177359691 ecr 177258737], length 0
      

      Note that at this point there were no users configured.

      In the above instance, mongod was running as below -

      env KRB5_KTNAME=/etc/kserver1a.keytab /usr/local/bin/mongodb/bin/mongod --auth --setParameter authenticationMechanisms=GSSAPI --dbpath /data/db --fork --logpath /var/tmp/mongod_auth.log --smallfiles --nojournal --replSet realm5 --keyFile /etc/keyfile 
      

      The issue with replica set communications was fixed by adding MONGO-CR as an authentication mechanism.

      env KRB5_KTNAME=/etc/kserver1a.keytab /usr/local/bin/mongodb/bin/mongod --auth --setParameter authenticationMechanisms=GSSAPI,MONGO-CR --dbpath /data/db --fork --logpath /var/tmp/mongod_auth.log --smallfiles --nojournal --replSet realm5 --keyFile /etc/keyfile 
      
      realm5:PRIMARY> rs.add("kserver1b.realm5.10gen.me:27017")
      { "ok" : 1 }
      realm5:PRIMARY> rs.conf()
      {
      	"_id" : "realm5",
      	"version" : 2,
      	"members" : [
      		{
      			"_id" : 0,
      			"host" : "kserver1a.realm5.10gen.me:27017"
      		},
      		{
      			"_id" : 1,
      			"host" : "kserver1b.realm5.10gen.me:27017"
      		}
      	]
      }
      

            Assignee:
            schwerin@mongodb.com Andy Schwerin
            Reporter:
            mark Mark porter
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: