[SERVER-11311] SNMP: multiple hosts on same port collide snmpwalk namespace Created: 22/Oct/13  Updated: 10/Dec/14  Resolved: 24/Oct/13

Status: Closed
Project: Core Server
Component/s: Diagnostics
Affects Version/s: 2.5.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: John Morales Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: 26qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:
  • EC2, Amazon Linux, m1.medium
  • PSS replica set
  • All mongod instances running as SNMP subagents, reporting to snmpd functioning as SNMP master

Issue Links:
Related
Operating System: ALL
Steps To Reproduce:

0.) Install standard pre-req packages for SNMP.
1.) Edit /etc/snmp/snmpd.conf and /etc/snmp/mongod.conf config files for subagent connection to snmpd through TCP:1705.
2.) Start up snmpd daemon.
3.) Launch a PSS replica set, with each running with --snmp-subagent on a distinct host and same port.
3.) Run snmpwalk against the agentaddress defined for your running snmpd daemon
snmpwalk -m MONGO-MIB -v 2c -c mongodb ec2-foo-bar.amazonaws.com:1161 1.3.6.1.4.1.34601

Actual command-line flags used for each host:
PRI: ./bin/mongod --replSet snmpd --snmp-subagent --port 27017 --fork --dbpath data/snmpd-0/ --logpath logs/snmpd-0/mongod.log --smallfiles --nohttpinterface --oplogSize=50

SEC: ./bin/mongod --replSet snmpd --snmp-subagent --port 27017 --fork --dbpath data/snmpd-2/ --logpath logs/snmpd-2/mongod.log --smallfiles --nohttpinterface --oplogSize=50

SEC: ./bin/mongod --replSet snmpd --snmp-subagent --port 27017 --fork --dbpath data/snmpd-4/ --logpath logs/snmpd-4/mongod.log --smallfiles --nohttpinterface --oplogSize=50

Participants:

 Description   

If a replica set of SNMP subagents are all running on the same port on different hosts, then SNMP responses are ambiguous - only one "27017" entry is returned, and for which member appears to be arbitrary.

...
MONGO-MIB::globalOpInsert."27017" = Counter32: 1
MONGO-MIB::globalOpQuery."27017" = Counter32: 60
MONGO-MIB::globalOpUpdate."27017" = Counter32: 1
MONGO-MIB::globalOpDelete."27017" = Counter32: 0
...
MONGO-MIB::replSetName."27017" = STRING: "snmpd"
MONGO-MIB::replSetVersion."27017" = INTEGER: 3
MONGO-MIB::replIsMaster."27017" = INTEGER: 1
MONGO-MIB::replIsSecondary."27017" = INTEGER: 0
MONGO-MIB::replPrimary."27017" = STRING: "hostA:27017"
MONGO-MIB::replMe."27017" = STRING: "hostA:27017"

Furthermore, if the replica set member identified by the MONGO-MIB::replMe field is brought down (in the above example, say hostA:27017 becomes unavailable), then the snmpwalk becomes unresponsive - does not return any results from any of the other members.

This behavior would seem to limit the utility of the subagent model given a "typical" replica set deployment. However of course a workaround exists to always utilize a distinct set of ports when using SNMP.



 Comments   
Comment by Eric Milkie [ 24/Oct/13 ]

After some internal discussion, we decided that our recommendation is not to use subagents over the network. mongod as a subagent should only connect to a local snmp master. Therefore, the port number should always be a way to uniquely identify a mongod.

Generated at Thu Feb 08 03:25:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.