[SERVER-8998] SNMP: silent server crash when permission denied for AgentXSocket Created: 15/Mar/13  Updated: 11/Jul/16  Resolved: 02/Dec/13

Status: Closed
Project: Core Server
Component/s: Diagnostics, Stability
Affects Version/s: 2.2.3
Fix Version/s: 2.5.5

Type: Bug Priority: Major - P3
Reporter: John Morales Assignee: Andy Schwerin
Resolution: Done Votes: 0
Labels: snmp, subscriber
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Subscriber / Enterprise edition, EC2 amzn Linux


Operating System: ALL
Steps To Reproduce:

E.g. /etc/snmp/mongod.conf:

...
#only bind to localhost port 1161
agentaddress 127.0.0.1:1161
 
master agentx #Added
AgentXSocket tcp:localhost:705 #Added, privileged port
...

Now attempt to start mongod as any non-root user. Example command used:

$ ./bin/mongod -vvvvvvvvvvvv --snmp-master --port 3002 --fork --dbpath data --logpath logs/mongod.master.log --nojournal --smallfiles --nohttpinterface

Corresponding mongod log (complete):

Fri Mar 15 22:26:56 versionCmpTest passed
Fri Mar 15 22:26:56 versionArrayTest passed
Fri Mar 15 22:26:56 shardKeyTest passed
Fri Mar 15 22:26:56 isInRangeTest passed
Fri Mar 15 22:26:56 shardObjTest passed
Fri Mar 15 22:26:56 Matcher::matches() { abcd: 3.1, abcdef: "123456789" }
Fri Mar 15 22:26:56 Matcher::matches() { abcd: 3.1, abcdef: "123456789" }
Fri Mar 15 22:26:56 Matcher::matches() { abcd: 3.1, abcdef: "123456789" }
Fri Mar 15 22:26:56 Matcher::matches() { abcdef: "z23456789" }
Fri Mar 15 22:26:56 Matcher::matches() { abcd: 3.1, abcdef: "123456789" }
Fri Mar 15 22:26:56 Matcher::matches() { abcdef: "z23456789" }
Fri Mar 15 22:26:56 BackgroundJob starting: DataFileSync
Fri Mar 15 22:26:56 [initandlisten] MongoDB starting : pid=19032 port=3002 dbpath=/home/ec2-user/mongo-subscriber/mongodb-linux-x86_64-subscription-amzn64-2.2.3/data 64-bit host=ip-10-151-20-245
Fri Mar 15 22:26:56 [initandlisten] db version v2.2.3, pdfile version 4.5
Fri Mar 15 22:26:56 [initandlisten] git version: f570771a5d8a3846eb7586eaffcf4c2f4a96bf08 modules: subscription
Fri Mar 15 22:26:56 [initandlisten] build info: Linux bs-e-amzn64 3.2.12-3.2.4.amzn1.x86_64 #1 SMP Thu Mar 22 08:00:08 UTC 2012 x86_64 BOOST_LIB_VERSION=1_49
Fri Mar 15 22:26:56 [initandlisten] options: { dbpath: "data", fork: true, logpath: "logs/mongod.master.log", nohttpinterface: true, nojournal: true, port: 3002, smallfiles: true, snmp-master: true, vvvvvvvvvvvv: true }
Fri Mar 15 22:26:56 [initandlisten] Unable to check for journal files due to: boost::filesystem::basic_directory_iterator constructor: No such file or directory: "/home/ec2-user/mongo-subscriber/mongodb-linux-x86_64-subscription-amzn64-2.2.3/data/journal"
Fri Mar 15 22:26:56 [initandlisten] flushing directory /home/ec2-user/mongo-subscriber/mongodb-linux-x86_64-subscription-amzn64-2.2.3/data
Fri Mar 15 22:26:56 [initandlisten] opening db:  local
Fri Mar 15 22:26:56 [initandlisten] enter repairDatabases (to check pdfile version #)
Fri Mar 15 22:26:56 [initandlisten] done repairDatabases
Fri Mar 15 22:26:56 [initandlisten] fd limit hard:4096 soft:1024 max conn: 819
Fri Mar 15 22:26:56 [initandlisten] waiting for connections on port 3002
Fri Mar 15 22:26:56 BackgroundJob starting: TTLMonitor
Fri Mar 15 22:26:56 BackgroundJob starting: PeriodicTask::Runner
Fri Mar 15 22:26:56 BackgroundJob starting: ClientCursorMonitor
Fri Mar 15 22:26:56 BackgroundJob starting: snapshot
Fri Mar 15 22:26:56 BackgroundJob starting: SNMPAgent
Fri Mar 15 22:26:56 [SNMPAgent] SNMPAgent num things: 10

Participants:

 Description   

If AgentX (sub-agent) is configured in /etc/snmp/mongod.conf, but the user the server's running as doesn't have access to the AgentXSocket definition, the server crashes (w/o stack trace or log message) and is left in "needs repair" state.

Note: this can be the case because either:
a.) no FS permissions to unix socket file, or
b.) non-root user trying to listen on privileged (< 1024) port.



 Comments   
Comment by Githook User [ 02/Dec/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-8998 Configure net-snmp library to log through the MongoDB logging system.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/9a3e5d4e50c314fbc262bca51bb83a61381addf6

Comment by Githook User [ 02/Dec/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-8998 Register an atexit handler to call _exit if a library calls ::exit().

This prevents static destructors from running if the program exits via ::exit()
or by returning from main(), until such time as we fix normal exit behavior.
Branch: master
https://github.com/mongodb/mongo/commit/5ea897e30b45447d55289e33f636da3017b1e8db

Comment by Eric Milkie [ 01/Nov/13 ]

Indeed, I found the call to exit(0) in the agentx code when it fails to open the socket.
However, it does attempt to log a message about the problem. I think we should change the default .conf files to make better use of logOption, so that the log messages go to the terminal by default with a comment saying the user should change this as they see fit. From what I can tell, by default the log messages go into /var/log somewhere I think which is unhelpful in the above situation.

Comment by Matt Dannenberg [ 31/Oct/13 ]

It seems like when SNMPAgent calls init_master_agent() (a function from snmp.h) that function then calls exit() rather than _exit(). I looked around online for a while for a solution, but didn't find anything helpful. In their release notes ( http://www.net-snmp.org/docs/NEWS.html ) for version 4.1.2 they "fix init_master_agent calling exit" which I guess that means fixed it to call exit()....

Generated at Thu Feb 08 03:19:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.