We have one primary and two secondary replica sets for our production env and the primary replica set is crashed every minute when I bring it to PRIMARY state and when I check the status of Mongod service it shows as failed: 

mongod.service - SYSV: Mongo is a scalable, document-oriented database.

   Loaded: loaded (/etc/rc.d/init.d/mongod)

   Active: failed (Result: exit-code) since Wed 2018-05-02 21:56:03 UTC; 9min ago

  Process: 27285 ExecStart=/etc/rc.d/init.d/mongod start (code=exited, status=1/FAILURE)

production mongod[27285]: Error starting mongod. /var/run/mongodb/mongod.pid exists.

production systemd[1]: mongod.service: control process exited, code=exited status=1

production systemd[1]: Failed to start SYSV: Mongo is a scalable, document-oriented database..

production systemd[1]: Unit mongod.service entered failed state.

 

Tried restarting the service several times and still unable to get the Primary replica set to primary state. As followed from other blogs, I've tried deleting the /tmp/mongodb-27017.sock file several times and restarted the service with our .congf file.

I have the Logs generated in /var/log/mongodb/mongod.log 

*2017-11-25T09:01:33.834+0000 I CONTROL  [main] ***** SERVER RESTARTED ******

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] MongoDB starting : pid=16025 port=27017 dbpath=/var/lib/mongo 64-bit host=new-mittens-production

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] db version v3.2.12

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] git version: ef3e1bc78e997f0d9f22f45aeb1d8e3b6ac14a14

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] allocator: tcmalloc

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] modules: none

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] build environment:

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten]     distmod: rhel70

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten]     distarch: x86_64

2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten]     target_arch: x86_64

*2017-11-25T09:01:33.846+0000 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net:

{ bindIp: "0.0.0.0", port: 27017 }

, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, storage: { dbPath: "/var/lib/mongo", journal:

{ enabled: true }

}, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }*

2017-11-25T09:01:33.877+0000 E NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock errno:1 Operation not permitted

2017-11-25T09:01:33.877+0000 I -        [initandlisten] Fatal Assertion 28578

2017-11-25T09:01:33.877+0000 I -        [initandlisten]

 ** 

***aborting after fassert() failure

 

Any suggestions or help is appreciated.