I have a mongod master process with 7 slaves process on same machine, everything goes fine before I started using killall -SIGUSR1 mongod, every 2 days one of my server's master mongod will stop working. but slaves are fine, I can connect to the listening port using nc. But I can't connect to the mongod using mongo shell ,it hangs. And I can not kill the mongod master process using kill -2 <pid>, All I have to do is using kill -9 and repair my db, It is costy and unpractical,
I checked the log everything is fine, but the new log is empty, It seems mongod didn't rotate the log successfully. It caused the filesystem deadlock??