on all of our 80 appserver, 1 mongos-Daemon is installed. Very frequently, the mongos-Daemon stops working because of "too many open files", which is also spamming the mongos-logfiles. We increased the openfiles limit to 30000 and this fixed the problem for a short time. But soon the problem was there again, and I discovered the real source. Somehow the mongos-Daemons seems to create some kind of idleing zombie-connections. When I do a

lsof | grep 'mongos'

This is the output:
......thousands of lines .....
mongos 22843 root 8523u sock 0,5 2863975141 can't identify protocol
mongos 22843 root 8524u sock 0,5 2863975143 can't identify protocol
mongos 22843 root 8525u sock 0,5 2863975162 can't identify protocol
mongos 22843 root 8526u sock 0,5 2863975219 can't identify protocol
......thousands of lines ......

These zombie-connections fill up all the available filedescriptors.

For me it looks like a bug in mongos , or the mongodb-java connector (or both).