-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 2.6.10
-
Component/s: Networking
-
Labels:None
-
ALL
There are some logs as below show up in mongod start progress:
2015-06-13T04:08:01.238+0800 [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable 2015-06-13T04:08:01.238+0800 [initandlisten] can't create new thread, closing connection ... 2015-06-13T04:08:21.247+0800 [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable 2015-06-13T04:08:21.248+0800 [initandlisten] can't create new thread, closing connection
And the limits and threads count of mongod instance that I collected as below:
2015-06-13_04:08:03 limits: [mongod #28309 -- limits] Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 655350 655350 processes Max open files 655350 655350 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 513987 513987 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us threads_count: [mongod #28309 -- limits] 97 limits: [mongod #28309 -- limits] Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 655350 655350 processes Max open files 655350 655350 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 513987 513987 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us threads_count: [mongod #28309 -- limits] 97
I'm confused that the threads of mongod instance is less than 1000, and less than 655350, why log show that? Is it a bug?
ps. This (RS) member have more than 10k databases and 700k collections( with indexes).
ps. I use grab-mongodb-limits-and-threads.sh to get limits and threads count info above.
grab-mongodb-limits-and-threads.sh
#!/bin/bash return-limits(){ for process in $@; do process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2` if [ -z $@ ]; then echo "[no $process running]" else for pid in $process_pids; do echo "[$process #$pid -- limits]" cat /proc/$pid/limits done fi done } return-tasks(){ for process in $@; do process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2` if [ -z $@ ]; then echo "[no $process running]" else for pid in $process_pids; do echo "[$process #$pid -- limits]" ls /proc/$pid/task | wc -l done fi done } date +%F_%T echo "limits:" return-limits mongod echo "threads_count:" return-tasks mongod