Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major - P3
-
Resolution: Done
-
Affects Version/s: 2.6.10
-
Fix Version/s: None
-
Component/s: Networking
-
Labels:None
-
Operating System:ALL
Description
There are some logs as below show up in mongod start progress:
2015-06-13T04:08:01.238+0800 [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
2015-06-13T04:08:01.238+0800 [initandlisten] can't create new thread, closing connection
|
...
|
2015-06-13T04:08:21.247+0800 [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
2015-06-13T04:08:21.248+0800 [initandlisten] can't create new thread, closing connection
|
And the limits and threads count of mongod instance that I collected as below:
2015-06-13_04:08:03
|
limits:
|
[mongod #28309 -- limits]
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 655350 655350 processes
|
Max open files 655350 655350 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 513987 513987 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
threads_count:
|
[mongod #28309 -- limits]
|
97
|
|
|
limits:
|
[mongod #28309 -- limits]
|
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 655350 655350 processes
|
Max open files 655350 655350 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 513987 513987 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
threads_count:
|
[mongod #28309 -- limits]
|
97
|
I'm confused that the threads of mongod instance is less than 1000, and less than 655350, why log show that? Is it a bug?
ps. This (RS) member have more than 10k databases and 700k collections( with indexes).
ps. I use grab-mongodb-limits-and-threads.sh to get limits and threads count info above.
|
grab-mongodb-limits-and-threads.sh |
#!/bin/bash
|
|
|
return-limits(){
|
|
|
for process in $@; do
|
process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2`
|
|
|
if [ -z $@ ]; then
|
echo "[no $process running]"
|
else
|
for pid in $process_pids; do
|
echo "[$process #$pid -- limits]"
|
cat /proc/$pid/limits
|
done
|
fi
|
done
|
}
|
|
|
return-tasks(){
|
|
|
for process in $@; do
|
process_pids=`ps -C $process -o pid --no-headers | cut -d " " -f 2`
|
|
|
if [ -z $@ ]; then
|
echo "[no $process running]"
|
else
|
for pid in $process_pids; do
|
echo "[$process #$pid -- limits]"
|
ls /proc/$pid/task | wc -l
|
done
|
fi
|
done
|
}
|
|
|
date +%F_%T
|
echo "limits:"
|
return-limits mongod
|
echo "threads_count:"
|
return-tasks mongod
|