Details
-
Bug
-
Status: Closed
-
Critical - P2
-
Resolution: Duplicate
-
2.1.2
-
None
-
None
-
None
-
"version" : "2.1.2-pre-",
"gitVersion" : "d15b244ccabe1d8d0d1e93f2cb8de03d6b1d09d8",
-
ALL
Description
mongos process eat all memory (94%)
# cat /proc/12220/status
|
Name: mongos
|
State: S (sleeping)
|
Tgid: 12220
|
Pid: 12220
|
PPid: 1
|
TracerPid: 0
|
Uid: 113 113 113 113
|
Gid: 65534 65534 65534 65534
|
FDSize: 1024
|
Groups: 120 65534
|
VmPeak: 79183616 kB
|
VmSize: 78010392 kB
|
VmLck: 0 kB
|
VmHWM: 46019552 kB
|
VmRSS: 46019552 kB
|
VmData: 77986428 kB
|
VmStk: 88 kB
|
VmExe: 5980 kB
|
VmLib: 3344 kB
|
VmPTE: 151204 kB
|
Threads: 82
|
SigQ: 0/16382
|
SigPnd: 0000000000000000
|
ShdPnd: 0000000000000000
|
SigBlk: 0000000000000000
|
SigIgn: 0000000000000000
|
SigCgt: 00000001800046e7
|
CapInh: 0000000000000000
|
CapPrm: 0000000000000000
|
CapEff: 0000000000000000
|
CapBnd: ffffffffffffffff
|
Cpus_allowed: ffffff
|
Cpus_allowed_list: 0-23
|
Mems_allowed: 00000000,00000003
|
Mems_allowed_list: 0-1
|
voluntary_ctxt_switches: 32887795
|
nonvoluntary_ctxt_switches: 1434
|
# ps u 12220
|
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
|
mongodb 12220 2.6 92.8 78010920 46027844 ? Sl Jul05 147:46 /usr/bin/mongos --config /etc/mongos.conf --pidfilepath /work/mongo/mongos.pid
|
While on docs.mongodb.org I read that it a process that don't need many memory
Open files
# ls -l /proc/12220/fd | fgrep -v socket
|
total 0
|
lrwx------ 1 mongodb nogroup 64 Jul 9 08:24 0 -> /dev/null
|
l-wx------ 1 mongodb nogroup 64 Jul 9 08:24 1 -> /var/log/mongodb/mongos.log
|
l-wx------ 1 mongodb nogroup 64 Jul 9 08:24 2 -> /var/log/mongodb/mongos.log
|
lr-x------ 1 mongodb nogroup 64 Jul 9 08:24 3 -> /dev/urandom
|
Sockets:
# ls -l /proc/12220/fd | fgrep socket | wc -l
|
318
|
"mongod" instance is down because of this, that locate at this host too
[1209528.992738] active_anon:11272433 inactive_anon:867823 isolated_anon:0
|
[1209528.992739] active_file:465 inactive_file:607 isolated_file:96
|
[1209528.992740] unevictable:0 dirty:8 writeback:0 unstable:0
|
[1209528.992741] free:35619 slab_reclaimable:11632 slab_unreclaimable:46570
|
[1209528.992741] mapped:2193 shmem:1089 pagetables:103091 bounce:0
|
[1209528.992743] Node 0 DMA free:15736kB min:8kB low:8kB high:12kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15152kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
|
[1209528.992751] lowmem_reserve[]: 0 2991 24201 24201
|
[1209528.992753] Node 0 DMA32 free:90612kB min:1736kB low:2168kB high:2604kB active_anon:1954904kB inactive_anon:498024kB active_file:428kB inactive_file:508kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:3063520kB mlocked:0kB dirty:4kB writeback:0kB mapped:2084kB shmem:56kB slab_reclaimable:7820kB slab_unreclaimable:93904kB kernel_stack:5368kB pagetables:21756kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
|
[1209528.992762] lowmem_reserve[]: 0 0 21210 21210
|
[1209528.992764] Node 0 Normal free:14140kB min:12332kB low:15412kB high:18496kB active_anon:20224444kB inactive_anon:1445744kB active_file:476kB inactive_file:908kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:21719040kB mlocked:0kB dirty:0kB writeback:0kB mapped:2400kB shmem:1668kB slab_reclaimable:20684kB slab_unreclaimable:36336kB kernel_stack:3416kB pagetables:200632kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
|
[1209528.992773] lowmem_reserve[]: 0 0 0 0
|
[1209528.992775] Node 1 Normal free:22748kB min:14092kB low:17612kB high:21136kB active_anon:22910384kB inactive_anon:1527524kB active_file:956kB inactive_file:1012kB unevictable:0kB isolated(anon):0kB isolated(file):256kB present:24821760kB mlocked:0kB dirty:28kB writeback:0kB mapped:4288kB shmem:2632kB slab_reclaimable:18024kB slab_unreclaimable:56040kB kernel_stack:4688kB pagetables:189976kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no
|
[1209528.992783] lowmem_reserve[]: 0 0 0 0
|
[1209528.992785] Node 0 DMA: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15736kB
|
[1209528.992792] Node 0 DMA32: 14434*4kB 4164*8kB 34*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 91688kB
|
[1209528.992798] Node 0 Normal: 3459*4kB 6*8kB 0*16kB 2*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 14140kB
|
[1209528.992804] Node 1 Normal: 5917*4kB 47*8kB 2*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 24108kB
|
[1209528.992810] 17720 total pagecache pages
|
[1209528.992811] 14833 pages in swap cache
|
[1209528.992813] Swap cache stats: add 8503273, delete 8488440, find 2030254/2098475
|
[1209528.992814] Free swap = 0kB
|
[1209528.992815] Total swap = 31249400kB
|
[1209529.149310] 12582896 pages RAM
|
[1209529.149312] 193293 pages reserved
|
[1209529.149314] 346058 pages shared
|
[1209529.149315] 12349055 pages non-shared
|
[1209529.149318] Out of memory: kill process 9308 (mongod) score 459992 or a child
|
[1209529.149364] Killed process 9308 (mongod)
|
Swap is also already full
# free -m
|
total used free shared buffers cached
|
Mem: 48396 48074 322 0 15 2126
|
-/+ buffers/cache: 45932 2464
|
Swap: 30516 30512 4
|
And as shows 'free -m', cached or buffers are very small.
So maybe "mongos" have memory leak
Attachments
Issue Links
- is duplicated by
-
SERVER-6785 Possible mongos memory leak
-
- Closed
-