-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Critical - P2
-
None
-
Affects Version/s: 2.1.2
-
Component/s: None
-
None
-
Environment:"version" : "2.1.2-pre-",
"gitVersion" : "d15b244ccabe1d8d0d1e93f2cb8de03d6b1d09d8",
-
ALL
-
None
-
0
-
None
-
None
-
None
-
None
-
None
-
None
mongos process eat all memory (94%)
# cat /proc/12220/status Name: mongos State: S (sleeping) Tgid: 12220 Pid: 12220 PPid: 1 TracerPid: 0 Uid: 113 113 113 113 Gid: 65534 65534 65534 65534 FDSize: 1024 Groups: 120 65534 VmPeak: 79183616 kB VmSize: 78010392 kB VmLck: 0 kB VmHWM: 46019552 kB VmRSS: 46019552 kB VmData: 77986428 kB VmStk: 88 kB VmExe: 5980 kB VmLib: 3344 kB VmPTE: 151204 kB Threads: 82 SigQ: 0/16382 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000000000 SigCgt: 00000001800046e7 CapInh: 0000000000000000 CapPrm: 0000000000000000 CapEff: 0000000000000000 CapBnd: ffffffffffffffff Cpus_allowed: ffffff Cpus_allowed_list: 0-23 Mems_allowed: 00000000,00000003 Mems_allowed_list: 0-1 voluntary_ctxt_switches: 32887795 nonvoluntary_ctxt_switches: 1434
# ps u 12220 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND mongodb 12220 2.6 92.8 78010920 46027844 ? Sl Jul05 147:46 /usr/bin/mongos --config /etc/mongos.conf --pidfilepath /work/mongo/mongos.pid
While on docs.mongodb.org I read that it a process that don't need many memory
Open files
# ls -l /proc/12220/fd | fgrep -v socket total 0 lrwx------ 1 mongodb nogroup 64 Jul 9 08:24 0 -> /dev/null l-wx------ 1 mongodb nogroup 64 Jul 9 08:24 1 -> /var/log/mongodb/mongos.log l-wx------ 1 mongodb nogroup 64 Jul 9 08:24 2 -> /var/log/mongodb/mongos.log lr-x------ 1 mongodb nogroup 64 Jul 9 08:24 3 -> /dev/urandom
Sockets:
# ls -l /proc/12220/fd | fgrep socket | wc -l 318
"mongod" instance is down because of this, that locate at this host too
[1209528.992738] active_anon:11272433 inactive_anon:867823 isolated_anon:0 [1209528.992739] active_file:465 inactive_file:607 isolated_file:96 [1209528.992740] unevictable:0 dirty:8 writeback:0 unstable:0 [1209528.992741] free:35619 slab_reclaimable:11632 slab_unreclaimable:46570 [1209528.992741] mapped:2193 shmem:1089 pagetables:103091 bounce:0 [1209528.992743] Node 0 DMA free:15736kB min:8kB low:8kB high:12kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15152kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [1209528.992751] lowmem_reserve[]: 0 2991 24201 24201 [1209528.992753] Node 0 DMA32 free:90612kB min:1736kB low:2168kB high:2604kB active_anon:1954904kB inactive_anon:498024kB active_file:428kB inactive_file:508kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:3063520kB mlocked:0kB dirty:4kB writeback:0kB mapped:2084kB shmem:56kB slab_reclaimable:7820kB slab_unreclaimable:93904kB kernel_stack:5368kB pagetables:21756kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [1209528.992762] lowmem_reserve[]: 0 0 21210 21210 [1209528.992764] Node 0 Normal free:14140kB min:12332kB low:15412kB high:18496kB active_anon:20224444kB inactive_anon:1445744kB active_file:476kB inactive_file:908kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:21719040kB mlocked:0kB dirty:0kB writeback:0kB mapped:2400kB shmem:1668kB slab_reclaimable:20684kB slab_unreclaimable:36336kB kernel_stack:3416kB pagetables:200632kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [1209528.992773] lowmem_reserve[]: 0 0 0 0 [1209528.992775] Node 1 Normal free:22748kB min:14092kB low:17612kB high:21136kB active_anon:22910384kB inactive_anon:1527524kB active_file:956kB inactive_file:1012kB unevictable:0kB isolated(anon):0kB isolated(file):256kB present:24821760kB mlocked:0kB dirty:28kB writeback:0kB mapped:4288kB shmem:2632kB slab_reclaimable:18024kB slab_unreclaimable:56040kB kernel_stack:4688kB pagetables:189976kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no [1209528.992783] lowmem_reserve[]: 0 0 0 0 [1209528.992785] Node 0 DMA: 0*4kB 1*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15736kB [1209528.992792] Node 0 DMA32: 14434*4kB 4164*8kB 34*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 91688kB [1209528.992798] Node 0 Normal: 3459*4kB 6*8kB 0*16kB 2*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 14140kB [1209528.992804] Node 1 Normal: 5917*4kB 47*8kB 2*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 24108kB [1209528.992810] 17720 total pagecache pages [1209528.992811] 14833 pages in swap cache [1209528.992813] Swap cache stats: add 8503273, delete 8488440, find 2030254/2098475 [1209528.992814] Free swap = 0kB [1209528.992815] Total swap = 31249400kB [1209529.149310] 12582896 pages RAM [1209529.149312] 193293 pages reserved [1209529.149314] 346058 pages shared [1209529.149315] 12349055 pages non-shared [1209529.149318] Out of memory: kill process 9308 (mongod) score 459992 or a child [1209529.149364] Killed process 9308 (mongod)
Swap is also already full
# free -m
total used free shared buffers cached
Mem: 48396 48074 322 0 15 2126
-/+ buffers/cache: 45932 2464
Swap: 30516 30512 4
And as shows 'free -m', cached or buffers are very small.
So maybe "mongos" have memory leak
- is duplicated by
-
-
- Closed
-