Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-9415

MongoDB only uses small amount of RAM

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker - P1 Blocker - P1
    • None
    • None
    • Performance
    • None
    • 4-CPU 16GB RAM CenOS 6.2 VM
    • ALL

    Description

      I've a single Mongo instance running on a 4-CPU 16GB RAM CenOS 6.2 VM. It's running very slowly mostly of the time. Here is the mongostat output:

      insert  query update delete getmore command flushes mapped  vsize    res        faults       locked db         idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time
          *0     62     12        *0           0             9|0               0          82g   174g       2.61g     10       gowmain:13.2%          0            52|0     1|5      12k    64k     315  PRI   14:41:27
          *0     90      26       *0           0            21|0               0         82g   174g       2.61g     12       gowmain:56.0%          0            63|0     0|1      23k   145k    315  PRI   14:41:29
          *0    157     38       *0           0            19|0              1         82g   174g        2.6g        44       gowmain:56.4%         0            61|0     1|0      36k   330k    315  PRI   14:41:31
          *0    372     95       *0           0            62|0              0         82g   174g       2.61g      141      gowmain:38.1%         0            53|0     1|0     208k    1m    315  PRI   14:41:32
          *0    303     58       *0           0            31|0              0         82g   174g       2.62g       43       gowmain:33.5%         0            52|0     1|0      68k    429k    315  PRI   14:41:33
          *0    277     43       *0           0            17|0              0         82g   174g       2.61g       53       gowmain:21.8%          0            55|0     0|1    103k   1m      315  PRI   14:41:34
          *0    176     24       *0           0             9|0               0         82g   174g       2.61g       17       gowmain:63.8%          0            58|0     1|0     48k     581k   315  PRI   14:41:35
          *0    506     79       *0           0            38|0              0         82g   174g        2.6g        91       gowmain:47.6%          0            62|0     1|0     143k   852k   316  PRI   14:41:37
          *0    236     39       *0           0            15|0              0         82g   174g       2.61g       95       gowmain:21.2%          0            55|1     0|1     86k    1m      316  PRI   14:41:38
          *0    195     41       *0           0            15|0              0         82g   174g        2.6g       86       gowmain:28.3%          0            50|1     2|0       89k   1m      316  PRI   14:41:40
       

      The command line options are: --rest --fork --master --oplogSize=22770 --port 27017 --dbpath /data/mongodb/dbdata/ --logpath /data/logs/mongodb.log --auth

      Here is the top output:

      top - 15:05:58 up 18 days, 21:03,  2 users,  load average: 2.37, 2.79, 2.83
      Tasks: 121 total,   1 running, 120 sleeping,   0 stopped,   0 zombie
      Cpu(s):  0.5%us,  0.5%sy,  0.0%ni, 74.9%id, 24.1%wa,  0.0%hi,  0.0%si,  0.0%st
      Mem:  16139308k total, 15988616k used,   150692k free,    11336k buffers
      Swap:  4194296k total,      536k used,  4193760k free, 14952988k cached
       
        PID   USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   COMMAND
      21253  root        20    0   173g  2.6g   2.4g  S    7.0        17.0   468:05.07  mongod

      The weird thing is MongoDB consistently only consumes only 2.6G of RAM! However, the output from 'top' shows all 16GB RAM is used up.

      A couple of other observations:

      • The CPU load from 'top' is usually between 2.00 to 3.00
      • The lock% range between 40 to 80.
      • When the read queue goes up to 150 to 200, the system is unusable.
      • The number of faults is not too bad, but iostat shows 100% IO itilization on the mongo partition. Also, sometimes the await time in iostat is more than 1000ms!!!

      The DB has one large collection having about 50 million documents; all other collections are small. The DB serves about 2000 users frontended by Tomcat (which has almost no load).

      I cannot figure out why it uses less than 3GB of RAM while the system has 16GB.
      Are we doing something obviously wrong here?

      Attachments

        Activity

          People

            adamc Adam Comerford
            liujeff jeff liu
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: