Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-47650

mongostat queue qrw 600+ arw fixed 128

    • Type: Icon: Bug Bug
    • Resolution: Community Answered
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.0.16
    • Component/s: Diagnostics
    • Labels:
      None
    • Fully Compatible
    • ALL

      mongo cluster has two shard using hash policy.i test on mongo 4.0.16.

       

      mongostat on shard 1 show qrw+arw less queue.but mongostat on shard arw fixed 128 queue,qarw 600+,mongotop for shard collection,400-600s write,but in mongo log,sql execute time is 100-500ms.so my cluster ops is 5000/s,my machine is 56c +512g +12tb ssd 10 with sas interface.

      mongo cluster:

      p  s a (shard1)

      a  s  p(shard2)

      one machine:shard 1 primary + shard2 a

      one mchine:shard 1 second + shard2 second

      one machine:shard2 primary + shard 1 a

      [shard1]

      insert query update delete getmore command dirty used flushes vsize res qrw arw net_in net_out conn set repl time
       1298 1296 1295 *0 497 598|0 4.0% 80.0% 0 235G 224G 0|1 1|0 5.23m 5.04m 70 myShard_1 PRI Apr 18 17:42:01.888
       1306 1308 1309 *0 499 598|0 4.0% 80.0% 0 235G 224G 0|0 1|0 5.28m 5.02m 70 myShard_1 PRI Apr 18 17:42:02.888
       1264 1263 1262 *0 522 617|0 4.0% 80.0% 0 235G 224G 0|0 1|0 5.15m 4.92m 70 myShard_1 PRI Apr 18 17:42:03.887
       1383 1382 1382 *0 507 614|0 4.1% 80.0% 0 235G 224G 0|0 1|0 5.56m 5.29m 70 myShard_1 PRI Apr 18 17:42:04.887
       1316 1316 1318 *0 517 623|0 4.1% 80.0% 0 235G 224G 0|0 2|0 5.33m 5.09m 70 myShard_1 PRI Apr 18 17:42:05.887
      

      [shard2]

      insert query update delete getmore command dirty used flushes vsize res qrw arw net_in net_out conn set repl time
       1169 1188 1189 *0 0 7|0 15.5% 68.4% 0 242G 206G 0|665 5|128 4.18m 2.28m 926 myShard_2 PRI Apr 18 17:29:24.155
       1075 1074 1072 *0 0 4|0 15.5% 68.4% 0 242G 206G 0|658 3|128 3.84m 2.09m 926 myShard_2 PRI Apr 18 17:29:25.153
       1115 1111 1112 *0 0 6|0 15.5% 68.4% 0 242G 206G 0|666 3|128 3.92m 2.18m 926 myShard_2 PRI Apr 18 17:29:26.155
       1198 1207 1206 *0 0 5|0 15.5% 68.4% 0 242G 206G 0|662 4|128 4.31m 2.33m 925 myShard_2 PRI Apr 18 17:29:27.153
       1182 1173 1173 *0 0 2|0 15.5% 68.4% 0 242G 206G 0|661 5|128 4.15m 2.26m 926 myShard_2 PRI Apr 18 17:29:28.153
       1212 1219 1218 *0 0 7|0 15.5% 68.4% 0 242G 206G 0|662 4|128 4.34m 2.35m 926 myShard_2 PRI Apr 18 17:29:29.154
       1174 1190 1188 *0 0 5|0 15.5% 68.4% 0 242G 207G 0|658 5|128 4.23m 2.29m 926 myShard_2 PRI Apr 18 17:29:30.154
       1247 1252 1251 *0 0 13|0 15.5% 68.4% 0 242G 207G 0|652 5|128 4.43m 2.45m 926 myShard_2 PRI Apr 18 17:29:31.155
       1234 1229 1221 *0 0 45|0 15.5% 68.4% 0 242G 207G 0|659 5|128 4.35m 2.58m 926 myShard_2 PRI Apr 18 17:29:32.154
       1214 1221 1221 *0 0 24|0 15.5% 68.4% 0 242G 207G 0|660 5|128 4.31m 2.42m 926 myShard_2 PRI Apr 18 17:29:33.154
      insert query update delete getmore command dirty used flushes vsize res qrw arw net_in net_out conn set repl time
       1236 1225 1224 *0 0 15|0 15.5% 68.4% 0 242G 207G 0|657 5|128 4.36m 2.37m 920 myShard_2 PRI Apr 18 17:29:34.154
       1197 1186 1188 *0 0 13|0 15.5% 68.4% 0 242G 207G 0|661 5|128 4.25m 2.31m 920 myShard_2 PRI Apr 18 17:29:35.154
       1259 1297 1296 *0 0 16|0 15.5% 68.4% 0 242G 207G 0|657 6|128 4.55m 2.54m 920 myShard_2 PRI Apr 18 17:29:36.155
      

        1. mongstat.jpg
          237 kB
          jing xu
        2. mongtop.jpg
          93 kB
          jing xu
        3. myShard_3.png
          133 kB
          jing xu
        4. nopswithmoredirty.jpg
          216 kB
          jing xu
        5. ops.jpg
          55 kB
          jing xu
        6. quque1.jpg
          73 kB
          jing xu
        7. readconcern.png
          40 kB
          jing xu
        8. shard1mongstat.jpg
          227 kB
          jing xu
        9. shard2mongo.tar.gz
          47.85 MB
          jing xu
        10. shard2topandstat.jpg
          114 kB
          jing xu
        11. shardops.jpg
          72 kB
          jing xu
        12. shardshowlog.jpg
          37 kB
          jing xu
        13. srvdb1.yto.cloud.png
          139 kB
          Dmitry Agranat
        14. upload ftdc.png
          8 kB
          jing xu

            Assignee:
            dmitry.agranat@mongodb.com Dmitry Agranat
            Reporter:
            601290552@qq.com jing xu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: