Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32418

Due to a lag, query is very slow in the config server

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.4.10
    • Component/s: Performance, Sharding
    • Labels:
      None
    • ALL

      System environment
      System: CentOS release 6.6
      Kernel: 2.6.32-504.el6.x86_64
      Memory: 256G
      Disk: SSD-PCIE
      File System: xfs
      mongo version: 3.4.10

      The config server has three members in my cluster, one of members that is secondary shows some slow queries in logs file. When I executed sh.disableAutoSplit(), there was more slow queries on this secondary's log. And the ops of all members have risen to 2k/s.

      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4490] command config.settings command: find { find: "settings", filter: { _id: "autosplit" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:391 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4997ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4585] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4997ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4602] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4989ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4583] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4989ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4580] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4994ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4545] command config.settings command: find { find: "settings", filter: { _id: "autosplit" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:391 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4997ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4600] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4963ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4451] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4988ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4572] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4976ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4387] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4908ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4578] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4974ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4607] command config.settings command: find { find: "settings", filter: { _id: "autosplit" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:391 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4990ms
      

        1. config.log.rar
          113.79 MB
        2. config.log.tgz
          105 kB
        3. diagnostic.data.tgz
          16.34 MB
        4. ftdc.png
          ftdc.png
          53 kB
        5. lag.png
          lag.png
          132 kB
        6. log-20180223-16.rar
          75 kB

            Assignee:
            dmitry.agranat@mongodb.com Dmitry Agranat
            Reporter:
            ufozhangliang@163.com liangzhang
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: