Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32418

Due to a lag, query is very slow in the config server

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.4.10
    • Component/s: Performance, Sharding
    • Labels:
      None
    • ALL

      System environment
      System: CentOS release 6.6
      Kernel: 2.6.32-504.el6.x86_64
      Memory: 256G
      Disk: SSD-PCIE
      File System: xfs
      mongo version: 3.4.10

      The config server has three members in my cluster, one of members that is secondary shows some slow queries in logs file. When I executed sh.disableAutoSplit(), there was more slow queries on this secondary's log. And the ops of all members have risen to 2k/s.

      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4490] command config.settings command: find { find: "settings", filter: { _id: "autosplit" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:391 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4997ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4585] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4997ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4602] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4989ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4583] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4989ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4580] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4994ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4545] command config.settings command: find { find: "settings", filter: { _id: "autosplit" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:391 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4997ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4600] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4963ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4451] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4988ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4572] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4976ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4387] command config.settings command: find { find: "settings", filter: { _id: "balancer" }, readConcern: { level: "majority", 
      afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturn
      ed:1 reslen:404 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4908ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4578] command config.settings command: find { find: "settings", filter: { _id: "chunksize" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:392 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4974ms
      2017-12-19T20:14:12.749+0800 I COMMAND  [conn4607] command config.settings command: find { find: "settings", filter: { _id: "autosplit" }, readConcern: { level: "majority",
       afterOpTime: { ts: Timestamp 1513685647000|1, t: 9 } }, limit: 1, maxTimeMS: 30000 } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nretur
      ned:1 reslen:391 locks:{ Global: { acquireCount: { r: 2 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } protocol:op_command 4990ms
      

        1. config.log.rar
          113.79 MB
          liangzhang
        2. config.log.tgz
          105 kB
          liangzhang
        3. diagnostic.data.tgz
          16.34 MB
          liangzhang
        4. ftdc.png
          53 kB
          Mark Agarunov
        5. lag.png
          132 kB
          Dmitry Agranat
        6. log-20180223-16.rar
          75 kB
          liangzhang

            Assignee:
            dmitry.agranat@mongodb.com Dmitry Agranat
            Reporter:
            ufozhangliang@163.com liangzhang
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: