Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29206

MongoS [NetworkInterfaceASIO-TaskExecutorPool-0-0] ExceededTimeLimit: Operation timed out

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 3.2.12
    • Component/s: Networking
    • Labels:
      None
    • Fully Compatible
    • ALL

      We continue to receive the below errors on our mongos logs; which causes connection issues to our mongo cluster. This happens throughout the day every day.

      We have 4 mongos nodes. About to have 8 to see if that helps. All 4 will see the issue.

      mongos.log:  2017-05-15T14:30:16.501+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r4-data-001.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:16.501+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r11-data-001.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:16.501+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r9-data-003.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:16.501+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r9-data-003.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:16.534+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r6-data-003.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:16.534+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r6-data-003.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:16.534+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r8-data-003.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out
mongos.log:  2017-05-15T14:30:18.442+0000 I ASIO     [NetworkInterfaceASIO-TaskExecutorPool-0-0] Failed to connect to c0-r8-data-003.mongo-euc-1.com:27017 - ExceededTimeLimit: Operation timed out

      Below is our configuration file:

      ---
net:
  port: '27017'
  bindIp: 0.0.0.0
systemLog:
  path: "/var/log/mongodb/mongos.log"
  logAppend: true
  destination: file
sharding:
  autoSplit: true
  configDB: c0-config-001.mongo-euc-1.com:27017,c0-config-002.mongo-euc-1.com:27017,c0-config-003.mongo-euc-1.com:27017
  chunkSize: 64
setParameter:
  cursorTimeoutMillis: '1800000'
security:
  keyFile: "*********"

      We have tried a variety of options none have helped:

      setParameter:
  cursorTimeoutMillis: '1800000'
  ShardingTaskExecutorPoolHostTimeoutMS: '300000000'
  ShardingTaskExecutorPoolMinSize: '50'

      setParameter:
  cursorTimeoutMillis: '1800000'
  ShardingTaskExecutorPoolRefreshTimeoutMS: 60000
  ShardingTaskExecutorPoolHostTimeoutMS: 300000000
  ShardingTaskExecutorPoolMinSize: 125

      When using any of the config options above; we see the errors occur pretty quickly.

      "opcounters" : {
        "insert" : 5369,
        "query" : 431092,
        "update" : 283293,
        "delete" : 5692,
        "getmore" : 88,
        "command" : 760326
    },

            Assignee:
            mira.carey@mongodb.com Mira Carey
            Reporter:
            wegerer Wayne Egerer
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: