Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1077

can't start new thread

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Environment:
      centos 6 x64, mongodb 2.6.3 x64, python 2.7 x64, pymongo 3.2

      Description

      I use supervisor to start 200 python worker, each worker would import a MongoClient, I had tried `maxPoolSize=None` and `maxPoolSize=3000`, just same error.

      I have check mongodb connection:

      > db.serverStatus().connections
      { "current" : 1621, "available" : 18379, "totalCreated" : NumberLong(2512) }
      

      It's quite roomy.

      My vps conf:

      [root@server1 ~]# cat /proc/sys/kernel/threads-max
      256653
      [root@server1 ~]# cat /proc/sys/kernel/pid_max
      32768
      [root@server1 ~]# ulimit
      unlimited
      

      So I think is a pymongo problem.

      in <module>
          from udbswp.handler.api import spider_api
        File "/mnt/apps/workspace/udb-spider/udbswp/handler/api.py", line 169, in <module>
          spider_api = Api()
        File "/mnt/apps/workspace/udb-spider/udbswp/handler/api.py", line 43, in __init__
          self.db = connect_database(API_DB_URL)
        File "/mnt/apps/workspace/udb-spider/pyspider/pyspider/webui/database/__init__.py", line 23, in connect_database
          return SpiderSettingDB(url)
        File "/mnt/apps/workspace/udb-spider/pyspider/pyspider/webui/database/mongodb.py", line 95, in __init__
          self.conn = MongoClient(url, maxPoolSize=3000)
        File "/usr/local/python27/lib/python2.7/site-packages/pymongo/mongo_client.py", line 398, in __init__
          executor.open()
        File "/usr/local/python27/lib/python2.7/site-packages/pymongo/periodic_executor.py", line 68, in open
          thread.start()
        File "/usr/local/python27/lib/python2.7/threading.py", line 745, in start
          _start_new_thread(self.__bootstrap, ())
      error: can't start new thread
      

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: