Details
-
Bug
-
Resolution: Works as Designed
-
Major - P3
-
None
-
3.2.12
-
None
-
ALL
-
Description
Too many connections consume all file descriptors (or some other resource?) which causes mongod to crash:
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] pthread_create failed: errno:11 Resource temporarily unavailable
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [initandlisten] failed to create thread after accepting new connection, closing connection
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [NetworkInterfaceASIO-BGSync-0] terminate() called. An exception is active; attempting to gather more information
|
Apr 23 17:07:23 mongo5 mongod.27017[14595]: [NetworkInterfaceASIO-BGSync-0] std::exception::what(): Resource temporarily unavailable
|
Actual exception type: std::system_error
|
|
|
0x1315b12 0x1315432 0x1b143c6 0x1b143f3 0x12a5f7a 0x12a64d8 0x10d86ae 0x10d8f8e 0x10d96d6 0x10c100d 0x10c1cca 0x10c22c8 0x10bf9b0 0x1093aa0 0x10a282c 0x10a2ce8 0x13334e1 0x1333701 0x10b92e9 0x1b5c600 0x7fc272ae8064 0x7fc27281d62d
|
----- BEGIN BACKTRACE -----
|
{"backtrace":[{"b":"400000","o":"F15B12","s":"_ZN5mongo15printStackTraceERSo"},{"b":"400000","o":"F15432"},{"b":"400000","o":"17143C6","s":"_ZN10__cxxabiv111__terminateEPFvvE"},{"b":"400000","o":"17143F3"},{"b":"400000","o":"EA5F7A","s":"_ZN5mongo10ThreadPool25_startWorkerThread_inlockEv"},{"b":"400000","o":"EA64D8","s":"_ZN5mongo10ThreadPool8scheduleESt8functionIFvvEE"},{"b":"400000","o":"CD86AE","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPSt4listISt10shared_ptrINS1_13CallbackStateEESaIS5_EERKSt14_List_iteratorIS5_ESC_St11unique_lockISt5mutexE"},{"b":"400000","o":"CD8F8E","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPSt4listISt10shared_ptrINS1_13CallbackStateEESaIS5_EERKSt14_List_iteratorIS5_ESt11unique_lockISt5mutexE"},{"b":"400000","o":"CD96D6"},{"b":"400000","o":"CC100D","s":"_ZN5mongo8executor20NetworkInterfaceASIO18_completeOperationEPNS1_7AsyncOpERKNS_10StatusWithINS0_21RemoteCommandResponseEEE"},{"b":"400000","o":"CC1CCA","s":"_ZN5mongo8executor20NetworkInterfaceASIO20_completedOpCallbackEPNS1_7AsyncOpE"},{"b":"400000","o":"CC22C8"},{"b":"400000","o":"CBF9B0"},{"b":"400000","o":"C93AA0","s":"_ZN4asio6detail14strand_service8dispatchINS0_7binder2IRSt8functionIFvSt10error_codemEES5_mEEEEvRPNS1_11strand_implERT_"},{"b":"400000","o":"CA282C","s":"_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder2INS0_7read_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS8_EEEENS_17mutable_buffers_1ENS0_14transfer_all_tENS0_15wrapped_handlerINS_10io_service6strandESt8functionIFvSt10error_codemEENS0_26is_continuation_if_runningEEEEESI_mEESK_EEEEvRPNS1_11strand_implERT_"},{"b":"400000","o":"CA2CE8","s":"_ZN4asio6detail23reactive_socket_recv_opINS_17mutable_buffers_1ENS0_7read_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS6_EEEES2_NS0_14transfer_all_tENS0_15wrapped_handlerINS_10io_service6strandESt8functionIFvSt10error_codemEENS0_26is_continuation_if_runningEEEEEE11do_completeEPvPNS0_19scheduler_operationERKSF_m"},{"b":"400000","o":"F334E1","s":"_ZN4asio6detail9scheduler10do_run_oneERNS0_11scoped_lockINS0_11posix_mutexEEERNS0_21scheduler_thread_infoERKSt10error_code"},{"b":"400000","o":"F33701","s":"_ZN4asio6detail9scheduler3runERSt10error_code"},{"b":"400000","o":"CB92E9"},{"b":"400000","o":"175C600","s":"execute_native_thread_routine"},{"b":"7FC272AE0000","o":"8064"},{"b":"7FC272735000","o":"E862D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.2.12", "gitVersion" : "ef3e1bc78e997f0d9f22f45aeb1d8e3b6ac14a14", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.16.0-4-amd64", "version" : "#1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07)", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "A9B18E620F02487C1F2B5F355EB119AC75B94CD7" }, { "b" : "7FFF697E2000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "1F644FA66B1BCBBE1359CEF9A63CFD4C8F0C6011" }, { "b" : "7FC273A1C000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "21115992A1F885E1ACE88AADA60F126AD9759D03" }, { "b" : "7FC273620000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "32E9A5B9EED626E93DEEB00A49033F78652DB9A3" }, { "b" : "7FC273418000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "906B9D78305E46BC76994F552FA63751C51CD065" }, { "b" : "7FC273214000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "EDDA40FF0B16D74E776AEA74FAAE6B898ACD2D15" }, { "b" : "7FC272F13000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "1B8F009691E3224A991F1F6517A74DA30A065B9A" }, { "b" : "7FC272CFD000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "D5FB04F64B3DAEA6D6B68B5E8B9D4D2BC1A6E1FC" }, { "b" : "7FC272AE0000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "1ADC4ADBA1D853EEA9A5B3CD49E25AF85DCA0100" }, { "b" : "7FC272735000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "7A02D454BA0E8AF69E3A284C381318B55908DEDA" }, { "b" : "7FC273C7D000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "060BF28EEE293312DDF82D4DBEF40B3BA8927F0A" } ] }}
|
mongod(_ZN5mongo15printStackTraceERSo+0x32) [0x1315b12]
|
mongod(+0xF15432) [0x1315432]
|
mongod(_ZN10__cxxabiv111__terminateEPFvvE+0x6) [0x1b143c6]
|
mongod(+0x17143F3) [0x1b143f3]
|
mongod(_ZN5mongo10ThreadPool25_startWorkerThread_inlockEv+0xA0A) [0x12a5f7a]
|
mongod(_ZN5mongo10ThreadPool8scheduleESt8functionIFvvEE+0x348) [0x12a64d8]
|
mongod(_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPSt4listISt10shared_ptrINS1_13CallbackStateEESaIS5_EERKSt14_List_iteratorIS5_ESC_St11unique_lockISt5mutexE+0x2AE) [0x10d86ae]
|
mongod(_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPSt4listISt10shared_ptrINS1_13CallbackStateEESaIS5_EERKSt14_List_iteratorIS5_ESt11unique_lockISt5mutexE+0x3E) [0x10d8f8e]
|
mongod(+0xCD96D6) [0x10d96d6]
|
mongod(_ZN5mongo8executor20NetworkInterfaceASIO18_completeOperationEPNS1_7AsyncOpERKNS_10StatusWithINS0_21RemoteCommandResponseEEE+0x34D) [0x10c100d]
|
mongod(_ZN5mongo8executor20NetworkInterfaceASIO20_completedOpCallbackEPNS1_7AsyncOpE+0x6A) [0x10c1cca]
|
mongod(+0xCC22C8) [0x10c22c8]
|
mongod(+0xCBF9B0) [0x10bf9b0]
|
mongod(_ZN4asio6detail14strand_service8dispatchINS0_7binder2IRSt8functionIFvSt10error_codemEES5_mEEEEvRPNS1_11strand_implERT_+0x70) [0x1093aa0]
|
mongod(_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder2INS0_7read_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS8_EEEENS_17mutable_buffers_1ENS0_14transfer_all_tENS0_15wrapped_handlerINS_10io_service6strandESt8functionIFvSt10error_codemEENS0_26is_continuation_if_runningEEEEESI_mEESK_EEEEvRPNS1_11strand_implERT_+0x89C) [0x10a282c]
|
mongod(_ZN4asio6detail23reactive_socket_recv_opINS_17mutable_buffers_1ENS0_7read_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS6_EEEES2_NS0_14transfer_all_tENS0_15wrapped_handlerINS_10io_service6strandESt8functionIFvSt10error_codemEENS0_26is_continuation_if_runningEEEEEE11do_completeEPvPNS0_19scheduler_operationERKSF_m+0x228) [0x10a2ce8]
|
mongod(_ZN4asio6detail9scheduler10do_run_oneERNS0_11scoped_lockINS0_11posix_mutexEEERNS0_21scheduler_thread_infoERKSt10error_code+0x2F1) [0x13334e1]
|
mongod(_ZN4asio6detail9scheduler3runERSt10error_code+0xC1) [0x1333701]
|
mongod(+0xCB92E9) [0x10b92e9]
|
mongod(execute_native_thread_routine+0x20) [0x1b5c600]
|
libpthread.so.0(+0x8064) [0x7fc272ae8064]
|
libc.so.6(clone+0x6D) [0x7fc27281d62d]
|
----- END BACKTRACE -----
|
Effective limits:
Limit Soft Limit Hard Limit Units
|
Max cpu time unlimited unlimited seconds
|
Max file size unlimited unlimited bytes
|
Max data size unlimited unlimited bytes
|
Max stack size 8388608 unlimited bytes
|
Max core file size 0 unlimited bytes
|
Max resident set unlimited unlimited bytes
|
Max processes 64000 64000 processes
|
Max open files 64000 64000 files
|
Max locked memory 65536 65536 bytes
|
Max address space unlimited unlimited bytes
|
Max file locks unlimited unlimited locks
|
Max pending signals 1033430 1033430 signals
|
Max msgqueue size 819200 819200 bytes
|
Max nice priority 0 0
|
Max realtime priority 0 0
|
Max realtime timeout unlimited unlimited us
|
Attachments
Issue Links
- is related to
-
SERVER-28974 Mongos leak connections to mongods
-
- Closed
-