[SERVER-25547] mongos segfault during shutdown if port is already in use Created: 10/Aug/16  Updated: 06/Dec/22  Resolved: 08/Sep/16

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-25997 Mongos may segfault at shutdown if gr... Closed
Assigned Teams:
Sharding
Operating System: ALL
Steps To Reproduce:

start up a mongos, then start another mongos on the same port

Sprint: Sharding 2016-08-29, Sharding 2016-09-19
Participants:

 Description   

If mongos fails to bind to the port it's supposed to listen on because the port is already in use, it segfaults during shutdown

/home/spencer/mongo4/mongos  --configdb configRS/ubuntu:50001,ubuntu:50002,ubuntu:50003
2016-08-10T16:18:37.212-0400 I CONTROL  [main] 
2016-08-10T16:18:37.212-0400 I CONTROL  [main] ** NOTE: This is a development version (3.3.10-342-g7f34776) of MongoDB.
2016-08-10T16:18:37.212-0400 I CONTROL  [main] **       Not recommended for production.
2016-08-10T16:18:37.212-0400 I CONTROL  [main] 
2016-08-10T16:18:37.212-0400 I CONTROL  [main] ** WARNING: Access control is not enabled for the database.
2016-08-10T16:18:37.212-0400 I CONTROL  [main] **          Read and write access to data and configuration is unrestricted.
2016-08-10T16:18:37.212-0400 I CONTROL  [main] 
2016-08-10T16:18:37.212-0400 I SHARDING [mongosMain] mongos version v3.3.10-342-g7f34776
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] git version: 7f34776bdc67190585b938ea5c04190601a1ec12
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] allocator: tcmalloc
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] modules: none
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] build environment:
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain]     distarch: x86_64
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain]     target_arch: x86_64
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] db version v3.3.10-342-g7f34776
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] git version: 7f34776bdc67190585b938ea5c04190601a1ec12
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] allocator: tcmalloc
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] modules: none
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] build environment:
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain]     distarch: x86_64
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain]     target_arch: x86_64
2016-08-10T16:18:37.212-0400 I CONTROL  [mongosMain] options: { sharding: { configDB: "configRS/ubuntu:50001,ubuntu:50002,ubuntu:50003" } }
2016-08-10T16:18:37.236-0400 E NETWORK  [mongosMain] listen(): bind() failed Address already in use for socket: 0.0.0.0:27017
2016-08-10T16:18:37.236-0400 E NETWORK  [mongosMain]   addr already in use
2016-08-10T16:18:37.236-0400 E NETWORK  [mongosMain] Failed to set up sockets during startup.
2016-08-10T16:18:37.236-0400 F -        [mongosMain] Invalid access at address: 0x18
2016-08-10T16:18:37.240-0400 F -        [mongosMain] Got signal: 11 (Segmentation fault).
 
 0x7f9c6336d5f1 0x7f9c6336c389 0x7f9c6336ca0d 0x7f9c6143d330 0x7f9c61437404 0x7f9c632540c3 0x7f9c62be5e35 0x7f9c632f3a62 0x7f9c62baa6b6 0x7f9c62bad790 0x7f9c61089f45 0x7f9c62be550f
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"7F9C627D3000","o":"B9A5F1","s":"_ZN5mongo15printStackTraceERSo"},{"b":"7F9C627D3000","o":"B99389"},{"b":"7F9C627D3000","o":"B99A0D"},{"b":"7F9C6142D000","o":"10330"},{"b":"7F9C6142D000","o":"A404","s":"pthread_mutex_lock"},{"b":"7F9C627D3000","o":"A810C3","s":"_ZN5mongo20ClusterCursorManager8shutdownEv"},{"b":"7F9C627D3000","o":"412E35"},{"b":"7F9C627D3000","o":"B20A62"},{"b":"7F9C627D3000","o":"3D76B6","s":"_ZN5mongo8shutdownENS_8ExitCodeE"},{"b":"7F9C627D3000","o":"3DA790"},{"b":"7F9C61068000","o":"21F45","s":"__libc_start_main"},{"b":"7F9C627D3000","o":"41250F"}],"processInfo":{ "mongodbVersion" : "3.3.10-342-g7f34776", "gitVersion" : "7f34776bdc67190585b938ea5c04190601a1ec12", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.13.0-91-generic", "version" : "#138-Ubuntu SMP Fri Jun 24 17:00:34 UTC 2016", "machine" : "x86_64" }, "somap" : [ { "b" : "7F9C627D3000", "elfType" : 3, "buildId" : "ACE260CC09D09D844B36F15351ED7E16A74B018E" }, { "b" : "7FFECB1EF000", "elfType" : 3, "buildId" : "4EAD28F888EF3520872B73C6F2FE7306C41AF22F" }, { "b" : "7F9C6234F000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "74864DB9D5F69D39A67E4755012FB6573C469B3D" }, { "b" : "7F9C61F73000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "AAE7CFF8351B730830BDBCE0DCABBE06574B7144" }, { "b" : "7F9C61D6B000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "E2A6DD5048A0A051FD61043BDB69D8CC68192AB7" }, { "b" : "7F9C61B67000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "DA9B8C234D0FE9FD8CAAC8970A7EC1B6C8F6623F" }, { "b" : "7F9C61861000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "D144258E614900B255A31F3FD2283A878670D5BC" }, { "b" : "7F9C6164B000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "36311B4457710AE5578C4BF00791DED7359DBB92" }, { "b" : "7F9C6142D000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "31E9F21AE8C10396171F1E13DA15780986FA696C" }, { "b" : "7F9C61068000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "CF699A15CAAE64F50311FC4655B86DC39A479789" }, { "b" : "7F9C625AE000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "D0F537904076D73F29E4A37341F8A449E2EF6CD0" } ] }}
 mongos(_ZN5mongo15printStackTraceERSo+0x41) [0x7f9c6336d5f1]
 mongos(+0xB99389) [0x7f9c6336c389]
 mongos(+0xB99A0D) [0x7f9c6336ca0d]
 libpthread.so.0(+0x10330) [0x7f9c6143d330]
 libpthread.so.0(pthread_mutex_lock+0x4) [0x7f9c61437404]
 mongos(_ZN5mongo20ClusterCursorManager8shutdownEv+0x43) [0x7f9c632540c3]
 mongos(+0x412E35) [0x7f9c62be5e35]
 mongos(+0xB20A62) [0x7f9c632f3a62]
 mongos(_ZN5mongo8shutdownENS_8ExitCodeE+0x250) [0x7f9c62baa6b6]
 mongos(+0x3DA790) [0x7f9c62bad790]
 libc.so.6(__libc_start_main+0xF5) [0x7f9c61089f45]
 mongos(+0x41250F) [0x7f9c62be550f]
-----  END BACKTRACE  -----
2016-08-10T16:18:37.240-0400 F -        [mongosMain] /proc/self/maps:
7f9c60066000-7f9c60067000 ---p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c60067000-7f9c60867000 rw-p 00000000 00:00 0                          [stack:16131]
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c60867000-7f9c60868000 ---p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c60868000-7f9c61068000 rw-p 00000000 00:00 0                          [stack:16130]
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61068000-7f9c61222000 r-xp 00000000 fc:00 2363139                    /lib/x86_64-linux-gnu/libc-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61222000-7f9c61422000 ---p 001ba000 fc:00 2363139                    /lib/x86_64-linux-gnu/libc-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61422000-7f9c61426000 r--p 001ba000 fc:00 2363139                    /lib/x86_64-linux-gnu/libc-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61426000-7f9c61428000 rw-p 001be000 fc:00 2363139                    /lib/x86_64-linux-gnu/libc-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61428000-7f9c6142d000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c6142d000-7f9c61446000 r-xp 00000000 fc:00 2363085                    /lib/x86_64-linux-gnu/libpthread-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61446000-7f9c61645000 ---p 00019000 fc:00 2363085                    /lib/x86_64-linux-gnu/libpthread-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61645000-7f9c61646000 r--p 00018000 fc:00 2363085                    /lib/x86_64-linux-gnu/libpthread-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61646000-7f9c61647000 rw-p 00019000 fc:00 2363085                    /lib/x86_64-linux-gnu/libpthread-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61647000-7f9c6164b000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c6164b000-7f9c61661000 r-xp 00000000 fc:00 2363099                    /lib/x86_64-linux-gnu/libgcc_s.so.1
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61661000-7f9c61860000 ---p 00016000 fc:00 2363099                    /lib/x86_64-linux-gnu/libgcc_s.so.1
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61860000-7f9c61861000 rw-p 00015000 fc:00 2363099                    /lib/x86_64-linux-gnu/libgcc_s.so.1
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61861000-7f9c61966000 r-xp 00000000 fc:00 2360647                    /lib/x86_64-linux-gnu/libm-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61966000-7f9c61b65000 ---p 00105000 fc:00 2360647                    /lib/x86_64-linux-gnu/libm-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61b65000-7f9c61b66000 r--p 00104000 fc:00 2360647                    /lib/x86_64-linux-gnu/libm-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61b66000-7f9c61b67000 rw-p 00105000 fc:00 2360647                    /lib/x86_64-linux-gnu/libm-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61b67000-7f9c61b6a000 r-xp 00000000 fc:00 2360718                    /lib/x86_64-linux-gnu/libdl-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61b6a000-7f9c61d69000 ---p 00003000 fc:00 2360718                    /lib/x86_64-linux-gnu/libdl-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61d69000-7f9c61d6a000 r--p 00002000 fc:00 2360718                    /lib/x86_64-linux-gnu/libdl-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61d6a000-7f9c61d6b000 rw-p 00003000 fc:00 2360718                    /lib/x86_64-linux-gnu/libdl-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61d6b000-7f9c61d72000 r-xp 00000000 fc:00 2363138                    /lib/x86_64-linux-gnu/librt-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61d72000-7f9c61f71000 ---p 00007000 fc:00 2363138                    /lib/x86_64-linux-gnu/librt-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61f71000-7f9c61f72000 r--p 00006000 fc:00 2363138                    /lib/x86_64-linux-gnu/librt-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61f72000-7f9c61f73000 rw-p 00007000 fc:00 2363138                    /lib/x86_64-linux-gnu/librt-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c61f73000-7f9c62126000 r-xp 00000000 fc:00 2359535                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c62126000-7f9c62325000 ---p 001b3000 fc:00 2359535                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c62325000-7f9c62340000 r--p 001b2000 fc:00 2359535                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c62340000-7f9c6234b000 rw-p 001cd000 fc:00 2359535                    /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c6234b000-7f9c6234f000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c6234f000-7f9c623a4000 r-xp 00000000 fc:00 2359661                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c623a4000-7f9c625a4000 ---p 00055000 fc:00 2359661                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c625a4000-7f9c625a7000 r--p 00055000 fc:00 2359661                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c625a7000-7f9c625ae000 rw-p 00058000 fc:00 2359661                    /lib/x86_64-linux-gnu/libssl.so.1.0.0
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c625ae000-7f9c625d1000 r-xp 00000000 fc:00 2363088                    /lib/x86_64-linux-gnu/ld-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c627ae000-7f9c627b4000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c627cd000-7f9c627d0000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c627d0000-7f9c627d1000 r--p 00022000 fc:00 2363088                    /lib/x86_64-linux-gnu/ld-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c627d1000-7f9c627d2000 rw-p 00023000 fc:00 2363088                    /lib/x86_64-linux-gnu/ld-2.19.so
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c627d2000-7f9c627d3000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c627d3000-7f9c63f61000 r-xp 00000000 08:11 29624185                   /media/ssd/mongo4/mongos
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c63f61000-7f9c63fb1000 r--p 0178d000 08:11 29624185                   /media/ssd/mongo4/mongos
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c63fb1000-7f9c640a5000 rw-p 017dd000 08:11 29624185                   /media/ssd/mongo4/mongos
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c640a5000-7f9c640f9000 rw-p 00000000 00:00 0 
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7f9c64689000-7f9c65289000 rw-p 00000000 00:00 0                          [heap]
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7ffeca845000-7ffeca868000 rw-p 00000000 00:00 0                          [stack]
2016-08-10T16:18:37.240-0400 F -        [mongosMain] 7ffeca8ef000-7ffeca8f1000 r-xp 00000000 00:00 0                          [vdso]
2016-08-10T16:18:37.240-0400 F -        [mongosMain] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]


Generated at Thu Feb 08 04:09:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.