[SERVER-16581] Init script should return only after mongod port is open Created: 17/Dec/14  Updated: 06/Feb/23  Resolved: 06/Feb/23

Status: Closed
Project: Core Server
Component/s: Internal Code, Packaging, Usability
Affects Version/s: 2.8.0-rc2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ernie Hershey Assignee: [DO NOT ASSIGN] Backlog - Server Development Platform Team (SDP) (Inactive)
Resolution: Won't Do Votes: 1
Labels: build-later, sdp-backlog-purge
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-15579 MongoD MMAP V1 engine does not recove... Closed
Related
is related to SERVER-31206 SystemD services depending on mongod ... Closed
is related to SERVER-16292 mongod isn't ready for connections wh... Closed
Assigned Teams:
Server Development Platform
Operating System: ALL
Steps To Reproduce:

Here's a simple reproduction command and output.

[ernie@eahimac ~]$ mkdir -p /tmp/crashtest ; rm -rf /tmp/crashtest/* ; ./mongod --port 12312 --dbpath /tmp/crashtest --fork --logpath `tty` --pidfilepath /tmp/crashtest.pid ; kill `cat /tmp/crashtest.pid`
about to fork child process, waiting until server is ready for connections.
forked process: 86934
2014-12-17T18:22:41.050-0500 I CONTROL  [initandlisten] MongoDB starting : pid=86934 port=12312 dbpath=/tmp/crashtest 64-bit host=eahimac
2014-12-17T18:22:41.050-0500 I CONTROL  [initandlisten] db version v2.8.0-rc3-pre-
2014-12-17T18:22:41.050-0500 I CONTROL  [initandlisten] git version: fd08ed58f4d3ffe284e1d28d9ebe44ab10ffbe6f
2014-12-17T18:22:41.050-0500 I CONTROL  [initandlisten] build info: Darwin mci-osx108-7.build.10gen.cc 12.5.0 Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
2014-12-17T18:22:41.050-0500 I CONTROL  [initandlisten] allocator: system
2014-12-17T18:22:41.050-0500 I CONTROL  [initandlisten] options: { net: { port: 12312 }, processManagement: { fork: true, pidFilePath: "/tmp/crashtest.pid" }, storage: { dbPath: "/tmp/crashtest" }, systemLog: { destination: "file", path: "/dev/ttys005" } }
2014-12-17T18:22:41.418-0500 I JOURNAL  [initandlisten] journal dir=/tmp/crashtest/journal
2014-12-17T18:22:41.418-0500 I JOURNAL  [initandlisten] recover : no journal files present, no recovery needed
child process started successfully, parent exiting
2014-12-17T18:22:41.441-0500 I INDEX    [initandlisten] allocating new ns file /tmp/crashtest/local.ns, filling with zeroes...
2014-12-17T18:22:41.444-0500 I CONTROL  [signalProcessingThread] got signal 15 (Terminated: 15), will terminate after current cmd ends
[ernie@eahimac ~]$ 2014-12-17T18:22:41.502-0500 I STORAGE  [FileAllocator] allocating new datafile /tmp/crashtest/local.0, filling with zeroes...
2014-12-17T18:22:41.502-0500 I STORAGE  [FileAllocator] creating directory /tmp/crashtest/_tmp
2014-12-17T18:22:41.767-0500 I STORAGE  [FileAllocator] done allocating datafile /tmp/crashtest/local.0, size: 64MB,  took 0.264 secs
2014-12-17T18:22:41.832-0500 I -        [initandlisten] Assertion: 10357:shutdown in progress
2014-12-17T18:22:41.835-0500 I CONTROL  [initandlisten]
 0x1096b0949 0x109664870 0x109653aaa 0x10949076b 0x1094bac9f 0x1094bb854 0x1094c0d71 0x1094af6c0 0x1094b2d19 0x10908c00f 0x10908d100 0x108f7ce80 0x108f797b3 0x108f7e60c 0x108f79794 0xa
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"108F78000","o":"738949"},{"b":"108F78000","o":"6EC870"},{"b":"108F78000","o":"6DBAAA"},{"b":"108F78000","o":"51876B"},{"b":"108F78000","o":"542C9F"},{"b":"108F78000","o":"543854"},{"b":"108F78000","o":"548D71"},{"b":"108F78000","o":"5376C0"},{"b":"108F78000","o":"53AD19"},{"b":"108F78000","o":"11400F"},{"b":"108F78000","o":"115100"},{"b":"108F78000","o":"4E80"},{"b":"108F78000","o":"17B3"},{"b":"108F78000","o":"660C"},{"b":"108F78000","o":"1794"},{"b":"0","o":"A"}],"processInfo":{ "mongodbVersion" : "2.8.0-rc3-pre-", "gitVersion" : "fd08ed58f4d3ffe284e1d28d9ebe44ab10ffbe6f", "uname" : { "sysname" : "Darwin", "release" : "14.0.0", "version" : "Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64", "machine" : "x86_64" }, "somap" : [ { "path" : "/Users/ernie/./mongod", "machType" : 2, "b" : "108F78000", "buildId" : "33DA128AAAA43CBE8854F01EFDFD426D" }, { "path" : "/usr/lib/libSystem.B.dylib", "machType" : 6, "b" : "7FFF8C035000", "buildId" : "DA954461EC6A3DF085516FC810627627" }, { "path" : "/usr/lib/libc++.1.dylib", "machType" : 6, "b" : "7FFF8FFCA000", "buildId" : "1B9530FD989B3174BB1CBDC159501710" }, { "path" : "/usr/lib/system/libcache.dylib", "machType" : 6, "b" : "7FFF8BFB7000", "buildId" : "45E9A2E799C436B2BEE30C4E11614AD1" }, { "path" : "/usr/lib/system/libcommonCrypto.dylib", "machType" : 6, "b" : "7FFF89B02000", "buildId" : "D381EBC669D831D380845A80A32CB748" }, { "path" : "/usr/lib/system/libcompiler_rt.dylib", "machType" : 6, "b" : "7FFF8EF21000", "buildId" : "BF8FC133EE103DA69B9092039E28678F" }, { "path" : "/usr/lib/system/libcopyfile.dylib", "machType" : 6, "b" : "7FFF89B0E000", "buildId" : "0C68D3A6ACDD3EF3991ACC82C32AB836" }, { "path" : "/usr/lib/system/libcorecrypto.dylib", "machType" : 6, "b" : "7FFF88F5F000", "buildId" : "E178980139853949B7366B3378873301" }, { "path" : "/usr/lib/system/libdispatch.dylib", "machType" : 6, "b" : "7FFF86946000", "buildId" : "502CF32B669B3709886208188225E4F0" }, { "path" : "/usr/lib/system/libdyld.dylib", "machType" : 6, "b" : "7FFF8960D000", "buildId" : "19FAF435C16533749DEFD7BBA7D61DB6" }, { "path" : "/usr/lib/system/libkeymgr.dylib", "machType" : 6, "b" : "7FFF916C7000", "buildId" : "77845842DE703CC5BD01C3D14227CED5" }, { "path" : "/usr/lib/system/liblaunch.dylib", "machType" : 6, "b" : "7FFF91883000", "buildId" : "8A9889248BE735FEBF7D322E90EFE49E" }, { "path" : "/usr/lib/system/libmacho.dylib", "machType" : 6, "b" : "7FFF8AE85000", "buildId" : "126CA2EDDE91308F8881B9DAEC3C63B6" }, { "path" : "/usr/lib/system/libquarantine.dylib", "machType" : 6, "b" : "7FFF914B2000", "buildId" : "DC0416272D92361CBABFA869A5C72293" }, { "path" : "/usr/lib/system/libremovefile.dylib", "machType" : 6, "b" : "7FFF8AE24000", "buildId" : "3485B5F46CE83C628DFD8736ED6E8531" }, { "path" : "/usr/lib/system/libsystem_asl.dylib", "machType" : 6, "b" : "7FFF86194000", "buildId" : "F153AC5B0542356E88C820A62CA704E2" }, { "path" : "/usr/lib/system/libsystem_blocks.dylib", "machType" : 6, "b" : "7FFF8C609000", "buildId" : "9615D10AFCA73BE4AA1A1B195DACE1A1" }, { "path" : "/usr/lib/system/libsystem_c.dylib", "machType" : 6, "b" : "7FFF8C1CF000", "buildId" : "C185E86274243210B5286B822577A4B8" }, { "path" : "/usr/lib/system/libsystem_configuration.dylib", "machType" : 6, "b" : "7FFF8C266000", "buildId" : "9FBA1CE497D0347EA44393ED94512E92" }, { "path" : "/usr/lib/system/libsystem_coreservices.dylib", "machType" : 6, "b" : "7FFF88031000", "buildId" : "41B7C5785A5331C8A96FC73E030B0938" }, { "path" : "/usr/lib/system/libsystem_coretls.dylib", "machType" : 6, "b" : "7FFF91604000", "buildId" : "EBBF7EF680D83F8F825CB412BD6D22C0" }, { "path" : "/usr/lib/system/libsystem_dnssd.dylib", "machType" : 6, "b" : "7FFF8C07F000", "buildId" : "62B70ECAE40D3C63896E7F00EC386DDB" }, { "path" : "/usr/lib/system/libsystem_info.dylib", "machType" : 6, "b" : "7FFF9016B000", "buildId" : "B85A85D585303A93B0C34DEC41F79478" }, { "path" : "/usr/lib/system/libsystem_kernel.dylib", "machType" : 6, "b" : "7FFF8B7D1000", "buildId" : "93E0E0A975B63904BB4E4BC7C05F4B6B" }, { "path" : "/usr/lib/system/libsystem_m.dylib", "machType" : 6, "b" : "7FFF86877000", "buildId" : "1E12AB456D9636D0A226F24D9FB0D9D6" }, { "path" : "/usr/lib/system/libsystem_malloc.dylib", "machType" : 6, "b" : "7FFF8930A000", "buildId" : "19BCC25757173502A71F95D65AFA861B" }, { "path" : "/usr/lib/system/libsystem_network.dylib", "machType" : 6, "b" : "7FFF871B3000", "buildId" : "C0B2313D47BE38A9BEE62634A4F5E14B" }, { "path" : "/usr/lib/system/libsystem_networkextension.dylib", "machType" : 6, "b" : "7FFF8AD0B000", "buildId" : "29AB225BD7FB30ED960065D44B9A9442" }, { "path" : "/usr/lib/system/libsystem_notify.dylib", "machType" : 6, "b" : "7FFF89AF8000", "buildId" : "61147800F3203DAA850CBADF33855F29" }, { "path" : "/usr/lib/system/libsystem_platform.dylib", "machType" : 6, "b" : "7FFF86FA8000", "buildId" : "64E34079D7123D669CE2418624A5C040" }, { "path" : "/usr/lib/system/libsystem_pthread.dylib", "machType" : 6, "b" : "7FFF852B0000", "buildId" : "26B1897F0CD330F3B55A37CB45062D73" }, { "path" : "/usr/lib/system/libsystem_sandbox.dylib", "machType" : 6, "b" : "7FFF900A3000", "buildId" : "DB9962EF889831CC9B87E01F8CE74C9D" }, { "path" : "/usr/lib/system/libsystem_secinit.dylib", "machType" : 6, "b" : "7FFF8E8CF000", "buildId" : "581DAD0F6B633A48B63B917AF799ABAA" }, { "path" : "/usr/lib/system/libsystem_stats.dylib", "machType" : 6, "b" : "7FFF88B76000", "buildId" : "1DB0443659743F1686CC5FF5F390339C" }, { "path" : "/usr/lib/system/libsystem_trace.dylib", "machType" : 6, "b" : "7FFF8C078000", "buildId" : "A9E6B7D8C3273742AC5486C94218B1DF" }, { "path" : "/usr/lib/system/libunc.dylib", "machType" : 6, "b" : "7FFF9298B000", "buildId" : "5676F7EAC1DF329FB006D2C3022B7D70" }, { "path" : "/usr/lib/system/libunwind.dylib", "machType" : 6, "b" : "7FFF86F03000", "buildId" : "BE7E51A0B6EA3A549CCA9D88F683A6D6" }, { "path" : "/usr/lib/system/libxpc.dylib", "machType" : 6, "b" : "7FFF8535E000", "buildId" : "9437C02EA07B38C891CB299FAA63083D" }, { "path" : "/usr/lib/libobjc.A.dylib", "machType" : 6, "b" : "7FFF8F7D7000", "buildId" : "3B60CD9074A23A5D9686B0772159792A" }, { "path" : "/usr/lib/libauto.dylib", "machType" : 6, "b" : "7FFF860CE000", "buildId" : "A260789BD4D8316A9490254767B8A5F1" }, { "path" : "/usr/lib/libc++abi.dylib", "machType" : 6, "b" : "7FFF9204D000", "buildId" : "88A22A0F87C63002BFBAAC0F2808B8B9" }, { "path" : "/usr/lib/libDiagnosticMessagesClient.dylib", "machType" : 6, "b" : "7FFF8B46A000", "buildId" : "2EE8E4365CDC34C599595BA218D507FB" } ] }}
 mongod(_ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE+0x39) [0x1096b0949]
 mongod(_ZN5mongo10logContextEPKc+0x100) [0x109664870]
 mongod(_ZN5mongo11msgassertedEiPKc+0x13A) [0x109653aaa]
 mongod(_ZN5mongo8DataFile15allocExtentAreaEPNS_16OperationContextEi+0xEB) [0x10949076b]
 mongod(_ZN5mongo19MmapV1ExtentManager19_createExtentInFileEPNS_16OperationContextEiPNS_8DataFileEib+0x5F) [0x1094bac9f]
 mongod(_ZN5mongo19MmapV1ExtentManager14allocateExtentEPNS_16OperationContextEbib+0x74) [0x1094bb854]
 mongod(_ZN5mongo17RecordStoreV1Base19increaseStorageSizeEPNS_16OperationContextEib+0x41) [0x1094c0d71]
 mongod(_ZN5mongo26MMAPV1DatabaseCatalogEntry9_lazyInitEPNS_16OperationContextE+0x620) [0x1094af6c0]
 mongod(_ZN5mongo26MMAPV1DatabaseCatalogEntry16createCollectionEPNS_16OperationContextERKNS_10StringDataERKNS_17CollectionOptionsEb+0x39) [0x1094b2d19]
 mongod(_ZN5mongo8Database16createCollectionEPNS_16OperationContextERKNS_10StringDataERKNS_17CollectionOptionsEbb+0x22F) [0x10908c00f]
 mongod(_ZN5mongo12userCreateNSEPNS_16OperationContextEPNS_8DatabaseERKNS_10StringDataENS_7BSONObjEbb+0x410) [0x10908d100]
 mongod(_ZN5mongoL14_initAndListenEi+0x3180) [0x108f7ce80]
 mongod(_ZN5mongo13initAndListenEi+0x13) [0x108f797b3]
 mongod(main+0x3BC) [0x108f7e60c]
 mongod(start+0x34) [0x108f79794]
 ??? [0xa]
-----  END BACKTRACE  -----
2014-12-17T18:22:41.838-0500 I STORAGE  [initandlisten] exception in initAndListen: 10357 shutdown in progress, terminating
2014-12-17T18:22:41.838-0500 I COMMAND  [signalProcessingThread] now exiting
2014-12-17T18:22:41.839-0500 I NETWORK  [signalProcessingThread] shutdown: going to close listening sockets...
2014-12-17T18:22:41.839-0500 I NETWORK  [signalProcessingThread] removing socket file: /tmp/mongodb-12312.sock
2014-12-17T18:22:41.839-0500 I NETWORK  [signalProcessingThread] shutdown: going to flush diaglog...
2014-12-17T18:22:41.839-0500 I NETWORK  [signalProcessingThread] shutdown: going to close sockets...
2014-12-17T18:22:41.839-0500 I STORAGE  [signalProcessingThread] shutdown: waiting for fs preallocator...
2014-12-17T18:22:41.839-0500 I STORAGE  [signalProcessingThread] shutdown: final commit...
2014-12-17T18:22:41.853-0500 I STORAGE  [signalProcessingThread] shutdown: closing all files...
2014-12-17T18:22:41.854-0500 I STORAGE  [signalProcessingThread] closeAllFiles() finished
2014-12-17T18:22:41.854-0500 I JOURNAL  [signalProcessingThread] journalCleanup...
2014-12-17T18:22:41.854-0500 I JOURNAL  [signalProcessingThread] removeJournalFiles
2014-12-17T18:22:41.854-0500 I STORAGE  [signalProcessingThread] shutdown: removing fs lock...
2014-12-17T18:22:41.854-0500 I COMMAND  [signalProcessingThread] dbexit:  rc: 0
 

Participants:

 Description   

If I shutdown mongod right after the parent process returns with --fork, it exits without having finished preallocating files.

I encountered this after installing RPM's in Linux and trying to run this to preallocate files:
service mongod start
service mongod stop

At which point, I can't start a new mongod due to SERVER-15579.

I verified the behavior in OSX and Linux with binaries from the latest nightly tarballs.



 Comments   
Comment by Iryna Zhuravlova [ 16/Nov/22 ]

After a careful backlog refinement, the team decided to close this ticket because of its low priority and limited resource capacity. If you believe that this ticket requires additional attention from the team and should be re-opened, feel free to change the status to "Needs Scheduling" and ping me or @alexander.neben

Generated at Thu Feb 08 03:41:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.