[SERVER-44580] Crash of 3.6.11 Created: 12/Nov/19  Updated: 18/Dec/19  Resolved: 18/Dec/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 3.6.11
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Nick Hristov Assignee: Eric Sedor
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

I do not have much additional information to add. There was no special or unusual activity against the server when the crash happened.

 

Participants:

 Description   

Version:

db version v3.6.11
git version: b4339db12bf57ffee5b84a95c6919dbd35fe31c9
OpenSSL version: OpenSSL 1.0.0-fips 29 Mar 2010
allocator: tcmalloc
modules: none
build environment:
    distmod: amazon
    distarch: x86_64
    target_arch: x86_64

Backtrace:

 

----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"557BC4DE0000","o":"226F611","s":"_ZN5mongo15printStackTraceERSo"},{"b":"557BC4DE0000","o":"226EFF5"},{"b":"557BC4DE0000","o":"2363C16","s":"_ZN10__cxxabiv111__terminateEPFvvE"},{"b":"557BC4DE0000","o":"2363C61"},{"b":"557BC4DE0000","o":"1A4312F","s":"_ZN5mongo10ThreadPool25_startWorkerThread_inlockEv"},{"b":"557BC4DE0000","o":"1A43988","s":"_ZN5mongo10ThreadPool8scheduleESt8functionIFvvEE"},{"b":"557BC4DE0000","o":"1CE637C","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS1_13CallbackStateEESaIS6_EEERKSt14_List_iteratorIS6_ESD_St11unique_lockISt5mutexE"},{"b":"557BC4DE0000","o":"1CE6B7D","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS1_13CallbackStateEESaIS6_EEERKSt14_List_iteratorIS6_ESt11unique_lockISt5mutexE"},{"b":"557BC4DE0000","o":"1CE6CC0"},{"b":"557BC4DE0000","o":"1CFDBB1"},{"b":"557BC4DE0000","o":"1DB5439","s":"_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code"},{"b":"557BC4DE0000","o":"1DB5681","s":"_ZN4asio6detail9scheduler3runERSt10error_code"},{"b":"557BC4DE0000","o":"1CF8419"},{"b":"557BC4DE0000","o":"237ED90"},{"b":"7F53A1850000","o":"7DE5"},{"b":"7F53A148C000","o":"F830D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.6.11", "gitVersion" : "b4339db12bf57ffee5b84a95c6919dbd35fe31c9", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.9.51-10.52.amzn1.x86_64", "version" : "#1 SMP Fri Sep 29 01:16:19 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "557BC4DE0000", "elfType" : 3, "buildId" : "2A7F42D96E44D70F66EA62723F08E10604502182" }, { "b" : "7FFD7CD2A000", "elfType" : 3, "buildId" : "E1FD0678C5C561D462A1DAA29BD08CC5361EF117" }, { "b" : "7F53A2A5F000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "0BED68388AE751ACB77DA9FBB6B63CAA5E4622A9" }, { "b" : "7F53A2601000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "C2A116A82EE1814AD13688F059F7CB8A4B8D9AA2" }, { "b" : "7F53A2390000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "04B9A410FB1D3E1FB4A6684859AFF5E3A60E7810" }, { "b" : "7F53A218C000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "2CC91EC12043BC0357CD502BBB4D5A4940BFFB1B" }, { "b" : "7F53A1F84000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "BA23B2F2723B7C9BF5FFE9A8AF9E50219C03EB0E" }, { "b" : "7F53A1C82000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "31404F7BCB6B5A880B25E0007EA5C73BBEC87E16" }, { "b" : "7F53A1A6C000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "AC58019512A5359B077D2610DEA4AD6CF14CAC53" }, { "b" : "7F53A1850000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "4136F256787CE855F91B70793154EBCFCE047CFA" }, { "b" : "7F53A148C000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "FB411B5AB3944BD21963101275C0F9EB3F943BDC" }, { "b" : "7F53A2C79000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "CC0DF41E2210C483EFF9C95EDB5F2AE014752C70" }, { "b" : "7F53A1276000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "89C6AF118B6B4FB6A73AE1813E2C8BDD722956D1" }, { "b" : "7F53A1028000", "path" : "/usr/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "1BE9E6309ED365E35806E13FA9E23350D71F2513" }, { "b" : "7F53A0D41000", "path" : "/usr/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "9EE23694485D684651195C7B51766E47D0CB95E3" }, { "b" : "7F53A0B3E000", "path" : "/usr/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "5C01209C5AE1B1714F19B07EB58F2A1274B69DC8" }, { "b" : "7F53A090C000", "path" : "/usr/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "FD5974E4861D56DFFFFC8BF5DB35E74B1C20ABD5" }, { "b" : "7F53A06FD000", "path" : "/usr/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "1B55330B231D45AF433F7D9DCA507C5FB0609780" }, { "b" : "7F53A04FA000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "37A58210FA50C91E09387765408A92909468D25B" }, { "b" : "7F53A02D9000", "path" : "/usr/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "F5054DC94443326819FBF3065CFDF5E4726F57EE" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x557bc704f611] mongod(+0x226EFF5) [0x557bc704eff5] mongod(_ZN10__cxxabiv111__terminateEPFvvE+0x6) [0x557bc7143c16] mongod(+0x2363C61) [0x557bc7143c61] mongod(_ZN5mongo10ThreadPool25_startWorkerThread_inlockEv+0x99F) [0x557bc682312f] mongod(_ZN5mongo10ThreadPool8scheduleESt8functionIFvvEE+0x398) [0x557bc6823988] mongod(_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS1_13CallbackStateEESaIS6_EEERKSt14_List_iteratorIS6_ESD_St11unique_lockISt5mutexE+0x24C) [0x557bc6ac637c] mongod(_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS1_13CallbackStateEESaIS6_EEERKSt14_List_iteratorIS6_ESt11unique_lockISt5mutexE+0x4D) [0x557bc6ac6b7d] mongod(+0x1CE6CC0) [0x557bc6ac6cc0] mongod(+0x1CFDBB1) [0x557bc6addbb1] mongod(_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code+0x389) [0x557bc6b95439] mongod(_ZN4asio6detail9scheduler3runERSt10error_code+0xD1) [0x557bc6b95681] mongod(+0x1CF8419) [0x557bc6ad8419] mongod(+0x237ED90) [0x557bc715ed90] libpthread.so.0(+0x7DE5) [0x7f53a1857de5] libc.so.6(clone+0x6D) [0x7f53a158430d]
-----  END BACKTRACE  ----- 



 Comments   
Comment by Eric Sedor [ 18/Dec/19 ]

Understood nick@turvo.com; if you have trouble again please don't hesitate to comment here or open a new ticket.

Thanks,
Eric

Comment by Nick Hristov [ 09/Dec/19 ]

I am pretty sure its gone by now.

Comment by Eric Sedor [ 06/Dec/19 ]

nick@turvo.com,

We understand the information might not be available, but we would need it to potentially diagnose the problem. Are you able to provide the above?

Thanks,
Eric

Comment by Eric Sedor [ 15/Nov/19 ]

nick@turvo.com,

I've created a secure upload portal for you. Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time.

Can you please upload the mongodb log file leading up to the crash? The log lines preceding the backtrace may provide additional useful information.

Would you please also archive (tar or zip) the $dbpath/diagnostic.data directory (the contents are described here) and attach it to this ticket?

Thank you in advance.

Generated at Thu Feb 08 05:06:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.