[SERVER-80269] Mongo finished after a Backtrace Created: 21/Aug/23  Updated: 03/Oct/23  Resolved: 03/Oct/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.4.7
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Eduardo godoi dos santos Assignee: Alison Rhea Thorne
Resolution: Done Votes: 0
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-41368 mongocryptd terminates with backtrace... Closed
Operating System: ALL
Participants:

 Description   

My MongoDb stopped after I got this log.

After this backtrace happend, I restarted the aplication and everything it's running ok.

I had this issues 3 times, but is not a regular issue, I don't no what i need to do to replicate this problem.

{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31431,   "ctx":"ftdc","msg":"BACKTRACE: {bt}","attr":{"bt":{"backtrace":[{"a":"55D21AEBC5CA","b":"55D218113000","o":"2DA95CA","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"},{"a":"55D21AEBE059","b":"55D218113000","o":"2DAB059","s":"_ZN5mongo15printStackTraceEv","s+":"29"},{"a":"55D21AEBB226","b":"55D218113000","o":"2DA8226","s":"_ZN5mongo12_GLOBAL__N_111myTerminateEv","s+":"A6"},{"a":"55D21B04B286","b":"55D218113000","o":"2F38286","s":"_ZN10__cxxabiv111__terminateEPFvvE","s+":"6"},{"a":"55D21B0DF2C9","b":"55D218113000","o":"2FCC2C9","s":"__cxa_call_terminate","s+":"39"},{"a":"55D21B04AC75","b":"55D218113000","o":"2F37C75","s":"__gxx_personality_v0","s+":"275"},{"a":"7F97F3696E33","b":"7F97F3686000","o":"10E33","s":"_Unwind_GetTextRelBase","s+":"1E03"},{"a":"7F97F3697391","b":"7F97F3686000","o":"11391","s":"_Unwind_RaiseException","s+":"2B1"},{"a":"55D21B04B3E6","b":"55D218113000","o":"2F383E6","s":"__cxa_throw","s+":"36"},{"a":"55D219038EDF","b":"55D218113000","o":"F25EDF","s":"_ZN5mongo13error_details23throwExceptionForStatusERKNS_6StatusE","s+":"1B23"},{"a":"55D21904D10F","b":"55D218113000","o":"F3A10F","s":"_ZN5mongo21uassertedWithLocationERKNS_6StatusEPKcj","s+":"27B"},{"a":"55D218DAAE0B","b":"55D218113000","o":"C97E0B","s":"_ZN5mongo14FTDCController6doLoopEv.cold.395","s+":"2D"},{"a":"55D2195C736C","b":"55D218113000","o":"14B436C","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_14FTDCController5startEvEUlvE0_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"5C"},{"a":"55D21B066D2F","b":"55D218113000","o":"2F53D2F","s":"execute_native_thread_routine","s+":"F"},{"a":"7F97F366CFA3","b":"7F97F3665000","o":"7FA3","s":"start_thread","s+":"F3"},{"a":"7F97F359D4CF","b":"7F97F34A4000","o":"F94CF","s":"clone","s+":"3F"}],"processInfo":{"mongodbVersion":"4.4.7","gitVersion":"abb6b9c2bf675e9e2aeaecba05f0f8359d99e203","compiledModules":[],"uname":{"sysname":"Linux","release":"5.4.0-1059-azure","version":"#62~18.04.1-Ubuntu SMP Tue Sep 14 17:53:18 UTC 2021","machine":"x86_64"},"somap":[{"b":"55D218113000","elfType":3,"buildId":"D589348E02D9840EADD6321FBE671F05F68C3C2A"},{"b":"7F97F3686000","path":"/lib/x86_64-linux-gnu/libgcc_s.so.1","elfType":3,"buildId":"DE6B14E57AEA9BBEAF1E81EB6772E2222101AA6E"},{"b":"7F97F3665000","path":"/lib/x86_64-linux-gnu/libpthread.so.0","elfType":3,"buildId":"E91114987A0147BD050ADDBD591EB8994B29F4B3"},{"b":"7F97F34A4000","path":"/lib/x86_64-linux-gnu/libc.so.6","elfType":3,"buildId":"18B9A9A8C523E5CFE5B5D946D605D09242F09798"}]}}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21AEBC5CA","b":"55D218113000","o":"2DA95CA","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkE.constprop.606","s+":"1EA"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21AEBE059","b":"55D218113000","o":"2DAB059","s":"_ZN5mongo15printStackTraceEv","s+":"29"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21AEBB226","b":"55D218113000","o":"2DA8226","s":"_ZN5mongo12_GLOBAL__N_111myTerminateEv","s+":"A6"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21B04B286","b":"55D218113000","o":"2F38286","s":"_ZN10__cxxabiv111__terminateEPFvvE","s+":"6"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21B0DF2C9","b":"55D218113000","o":"2FCC2C9","s":"__cxa_call_terminate","s+":"39"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21B04AC75","b":"55D218113000","o":"2F37C75","s":"__gxx_personality_v0","s+":"275"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"7F97F3696E33","b":"7F97F3686000","o":"10E33","s":"_Unwind_GetTextRelBase","s+":"1E03"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"7F97F3697391","b":"7F97F3686000","o":"11391","s":"_Unwind_RaiseException","s+":"2B1"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21B04B3E6","b":"55D218113000","o":"2F383E6","s":"__cxa_throw","s+":"36"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D219038EDF","b":"55D218113000","o":"F25EDF","s":"_ZN5mongo13error_details23throwExceptionForStatusERKNS_6StatusE","s+":"1B23"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21904D10F","b":"55D218113000","o":"F3A10F","s":"_ZN5mongo21uassertedWithLocationERKNS_6StatusEPKcj","s+":"27B"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D218DAAE0B","b":"55D218113000","o":"C97E0B","s":"_ZN5mongo14FTDCController6doLoopEv.cold.395","s+":"2D"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D2195C736C","b":"55D218113000","o":"14B436C","s":"_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZN5mongo4stdx6threadC4IZNS3_14FTDCController5startEvEUlvE0_JELi0EEET_DpOT0_EUlvE_EEEEE6_M_runEv","s+":"5C"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"55D21B066D2F","b":"55D218113000","o":"2F53D2F","s":"execute_native_thread_routine","s+":"F"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"7F97F366CFA3","b":"7F97F3665000","o":"7FA3","s":"start_thread","s+":"F3"}}}
{"t":{"$date":"2023-08-19T02:25:01.112+00:00"},"s":"I",  "c":"CONTROL",  "id":31427,   "ctx":"ftdc","msg":"  Frame: {frame}","attr":{"frame":{"a":"7F97F359D4CF","b":"7F97F34A4000","o":"F94CF","s":"clone","s+":"3F"}}} 

 



 Comments   
Comment by Alison Rhea Thorne [ 03/Oct/23 ]

We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Comment by Alison Rhea Thorne [ 22/Sep/23 ]

We still need additional information to diagnose the problem. If this is still an issue for you, would you please archive (tar or zip) and upload to the secure upload portal:

  • the mongod logs
  • the $dbpath/diagnostic.data directory (the contents are described here)

Best,
Rhea

Comment by Alison Rhea Thorne [ 01/Sep/23 ]

Thank you for report,

Unfortunately, your report does not contain enough information for us to act upon. In order to begin diagnosing this problem, we will need further diagnostic data and logs. To facilitate this, I've created a secure upload portal for you. Files uploaded to this portal are hosted on Box, are visible only to MongoDB employees, and are routinely deleted after some time.

For each node in the replica set spanning a time period that includes the incident, would you please archive (tar or zip) and upload to that link:

  • the mongod logs
  • the $dbpath/diagnostic.data directory (the contents are described here)

 

Comment by Eduardo godoi dos santos [ 21/Aug/23 ]

I found this line inside the Log, I'm not sure about that, but I think this can be the backtrace problem.

{"t":{"$date":"2023-08-19T02:25:01.016+00:00"},"s":"F",  "c":"CONTROL",  "id":4757800, "ctx":"ftdc","msg":"Writing fatal message","attr":{"message":"DBException::toString(): FileNotOpen: Failed to open interim file /bitnami/mongodb/data/db/diagnostic.data/metrics.interim.temp\nActual exception type: mongo::error_details::ExceptionForImpl<(mongo::ErrorCodes::Error)38, mongo::AssertionException>\n"}} 

Generated at Thu Feb 08 06:43:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.