[SERVER-53025] Mongod primary node in replica set crashed with error "std::exception::what(): Resource temporarily unavailable" Created: 22/Nov/20  Updated: 08/Dec/20  Resolved: 08/Dec/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.0.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Dineshkumar Natarajan Assignee: Dmitry Agranat
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

QA


Participants:

 Description   

2020-11-22T13:36:51.552+0400 I REPL_HB [replexec-452] Error in heartbeat (requestId: 4614325) to akppmgd2:27019, response status: HostUnreachable: Error con
necting to akppmgd2:27019 (10.72.32.12:27019) :: caused by :: Connection refused
2020-11-22T13:36:51.562+0400 F - [replexec-452] terminate() called. An exception is active; attempting to gather more information
2020-11-22T13:36:51.612+0400 F - [replexec-452] std::exception::what(): Resource temporarily unavailable
Actual exception type: std::system_error
 0x55a6e5190061 0x55a6e518fa45 0x55a6e5284936 0x55a6e5284981 0x55a6e4986725 0x55a6e4986df0 0x55a6e4a1142a 0x55a6e4a118a2 0x55a6e4a16149 0x55a6e4a16263 0x55a6
e3b2176e 0x55a6e3b22d5b 0x55a6e4a0f688 0x55a6e4a10373 0x55a6e4a10a5b 0x55a6e4982b6c 0x55a6e498306c 0x55a6e4983a56 0x55a6e529fb60 0x7fa334590e25 0x7fa3342be34
d
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"55A6E2E08000","o":"2388061","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55A6E2E08000","o":"2387A45"},{"b":"55A6E2E08000","o":"247C936","s
":"_ZN10__cxxabiv111__terminateEPFvvE"},{"b":"55A6E2E08000","o":"247C981"},{"b":"55A6E2E08000","o":"1B7E725","s":"_ZN5mongo10ThreadPool25_startWorkerThread_i
nlockEv"},{"b":"55A6E2E08000","o":"1B7EDF0","s":"_ZN5mongo10ThreadPool8scheduleESt8functionIFvvEE"},{"b":"55A6E2E08000","o":"1C0942A","s":"_ZN5mongo8executor
22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS1_13CallbackStateEESaIS6_EEERKSt14_List_iteratorIS6_ESD_St11unique_lockI
St5mutexE"},{"b":"55A6E2E08000","o":"1C098A2","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor23scheduleIntoPool_inlockEPNSt7__cxx114listISt10shared_ptrINS1_1
3CallbackStateEESaIS6_EEESt11unique_lockISt5mutexE"},{"b":"55A6E2E08000","o":"1C0E149","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor12scheduleWorkERKSt8fun
ctionIFvRKNS0_12TaskExecutor12CallbackArgsEEE"},{"b":"55A6E2E08000","o":"1C0E263","s":"_ZN5mongo8executor22ThreadPoolTaskExecutor14scheduleWorkAtENS_6Date_tE
RKSt8functionIFvRKNS0_12TaskExecutor12CallbackArgsEE

 

replica set.

OS - redhat 7

version - server-4.0.1

 

 

 

[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /etc/mongod.conf"
EnvironmentFile=-/etc/sysconfig/mongod
ExecStart=/usr/bin/mongod $OPTIONS
ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb
ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb
ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb
PermissionsStartOnly=true
PIDFile=/var/run/mongodb/mongod.pid
Type=forking

  1. file size
    LimitFSIZE=infinity
  2. cpu time
    LimitCPU=infinity
  3. virtual memory size
    LimitAS=infinity
  4. open files
    LimitNOFILE=64000
  5. processes/threads
    LimitNPROC=64000
  6. locked memory
    LimitMEMLOCK=infinity
  7. total threads (user+kernel)
    TasksMax=infinity
    TasksAccounting=false

 



 Comments   
Comment by Dmitry Agranat [ 08/Dec/20 ]

Hi joydinesh07@gmail.com,

We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket.

Regards,
Dima

Comment by Dmitry Agranat [ 02/Dec/20 ]

Hi joydinesh07@gmail.com, if this is still a problem for you, we will need the data as described in my last comment.

Comment by Dmitry Agranat [ 23/Nov/20 ]

Hi joydinesh07@gmail.com,

Would you please archive (tar or zip) the full mongod.log files and the $dbpath/diagnostic.data directory (the contents are described here) and upload them to this support uploader location?

Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time.

Also, please post the current value of vm.max_map_count for the mongod process.

Thanks,
Dima

Generated at Thu Feb 08 05:29:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.