Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-50880

Mongod Server Failed with signal 6

    • Type: Icon: Bug Bug
    • Resolution: Community Answered
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.4.0
    • Component/s: WiredTiger
    • Labels:
    • ALL

      Let me first explain the scenario first!

      I've installed mongo 3.6 and it was running fine but then I decided to upgrade, so I just took the backup using mongo tools. I removed and purged all the mongo packages and reinstalled the latest version. Restored the backup edited few changes in unit file and conf file.

      [Unit] 
      Description=MongoDB Database Server 
      Documentation=https://docs.mongodb.org/manual 
      After=network.target 
       
      [Service] 
      User=mongodb 
      Group=mongodb 
      EnvironmentFile=-/etc/default/mongod 
      ExecStart=/usr/bin/mongod --config /etc/mongod.conf 
      PIDFile=/run/mongodb/mongod.pid 
      Restart=always 
      RestartSec=1 
      # file size 
      LimitFSIZE=infinity 
      # cpu time 
      LimitCPU=infinity 
      # virtual memory size 
      LimitAS=infinity 
      # open files 
      LimitNOFILE=64000 
      # processes/threads 
      LimitNPROC=64000 
      # locked memory 
      LimitMEMLOCK=65536 
      # total threads (user+kernel) 
      TasksMax=infinity 
      TasksAccounting=false 
       
      # Recommended limits for mongod as specified in 
      # https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings 
       
      [Install] 
      WantedBy=multi-user.target

      I was running two mongo instances, copied everything separately (I'm sure, I did this perfectly) as it was the same when mongo 3.6 was running.

      Now, sometimes my mongod instance is killed:

      Sep 11 09:28:05 mlnode systemd[1]: mongod.service: Main process exited, code=killed, status=6/ABRT
      Sep 11 09:28:05 mlnode systemd[1]: mongod.service: Failed with result 'signal'.
      

      I've looked into the mongo logs as well but I was unable to understand what is the exact reason also I've looked at other Jira threads which says maybe this is an issue of  MEMLOCK limits, I've corrected that still, the problem persists.

      {"t":{"$date":"2020-09-11T09:25:58.708+05:30"},"s":"F",  "c":"-",        "id":23083,   "ctx":"conn264761","msg":"In
      variant failure","attr":{"expr":"ret","error":"UnknownError: -31803: WT_NOTFOUND: item not found","file":"src/mongo
      /db/storage/wiredtiger/wiredtiger_record_store.cpp","line":1598}}
      
      {"t":{"$date":"2020-09-11T09:25:58.708+05:30"},"s":"F",  "c":"-",        "id":23084,   "ctx":"conn264761","msg":"\n
      \n***aborting after invariant() failure\n\n"}
      
      {"t":{"$date":"2020-09-11T09:25:58.708+05:30"},"s":"F",  "c":"CONTROL",  "id":4757800, "ctx":"conn264761","msg":"Wr
      iting fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
      

      I want urgent help please! If anyone wants additional information then I'm ready to provide ASAP.

      Thanks

            Assignee:
            dmitry.agranat@mongodb.com Dmitry Agranat
            Reporter:
            anandeshsharma@gmail.com Anandesh Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: