Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-101749

Segmentation fault after mongod upgrade from 5.0.23 to 8.0.4

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 8.0.4
    • Component/s: None
    • None
    • Cluster Scalability
    • ALL
    • Cluster Scalability 2025-03-17

      We already faced with that problem twice. Our setup is shard cluster with 3 mongos, 3 config servers, and 4 shard 3 members replica sets. Master shard is sometimes fail with  segmentation fault error.
      Errors from log

        },
        "s": "F",
        "c": "CONTROL",
        "id": 6384300,
        "svc": "S",
        "ctx": "conn240213",
        "msg": "Writing fatal message",
        "attr": {
          "message": "Invalid access at address: 0x7f5b0a915fe8\n"
        }
      }
      {
        "t": {
          "$date": "2025-03-02T22:20:09.706+00:00"
        },
        "s": "F",
        "c": "CONTROL",
        "id": 6384300,
        "svc": "S",
        "ctx": "conn240213",
        "msg": "Writing fatal message",
        "attr": {
          "message": "Dumping siginfo (si_code=2): 0b 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 e8 5f 91 0a 5b 7f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n"
        }
      }
      {
        "t": {
          "$date": "2025-03-02T22:20:09.706+00:00"
        },
        "s": "F",
        "c": "CONTROL",
        "id": 6384300,
        "svc": "S",
        "ctx": "conn240213",
        "msg": "Writing fatal message",
        "attr": {
          "message": "Got signal: 11 (Segmentation fault).\n"
        }
      } 

      Before in both cases there was migration command

      "t": {
          "$date": "2025-03-02T22:19:30.895+00:00"
        },
        "s": "I",
        "c": "COMMAND",
        "id": 51803,
        "svc": "S",
        "ctx": "conn240213",
        "msg": "Slow query",
        "attr": {
          "type": "command",
          "isFromUserConnection": true,
          "ns": "admin.$cmd",
          "collectionType": "admin",
          "command": {
            "_getNextSessionMods": 1,
            "sessionId": "shard-c_shard-b_67c4d96ca6e2e7f003df4726",
            "$clusterTime": {
              "clusterTime": {
                "$timestamp": {
                  "t": 1740953964,
                  "i": 5582
                }
              }
            },
            "$configTime": {
              "$timestamp": {
                "t": 1740953964,
                "i": 2595
              }
            },
            "$topologyTime": {
              "$timestamp": {
                "t": 1720009389,
                "i": 1
              }
            },
            "mayBypassWriteBlocking": false,
            "$db": "admin"
          },
          "numYields": 0,
          "reslen": 157,
          "locks": {
            "ReplicationStateTransition": {
              "acquireCount": {
                "w": 204
              }
            },
            "Global": {
              "acquireCount": {
                "r": 408
              }
            },
            "Database": {
              "acquireCount": {
                "r": 204
              }
            },
            "Collection": {
              "acquireCount": {
                "r": 204
              }
            },
            "Mutex": {
              "acquireCount": {
                "r": 612
              }
            }
          },
          "storage": {
            "data": {
              "bytesRead": 246444,
              "timeReadingMicros": 21
            }
          },
          "cpuNanos": 214485606,
          "remote": "*:37830",
          "protocol": "op_msg",
          "queues": {
            "ingress": {
              "admissions": 205
            },
            "execution": {
              "admissions": 409
            }
          },
          "workingMillis": 5988,
          "durationMillis": 5988
        }
      } 

      Backtraces in attachments

        1. backtrace-shard-d.json
          35 kB
        2. backtrace-shard-c.json
          34 kB

            Assignee:
            janna.golden@mongodb.com Janna Golden
            Reporter:
            cshgsergey@gmail.com Sergei Sh
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: