Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-33971

Nodes in MongoDB sharded cluster crashes with Invariant failure oplogEntry.getWallClockTime()

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 3.6.3
    • 3.6.4, 3.7.4
    • Sharding
    • Fully Compatible
    • ALL
    • v3.6

    Description

      We got strange error on our MongoDB cluster. It consists of 4 replicas with 3 nodes each.
      Today, during half an hour nodes from 3 replicas began to crash in chain. Primary crashes -> one of secondaries becames primary -> in 2 minutes it crashes too -> new node goes primary -> again crashes in some minutes and so on. Nodes have autorestart, and in 10-15 minutes of chaining crashes all is ok and works normal. To mention, balancer was disabled at that time.
      I've attached two parts of logs with assertion from 2 nodes of one replica. If you need more logs or info - just ask.

      We use ubuntu 16.04 on servers and docker containers for mongo.
      MongoDB version is 3.6.3

      Attachments

        1. rs0-1.1.log
          15 kB
        2. rs0-0.1.log
          10 kB

        Activity

          People

            jack.mulrow@mongodb.com Jack Mulrow
            Ubus Yuriy [X]
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: