Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-52935

Failed to start node of Replica Set

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • None
    • None
    • None
    • ALL

    Description

      Hello,

      MongoDB version v3.2.6

       

      I have MongoDB Replica Set with 3 members (primary, secondary and arbiter). primary node failed and after failover secondary became primary. after that I tried to start mongod process on failed node and receiving following Error:  "ERROR: child process failed, exited with error number 51".  

       

      there is content of log file: 
      E STORAGE [WT RecordStoreThread: local.oplog.rs] WiredTiger (0) [1605537697:593366][3880:0x7fc823621700], file:collection-6--87924298142942209182.wt, WT_SESSION.truncate: read checksum error for 4096B block at offset 18014208: block header checksum of 1701601889 doesn't match expected checksum of 2449756693
      E STORAGE [WT RecordStoreThread: local.oplog.rs] WiredTiger (0) [1605537698:967296][3880:0x7fc823621700], file:collection-6--87924298142942209182.wt, WT_SESSION.truncate: collection-6--87924298142942209182.wt: encountered an illegal file format or internal value
      E STORAGE [WT RecordStoreThread: local.oplog.rs] WiredTiger (-31804) [1605537698:967317][3880:0x7fc823621700], file:collection-6--87924298142942209182.wt, WT_SESSION.truncate: the process must exit and restart: WT_PANIC: WiredTiger library panic
      I - [WT RecordStoreThread: local.oplog.rs] Fatal Assertion 28558

       

       

      I tried initial resync, too many hours, I receveid Error: 

      W REPL [rsBackgroundSync] we are too stale to use hostname_pr_node:port_pr_node as a sync source
      I REPL [ReplicationExecutor] could not find member to sync from
      E REPL [rsBackgroundSync] too stale to catch up – entering maintenance mode
      I REPL [rsBackgroundSync] our last optime : (term: -1, timestamp: Nov 16 18:41:34:1)
      I REPL [rsBackgroundSync] oldest available is (term: -1, timestamp: Nov 16 19:17:21:34b)
      I REPL [rsBackgroundSync] See http://dochub.mongodb.org/core/resyncingaverystalereplicasetmember

       

      I tried change oplog size, but I have only two nodes (primary and arbiter) and  I can't shutdown primary node.

      Thank you

      Attachments

        Activity

          People

            edwin.zhou@mongodb.com Edwin Zhou
            gdvalishvili01@gmail.com Giorgi Dvalishvili
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: