Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-2822

secondary cannot recover after failure

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Blocker - P1 Blocker - P1
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:
      v.1.8.0
    • ALL

      two of our 80 mongo servers has been hanged for a while, then machine was restarted.
      now secondary cannot repair itself

      please help

      Wed Mar 23 01:10:39 [initandlisten] Assertion: 10334:Invalid BSONObj size: 1845624949 (0x7500026E) first element: : ?type=115
      0x562059 0x4ee45e 0x7840d8 0x64cfad 0x751db3 0x75b387 0x75c101 0x59b1dc 0x59b6f8 0x725981 0x726e04 0x72811d 0x781cda 0x8a7d48 0x8a8c84 0x8a9fcc 0x8aab98 0x8b1b8c 0x2ad8679378a4 0x4e0ff9
      /usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x129) [0x562059]
      /usr/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x46e) [0x4ee45e]
      /usr/bin/mongod(_ZN5mongo11BasicCursor7currentEv+0x68) [0x7840d8]
      /usr/bin/mongod(_ZN5mongo14processGetMoreEPKcixRNS_5CurOpEiRb+0x58d) [0x64cfad]
      /usr/bin/mongod(_ZN5mongo15receivedGetMoreERNS_10DbResponseERNS_7MessageERNS_5CurOpE+0x1f3) [0x751db3]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x14f7) [0x75b387]
      /usr/bin/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x81) [0x75c101]
      /usr/bin/mongod(_ZN5mongo14DBClientCursor11requestMoreEv+0x30c) [0x59b1dc]
      /usr/bin/mongod(_ZN5mongo14DBClientCursor4moreEv+0x58) [0x59b6f8]
      /usr/bin/mongod(_ZN5mongo6Cloner4copyEPKcS2_bbbbNS_5QueryE+0x5c1) [0x725981]
      /usr/bin/mongod(_ZN5mongo6Cloner2goEPKcRSsRKSsbbbb+0xdf4) [0x726e04]
      /usr/bin/mongod(_ZN5mongo9cloneFromEPKcRSsRKSsbbbb+0x3d) [0x72811d]
      /usr/bin/mongod(_ZN5mongo14repairDatabaseESsRSsbb+0x43a) [0x781cda]
      /usr/bin/mongod(_ZN5mongo11doDBUpgradeERKSsSsPNS_14DataFileHeaderE+0x68) [0x8a7d48]
      /usr/bin/mongod [0x8a8c84]
      /usr/bin/mongod(_ZN5mongo14_initAndListenEiPKc+0x41c) [0x8a9fcc]
      /usr/bin/mongod(_ZN5mongo13initAndListenEiPKc+0x18) [0x8aab98]
      /usr/bin/mongod(main+0x6e8c) [0x8b1b8c]
      /lib64/libc.so.6(__libc_start_main+0xf4) [0x2ad8679378a4]
      /usr/bin/mongod(__gxx_personality_v0+0x3b1) [0x4e0ff9]
      Wed Mar 23 01:10:39 [initandlisten] getmore local.oplog.rs cid:3581083591747888217 getMore: {} exception 10334 Invalid BSONObj size: 1845624949 (0x7500026E) first element: : ?type=115 bytes:20 nreturned:0 75ms
      Wed Mar 23 01:10:39 [initandlisten] exception in initAndListen std::exception: getMore: cursor didn't exist on server, possible restart or timeout?, terminating

            Assignee:
            kristina Kristina Chodorow (Inactive)
            Reporter:
            ofersa ofer samocha
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: