Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-9362

Mongod crashes at first getnonce call if process started without stdout and stderr file descriptors initialized.

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • 2.5.4
    • Affects Version/s: 2.4.1
    • Component/s: Internal Code
    • Labels:
      None
    • Environment:
      EC2, Amazon Linux AMI release 2013.03, MongoDB 2.4.1 or MongoDB master branch
    • Linux

      When initiating the replica set from the driver one of the three members of the set will always crash with the following stack trace (built with --d):

      # Stack trace from master as of today
Tue Apr 16 22:34:16.644 Backtrace:
0x10f29db 0xc21a20 0x7f0563f649c0 0x7f0563f64945 0x7f0563f6625b 0xfec39c 0xd45867 0xd456f7 0xd98639 0xd9936a 0xd99ce1 0xebafce 0xebf988 0xe6a26f 0xe6b19c 0xc29e81 0x10e312f 0x7f0564c6cc6b 0x7f05640125ed 
 /home/tyler/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x27) [0x10f29db]
 /home/tyler/mongo/mongod(_ZN5mongo10abruptQuitEi+0x110) [0xc21a20]
 /lib64/libc.so.6(+0x349c0) [0x7f0563f649c0]
 /lib64/libc.so.6(gsignal+0x35) [0x7f0563f64945]
 /lib64/libc.so.6(abort+0x17b) [0x7f0563f6625b]
 /home/tyler/mongo/mongod(_ZN5mongo23InputStreamSecureRandom9nextInt64Ev+0x56) [0xfec39c]
 /home/tyler/mongo/mongod(_ZN5mongo11CmdGetNonce12getNextNonceEv+0x43) [0xd45867]
 /home/tyler/mongo/mongod(_ZN5mongo11CmdGetNonce3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x4f) [0xd456f7]
 /home/tyler/mongo/mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x8c) [0xd98639]
 /home/tyler/mongo/mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xacc) [0xd9936a]
 /home/tyler/mongo/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x396) [0xd99ce1]
 /home/tyler/mongo/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x46) [0xebafce]
 /home/tyler/mongo/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x392) [0xebf988]
 /home/tyler/mongo/mongod() [0xe6a26f]
 /home/tyler/mongo/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x2e8) [0xe6b19c]
 /home/tyler/mongo/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xe3) [0xc29e81]
 /home/tyler/mongo/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4de) [0x10e312f]
 /lib64/libpthread.so.0(+0x7c6b) [0x7f0564c6cc6b]
 /lib64/libc.so.6(clone+0x6d) [0x7f05640125ed]

      This does NOT happen locally on Mac OS X when using the binary or building from source but does happen using on Amazon Linux on EC2 using master or a binary.

      I tried to reproduce on a single mongod by writing a program which runs the {{

      {getnonce: 1}

      }} many times but I was unable to crash mongod in that way.

      Running addr2line on the stack trace built with --d yields line 130 in src/mongo/platform/random.cpp as the source:

      int64_t nextInt64() {  
  int64_t r;                                                                                                                 
  _in->read( reinterpret_cast<char*>( &r ), sizeof(r) );    
  if ( _in->fail() ) { 
    abort();
  }
  return r;
}

            Assignee:
            schwerin@mongodb.com Andy Schwerin
            Reporter:
            tyler@10gen.com Tyler Brock
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: