Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-15381

Starting mongod with --upgrade and {--replSet, --master, or --configsvr} causes ReplCoordinator::shutdown to be called before startReplication, resulting in a crash

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.7.7
    • Affects Version/s: None
    • Component/s: Replication
    • None
    • Fully Compatible
    • ALL

      > ./mongod --upgrade --replSet mySet --oplogSize 1024
      2014-09-24T16:58:03.233-0400 I -        [initandlisten] MongoDB starting : pid=20413 port=27017 dbpath=/data/db 64-bit host=ubuntu
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] _DEBUG build (which is slower)
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] 
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] ** NOTE: This is a development version (2.7.6) of MongoDB.
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] **       Not recommended for production.
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] 
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] ** WARNING: You are running on a NUMA machine.
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] **          We suggest launching mongod like this to avoid performance problems:
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] **              numactl --interleave=all mongod [other options]
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] 
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] **        We suggest setting it to 'never'
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] 
      2014-09-24T16:58:03.234-0400 I -        [initandlisten] db version v2.7.6
      2014-09-24T16:58:03.235-0400 I -        [initandlisten] git version: a8828ca2f6d50eb9f183e3bc5d317a7e356083c7
      2014-09-24T16:58:03.235-0400 I -        [initandlisten] OpenSSL version: OpenSSL 1.0.1 14 Mar 2012
      2014-09-24T16:58:03.235-0400 I -        [initandlisten] build info: Linux ubuntu 3.13.0-35-generic #62~precise1-Ubuntu SMP Mon Aug 18 14:52:04 UTC 2014 x86_64 BOOST_LIB_VERSION=1_56
      2014-09-24T16:58:03.235-0400 I -        [initandlisten] allocator: tcmalloc
      2014-09-24T16:58:03.235-0400 I -        [initandlisten] options: { replication: { oplogSizeMB: 1024, replSet: "mySet" }, upgrade: true }
      2014-09-24T16:58:03.254-0400 I JOURNAL  [initandlisten] journal dir=/data/db/journal
      2014-09-24T16:58:03.254-0400 I JOURNAL  [initandlisten] recover begin
      2014-09-24T16:58:03.254-0400 I JOURNAL  [initandlisten] info no lsn file in journal/ directory
      2014-09-24T16:58:03.254-0400 I JOURNAL  [initandlisten] recover lsn: 0
      2014-09-24T16:58:03.254-0400 I JOURNAL  [initandlisten] recover /data/db/journal/j._0
      2014-09-24T16:58:03.255-0400 I JOURNAL  [initandlisten] Ending processFileBuffer at differing fileId want:4309420326219100146 got:15657314518909005299
      2014-09-24T16:58:03.255-0400 I JOURNAL  [initandlisten]   sect len:1038 seqnum:0
      2014-09-24T16:58:03.255-0400 I JOURNAL  [initandlisten] recover cleaning up
      2014-09-24T16:58:03.255-0400 I JOURNAL  [initandlisten] removeJournalFiles
      2014-09-24T16:58:03.296-0400 I JOURNAL  [initandlisten] recover done
      2014-09-24T16:58:03.296-0400 I JOURNAL  [initandlisten] preallocating a journal file /data/db/journal/prealloc.0
      2014-09-24T16:58:03.956-0400 I STORAGE  [initandlisten] opening db: test
      2014-09-24T16:58:03.968-0400 I STORAGE  [initandlisten] opening db: admin
      2014-09-24T16:58:03.978-0400 I STORAGE  [initandlisten] opening db: local
      2014-09-24T16:58:03.988-0400 I STORAGE  [initandlisten] finished checking dbs
      mongod: src/third_party/boost-1.56.0/boost/smart_ptr/scoped_ptr.hpp:99: T* boost::scoped_ptr<T>::operator->() const [with T = boost::thread]: Assertion `px != 0' failed.
      2014-09-24T16:58:04.002-0400 F -        [initandlisten] Got signal: 6 (Aborted).
      
       0x1dec5f2 0x1debb87 0x1debc1c 0x7f259d5c2150 0x7f259d5c20d5 0x7f259d5c583b 0x7f259d5bad9e 0x7f259d5bae42 0x199f23f 0x1a0996f 0x1a063d0 0x17c5781 0x13bb8d9 0x13bbd4c 0x13bd019 0x13bc05f 0x7f259d5ad76d 0x13b9089
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"400000","o":"19EC5F2"},{"b":"400000","o":"19EBB87"},{"b":"400000","o":"19EBC1C"},{"b":"7F259D58C000","o":"36150"},{"b":"7F259D58C000","o":"360D5"},{"b":"7F259D58C000","o":"3983B"},{"b":"7F259D58C000","o":"2ED9E"},{"b":"7F259D58C000","o":"2EE42"},{"b":"400000","o":"159F23F"},{"b":"400000","o":"160996F"},{"b":"400000","o":"16063D0"},{"b":"400000","o":"13C5781"},{"b":"400000","o":"FBB8D9"},{"b":"400000","o":"FBBD4C"},{"b":"400000","o":"FBD019"},{"b":"400000","o":"FBC05F"},{"b":"7F259D58C000","o":"2176D"},{"b":"400000","o":"FB9089"}],"processInfo":{ "mongodbVersion" : "2.7.6", "gitVersion" : "a8828ca2f6d50eb9f183e3bc5d317a7e356083c7", "uname" : { "sysname" : "Linux", "release" : "3.13.0-35-generic", "version" : "#62~precise1-Ubuntu SMP Mon Aug 18 14:52:04 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "DB75160A809F116A7C949B13A2530E9A96512A3B" }, { "b" : "7FFF9A2A8000", "elfType" : 3, "buildId" : "392CD1376FBB587C2CAC343245E09722F687D67C" }, { "b" : "7F259EBA2000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "C340AF9DEE97C17C730F7D03693286C5194A46B8" }, { "b" : "7F259E944000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "73FB5D3902679C4018AD92457B73F92C2F6B9C30" }, { "b" : "7F259E569000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "958CBDCD514DFBE79DBDC086E10100F3B6B56A49" }, { "b" : "7F259E361000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "352C5B373A50E6C4AB881A5DB6F5766FDF81EEE0" }, { "b" : "7F259E15D000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D181AF551DBBC43E9D55913D532635FDE18E7C4E" }, { "b" : "7F259DE5D000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "B534DA725D06A04267EB2FEB92B9CC14C838B57B" }, { "b" : "7F259DB61000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "817AA99B3DD02501F8BC04A3E9A9358A08F20D7D" }, { "b" : "7F259D94B000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "ECF322A96E26633C5D10F18215170DD4395AF82C" }, { "b" : "7F259D58C000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "E7863FF2EA3F4B5C6FF90689ADE4D6B5F3A78020" }, { "b" : "7F259EDBF000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "E25AD1A11CCF57E734116B8EC9C69F643DCA9F18" }, { "b" : "7F259D375000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "F695ECFCF3918D5D34989398A14B7ECDD9F46CD0" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x27) [0x1dec5f2]
       mongod(+0x19EBB87) [0x1debb87]
       mongod(+0x19EBC1C) [0x1debc1c]
       libc.so.6(+0x36150) [0x7f259d5c2150]
       libc.so.6(gsignal+0x35) [0x7f259d5c20d5]
       libc.so.6(abort+0x17B) [0x7f259d5c583b]
       libc.so.6(+0x2ED9E) [0x7f259d5bad9e]
       libc.so.6(+0x2EE42) [0x7f259d5bae42]
       mongod(_ZNK5boost10scoped_ptrINS_6threadEEptEv+0x37) [0x199f23f]
       mongod(_ZN5mongo4repl26ReplicationCoordinatorImpl8shutdownEv+0x135) [0x1a0996f]
       mongod(_ZN5mongo4repl28HybridReplicationCoordinator8shutdownEv+0x2E) [0x1a063d0]
       mongod(_ZN5mongo11exitCleanlyENS_8ExitCodeEPNS_16OperationContextE+0x76) [0x17c5781]
       mongod(+0xFBB8D9) [0x13bb8d9]
       mongod(_ZN5mongo13initAndListenEi+0x1F) [0x13bbd4c]
       mongod(+0xFBD019) [0x13bd019]
       mongod(main+0x28) [0x13bc05f]
       libc.so.6(__libc_start_main+0xED) [0x7f259d5ad76d]
       mongod(+0xFB9089) [0x13b9089]
      -----  END BACKTRACE  -----
      

            Assignee:
            spencer@mongodb.com Spencer Brody (Inactive)
            Reporter:
            spencer@mongodb.com Spencer Brody (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: