• Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 2.0.6
    • Component/s: Internal Code
    • None
    • Environment:
      RHEL5
    • Linux

      Server has been running well and had an uptime of 60 days or so. Then suddenly a crash. Mo ngod.log shows:

      {{

          • glibc detected *** /usr/bin/mongod: realloc(): invalid next size: 0x00002b394417f4e0 ***
            ======= Backtrace: =========
            /lib64/libc.so.6[0x3615e73214]
            /lib64/libc.so.6(realloc+0x102)[0x3615e73d02]
            /usr/bin/mongod(_ZN5mongo11_BufBuilderINS_16TrivialAllocatorEE15grow_reallocateEv+0x47)[0x5055c7]
            /usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_13StringBuilderEbb+0x26e)[0x50b10e]
            /usr/bin/mongod(_ZNK5mongo11BSONElement8toStringERNS_13StringBuilderEbb+0x5c0)[0x509810]
            /usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_13StringBuilderEbb+0x1d2)[0x50b072]
            /usr/bin/mongod(_ZNK5mongo11BSONElement8toStringERNS_13StringBuilderEbb+0x1185)[0x50a3d5]
            /usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_13StringBuilderEbb+0x1d2)[0x50b072]
            /usr/bin/mongod(_ZNK5mongo7OpDebug8toStringEv+0x1f3)[0x894bc3]
            /usr/bin/mongod(_ZNK5mongo14LazyStringImplINS_7OpDebugEE3valEv+0xd)[0x89148d]
            /usr/bin/mongod(_ZN5mongo9LogstreamlsERKNS_10LazyStringE+0x1a)[0x50bdca]
            /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x954)[0x88e7c4]
            /usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x78)[0xaa0cc8]
            /usr/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287)[0x638997]
            /lib64/libpthread.so.0[0x383e60677d]
            /lib64/libc.so.6(clone+0x6d)[0x3615ed325d]
            ======= Memory map: ========
            00400000-00c26000 r-xp 00000000 fd:02 98967 /usr/bin/mongod
            00e26000-00e46000 rw-p 00826000 fd:02 98967 /usr/bin/mongod
            00e46000-0184c000 rw-p 00e46000 00:00 0
            01a45000-01ae2000 rw-p 00845000 fd:02 98967 /usr/bin/mongod
            0cc59000-184f9000 rw-p 0cc59000 00:00 0 [heap]
            4031d000-4031e000 ---p 4031d000 00:00 0
            4031e000-4041e000 rw-p 4031e000 00:00 0
            40691000-40692000 ---p 40691000 00:00 0
            40692000-40792000 rw-p 40692000 00:00 0
            40c72000-40c73000 ---p 40c72000 00:00 0
            40c73000-40d73000 rw-p 40c73000 00:00 0
            40d73000-40d74000 ---p 40d73000 00:00 0
            40d74000-40e74000 rw-p 40d74000 00:00 0
            40e74000-40e75000 ---p 40e74000 00:00 0

      }}

      and later...

      {{

      Tue Jul 17 17:19:20 Got signal: 6 (Aborted).

      Tue Jul 17 17:19:20 Backtrace:
      0xa95df9 0x3615e302f0 0x3615e30285 0x3615e31d30 0x3615e692bb 0x3615e73214 0x3615e73d02 0x5055c7 0x50b10e 0x509810 0x50b072 0x50a3d5 0x50b072 0x894bc3 0x89148d 0x50bdca 0x88e7c4 0xaa0cc8 0x638997 0x383e60677d
      /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x3a9) [0xa95df9]
      /lib64/libc.so.6 [0x3615e302f0]
      /lib64/libc.so.6(gsignal+0x35) [0x3615e30285]
      /lib64/libc.so.6(abort+0x110) [0x3615e31d30]
      /lib64/libc.so.6 [0x3615e692bb]
      /lib64/libc.so.6 [0x3615e73214]
      /lib64/libc.so.6(realloc+0x102) [0x3615e73d02]
      /usr/bin/mongod(_ZN5mongo11_BufBuilderINS_16TrivialAllocatorEE15grow_reallocateEv+0x47) [0x5055c7]
      /usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_13StringBuilderEbb+0x26e) [0x50b10e]
      /usr/bin/mongod(_ZNK5mongo11BSONElement8toStringERNS_13StringBuilderEbb+0x5c0) [0x509810]
      /usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_13StringBuilderEbb+0x1d2) [0x50b072]
      /usr/bin/mongod(_ZNK5mongo11BSONElement8toStringERNS_13StringBuilderEbb+0x1185) [0x50a3d5]
      /usr/bin/mongod(_ZNK5mongo7BSONObj8toStringERNS_13StringBuilderEbb+0x1d2) [0x50b072]
      /usr/bin/mongod(_ZNK5mongo7OpDebug8toStringEv+0x1f3) [0x894bc3]
      /usr/bin/mongod(_ZNK5mongo14LazyStringImplINS_7OpDebugEE3valEv+0xd) [0x89148d]
      /usr/bin/mongod(_ZN5mongo9LogstreamlsERKNS_10LazyStringE+0x1a) [0x50bdca]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x954) [0x88e7c4]
      /usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x78) [0xaa0cc8]
      /usr/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287) [0x638997]
      /lib64/libpthread.so.0 [0x383e60677d]

      Tue Jul 17 17:19:20 Invalid access at address: 0x4

      Tue Jul 17 17:19:20 Got signal: 11 (Segmentation fault).

      Tue Jul 17 17:19:20 Backtrace:
      0xa95df9 0xa9651c 0x383e60ebe0 0x54ff42 0x55a35f 0x55b281 0x55bb54 0x8e0e85 0x9428b9 0x889767 0x88e3c9 0xaa0cc8 0x638997 0x383e60677d 0x3615ed325d
      /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x3a9) [0xa95df9]
      /usr/bin/mongod(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x22c) [0xa9651c]
      /lib64/libpthread.so.0 [0x383e60ebe0]
      /usr/bin/mongod(_ZN5mongo10FieldRangeC1ERKNS_11BSONElementEbbb+0x1a2) [0x54ff42]
      /usr/bin/mongod(_ZN5mongo13FieldRangeSet16processOpElementEPKcRKNS_11BSONElementEbb+0x7f) [0x55a35f]
      /usr/bin/mongod(_ZN5mongo13FieldRangeSet17processQueryFieldERKNS_11BSONElementEb+0x4a1) [0x55b281]
      /usr/bin/mongod(_ZN5mongo13FieldRangeSetC1EPKcRKNS_7BSONObjEbb+0x194) [0x55bb54]
      /usr/bin/mongod(_ZN5mongo16MultiPlanScannerC1EPKcRKNS_7BSONObjES5_PKNS_11BSONElementEbS5_S5_bb+0x205) [0x8e0e85]
      /usr/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0xbe9) [0x9428b9]
      /usr/bin/mongod [0x889767]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x559) [0x88e3c9]
      /usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x78) [0xaa0cc8]
      /usr/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287) [0x638997]
      /lib64/libpthread.so.0 [0x383e60677d]
      /lib64/libc.so.6(clone+0x6d) [0x3615ed325d]

      Logstream::get called in uninitialized state
      Tue Jul 17 17:19:20 [conn15118] query jerome5.Unit query: { wpm:

      { $gte: 0.7059999704360962 }

      , f:

      { $gte: 0 }

      , t:

      { $in: [ "un cadre", "digitale sans", "Excerpt", "fil", "communiquant son", "photo numérique", "photo digitale", "recherche un", "cadre", "propose", "fil et", "pour", "qui recherche", "cadre photo", "qui", "numérique", "et communicant", "communicant", "propose pour", "numérique communiquant", "Telefunken propose", "son", "DPF", "un", "photo", "recherche", "sans", "sans fil", "ceux", "communiquant", "Telefunken", "Wifi", "et", "tous", "digitale" ] }

      , lc:

      { $in: [ "ukr", "slk", "cat", "zho", "deu", "fra", "por", "fin", "hin", "ces", "slv", "nld", "kor", "est", "jap", "rus", "ara", "pol", "eng", "vie", "sve", "esl", "hun", "isl", "lav", "ell", "tur", "bel", "nor", "bul", "dan", "sqi", "ita", "mkd" ] }

      } nscanned:371 nreturned:101 reslen:5961 634ms
      Logstream::get called in uninitialized state
      Tue Jul 17 17:19:20 [conn15093] ERROR: Client::~Client _context should be null but is not; client:conn
      Logstream::get called in uninitialized state
      Tue Jul 17 17:19:20 [conn15093] ERROR: Client::shutdown not called: conn
      }}

            Assignee:
            milkie@mongodb.com Eric Milkie
            Reporter:
            niccottrell Nic Cottrell (Personal)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: