Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-11871

Bug in v8 memory allocator that causes long-running shell to crash

    • Fully Compatible
    • ALL

      Issue Status as of March 28, 2014

      ISSUE SUMMARY
      A bug in V8 memory allocation causes a long-running mongo shell to crash.

      USER IMPACT
      Long-running tasks using the mongo shell or Javascript server-side (like map/reduce or eval command) could be aborted before they finish.

      SOLUTION
      Apply the patch that fixed the issue upstream in the V8 code.

      WORKAROUNDS
      Avoid running operations in the mongo shell that take several hours or longer.

      AFFECTED VERSIONS
      All production releases from version 2.4.0 to 2.4.9 are affected.

      PATCHES
      The fix is included in the 2.4.10 production release and the 2.5.5 development release, which will evolve into the 2.6.0 production release.

      Original Description

      While running a long-running shell that was was doing a simple operation in a loop, the shell crashed with a V8 error.

      Here's what I was running:

      > for (var j = 0 ; true; j++){ if (j%500 == 0) {print("still going");}; db.eval("db.currentOp();")}
      

      This ran fine for several hours before crashing like this:

      #
      # Fatal error in src/third_party/v8/src/spaces-inl.h, line 167
      # CHECK(chunk->size() <= static_cast<size_t>(kPageSize)) failed
      #
      
      
      ==== Stack trace ============================================
      
      Security context: 0x2e5b44206361 <JS Object>#0#
          2: find [src/mongo/shell/collection.js:~158] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,limit=-1,skip=0,batchSize=0,options=0x2e5b44204121 <undefined>)
          3: findOne [src/mongo/shell/collection.js:~171] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,options=0x2e5b44204121 <undefined>)
          4: arguments adaptor frame: 1->3
          5: runCommand [src/mongo/shell/db.js:~47] (this=0x3d5d27d8c7f9 <a DB>#3#,obj=0x2f7bc94c91f9 <an Object>#2#)
          6: eval [src/mongo/shell/db.js:393] (this=0x3d5d27d8c7f9 <a DB>#3#,jsfunction=0x16237b856c09 <String[15]: db.currentOp();>)
          7: /* anonymous */(aka /* anonymous */) [(shell):~1] (this=0)
      
      ==== Details ================================================
      
      [2]: find [src/mongo/shell/collection.js:~158] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,limit=-1,skip=0,batchSize=0,options=0x2e5b44204121 <undefined>) {
      // optimized frame
      }
      [3]: findOne [src/mongo/shell/collection.js:~171] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,options=0x2e5b44204121 <undefined>) {
      // optimized frame
      }
      [4]: arguments adaptor frame: 1->3 {
        // actual arguments
        [00] : 0x2f7bc94c91f9 <an Object>#2#
      }
      
      [5]: runCommand [src/mongo/shell/db.js:~47] (this=0x3d5d27d8c7f9 <a DB>#3#,obj=0x2f7bc94c91f9 <an Object>#2#) {
      // optimized frame
      }
      [6]: eval [src/mongo/shell/db.js:393] (this=0x3d5d27d8c7f9 <a DB>#3#,jsfunction=0x16237b856c09 <String[15]: db.currentOp();>) {
        // stack-allocated locals
        var cmd = 0x2f7bc94c91f9 <an Object>#2#
        var arguments = 0x2f7bc94c9191 <an Arguments>#4#
        var res = 0x2e5b44204121 <undefined>
        // heap-allocated locals
        var jsfunction = 0x16237b856c09 <String[15]: db.currentOp();>
        // expression stack (top to bottom)
        [04] : 0x2f7bc94c91f9 <an Object>#2#
        [03] : 0x3d5d27d8c7f9 <a DB>#3#
      --------- s o u r c e   c o d e ---------
      function (jsfunction) {?    var cmd = { $eval : jsfunction };?    if ( arguments.length > 1 ) {?        cmd.args = argumentsToArray( arguments ).slice(1);?    }??    var res = this._dbCommand( cmd );??    if (!res.ok)?        throw tojson( res );??    return res.retval;?}
      -----------------------------------------
      }
      
      [7]: /* anonymous */(aka /* anonymous */) [(shell):~1] (this=0) {
      // optimized frame
      }
      ==== Key         ============================================
      
       #0# 0x2e5b44206361: 0x2e5b44206361 <JS Object>
       #1# 0x2f7bc94c9289: 0x2f7bc94c9289 <a DBCollection>
                     _db: 0x3d5d27d8c7f9 <a DB>#3#
              _shortName: 0xcc87ce20fb9 <String[4]: $cmd>
                  _mongo: 0x3d5d27d8c811 <a Mongo>#5#
               _fullName: 0x2f7bc94c9261 <String[9]: test.$cmd>
       #2# 0x2f7bc94c91f9: 0x2f7bc94c91f9 <an Object>
                   $eval: 0x16237b856c09 <String[15]: db.currentOp();>
       #3# 0x3d5d27d8c7f9: 0x3d5d27d8c7f9 <a DB>
                  _mongo: 0x3d5d27d8c811 <a Mongo>#5#
                   _name: 0xcc87ce19a19 <String[4]: test>
       #4# 0x2f7bc94c9191: 0x2f7bc94c9191 <an Arguments>
                  length: 1
                  callee: 0x2e5b442426a1 <JS Function>#6#
       #5# 0x3d5d27d8c811: 0x3d5d27d8c811 <a Mongo>
                 slaveOk: 0x2e5b44204161 <false>
                    host: 0x16237b884659 <String[9]: 127.0.0.1>
       #6# 0x2e5b442426a1: 0x2e5b442426a1 <JS Function>
      =====================
      

      Some googling reveals that this is a known issue in v8 that has already been fixed upstream.

            Assignee:
            mathias@mongodb.com Mathias Stearn
            Reporter:
            spencer@mongodb.com Spencer Brody (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: