[SERVER-11871] Bug in v8 memory allocator that causes long-running shell to crash Created: 26/Nov/13  Updated: 11/Jul/16  Resolved: 07/Jan/14

Status: Closed
Project: Core Server
Component/s: JavaScript, Shell, Stability
Affects Version/s: 2.5.4
Fix Version/s: 2.4.10, 2.5.5

Type: Bug Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   
Issue Status as of March 28, 2014

ISSUE SUMMARY
A bug in V8 memory allocation causes a long-running mongo shell to crash.

USER IMPACT
Long-running tasks using the mongo shell or Javascript server-side (like map/reduce or eval command) could be aborted before they finish.

SOLUTION
Apply the patch that fixed the issue upstream in the V8 code.

WORKAROUNDS
Avoid running operations in the mongo shell that take several hours or longer.

AFFECTED VERSIONS
All production releases from version 2.4.0 to 2.4.9 are affected.

PATCHES
The fix is included in the 2.4.10 production release and the 2.5.5 development release, which will evolve into the 2.6.0 production release.

Original Description

While running a long-running shell that was was doing a simple operation in a loop, the shell crashed with a V8 error.

Here's what I was running:

> for (var j = 0 ; true; j++){ if (j%500 == 0) {print("still going");}; db.eval("db.currentOp();")}

This ran fine for several hours before crashing like this:

#
# Fatal error in src/third_party/v8/src/spaces-inl.h, line 167
# CHECK(chunk->size() <= static_cast<size_t>(kPageSize)) failed
#
 
 
==== Stack trace ============================================
 
Security context: 0x2e5b44206361 <JS Object>#0#
    2: find [src/mongo/shell/collection.js:~158] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,limit=-1,skip=0,batchSize=0,options=0x2e5b44204121 <undefined>)
    3: findOne [src/mongo/shell/collection.js:~171] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,options=0x2e5b44204121 <undefined>)
    4: arguments adaptor frame: 1->3
    5: runCommand [src/mongo/shell/db.js:~47] (this=0x3d5d27d8c7f9 <a DB>#3#,obj=0x2f7bc94c91f9 <an Object>#2#)
    6: eval [src/mongo/shell/db.js:393] (this=0x3d5d27d8c7f9 <a DB>#3#,jsfunction=0x16237b856c09 <String[15]: db.currentOp();>)
    7: /* anonymous */(aka /* anonymous */) [(shell):~1] (this=0)
 
==== Details ================================================
 
[2]: find [src/mongo/shell/collection.js:~158] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,limit=-1,skip=0,batchSize=0,options=0x2e5b44204121 <undefined>) {
// optimized frame
}
[3]: findOne [src/mongo/shell/collection.js:~171] (this=0x2f7bc94c9289 <a DBCollection>#1#,query=0x2f7bc94c91f9 <an Object>#2#,fields=0x2e5b44204121 <undefined>,options=0x2e5b44204121 <undefined>) {
// optimized frame
}
[4]: arguments adaptor frame: 1->3 {
  // actual arguments
  [00] : 0x2f7bc94c91f9 <an Object>#2#
}
 
[5]: runCommand [src/mongo/shell/db.js:~47] (this=0x3d5d27d8c7f9 <a DB>#3#,obj=0x2f7bc94c91f9 <an Object>#2#) {
// optimized frame
}
[6]: eval [src/mongo/shell/db.js:393] (this=0x3d5d27d8c7f9 <a DB>#3#,jsfunction=0x16237b856c09 <String[15]: db.currentOp();>) {
  // stack-allocated locals
  var cmd = 0x2f7bc94c91f9 <an Object>#2#
  var arguments = 0x2f7bc94c9191 <an Arguments>#4#
  var res = 0x2e5b44204121 <undefined>
  // heap-allocated locals
  var jsfunction = 0x16237b856c09 <String[15]: db.currentOp();>
  // expression stack (top to bottom)
  [04] : 0x2f7bc94c91f9 <an Object>#2#
  [03] : 0x3d5d27d8c7f9 <a DB>#3#
--------- s o u r c e   c o d e ---------
function (jsfunction) {?    var cmd = { $eval : jsfunction };?    if ( arguments.length > 1 ) {?        cmd.args = argumentsToArray( arguments ).slice(1);?    }??    var res = this._dbCommand( cmd );??    if (!res.ok)?        throw tojson( res );??    return res.retval;?}
-----------------------------------------
}
 
[7]: /* anonymous */(aka /* anonymous */) [(shell):~1] (this=0) {
// optimized frame
}
==== Key         ============================================
 
 #0# 0x2e5b44206361: 0x2e5b44206361 <JS Object>
 #1# 0x2f7bc94c9289: 0x2f7bc94c9289 <a DBCollection>
               _db: 0x3d5d27d8c7f9 <a DB>#3#
        _shortName: 0xcc87ce20fb9 <String[4]: $cmd>
            _mongo: 0x3d5d27d8c811 <a Mongo>#5#
         _fullName: 0x2f7bc94c9261 <String[9]: test.$cmd>
 #2# 0x2f7bc94c91f9: 0x2f7bc94c91f9 <an Object>
             $eval: 0x16237b856c09 <String[15]: db.currentOp();>
 #3# 0x3d5d27d8c7f9: 0x3d5d27d8c7f9 <a DB>
            _mongo: 0x3d5d27d8c811 <a Mongo>#5#
             _name: 0xcc87ce19a19 <String[4]: test>
 #4# 0x2f7bc94c9191: 0x2f7bc94c9191 <an Arguments>
            length: 1
            callee: 0x2e5b442426a1 <JS Function>#6#
 #5# 0x3d5d27d8c811: 0x3d5d27d8c811 <a Mongo>
           slaveOk: 0x2e5b44204161 <false>
              host: 0x16237b884659 <String[9]: 127.0.0.1>
 #6# 0x2e5b442426a1: 0x2e5b442426a1 <JS Function>
=====================

Some googling reveals that this is a known issue in v8 that has already been fixed upstream.



 Comments   
Comment by Githook User [ 09/Mar/14 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-11871 Fix assert in Page::Initialize.

This is a clean backport from this commit from V8:
https://code.google.com/p/v8/source/detail?r=13896
Branch: v2.4
https://github.com/mongodb/mongo/commit/2c62563720d16b49abe24cbb799b70e5b00ed169

Comment by Githook User [ 07/Jan/14 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-11871 Fix assert in Page::Initialize.

This is a clean backport from this commit from V8:
https://code.google.com/p/v8/source/detail?r=13896
Branch: master
https://github.com/mongodb/mongo/commit/949a331bfd2fa878324879089da2e58e333d3928

Comment by Spencer Brody (Inactive) [ 02/Dec/13 ]

Last week I was able to reproduce the crash 3 different times after about 3-5 hours each time.

I manually applied the patch from https://chromiumcodereview.appspot.com/11275229 and then left a shell running doing the same thing and it ran for 5 days without crashing.

Generated at Thu Feb 08 03:26:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.