[SERVER-8021] jstests/slowNightly/memory.js failing with V8::FatalProcessOutOfMemory Created: 23/Dec/12  Updated: 11/Jul/16  Resolved: 24/Dec/12

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 2.3.2

Type: Bug Priority: Major - P3
Reporter: Tad Marshall Assignee: Ben Becker
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Builder Nightly Windows 64-bit 2008+
Builder Linux 64-bit DUR OFF


Issue Links:
Related
Operating System: ALL
Participants:

 Description   

http://buildlogs.mongodb.org/Nightly%20Windows%2064-bit%202008%2B/builds/372/test/slow%20nightly/memory.js

Sun Dec 23 05:36:32.346 [conn90] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9959" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:437543 reslen:45 437ms
Sun Dec 23 05:36:32.877 [conn90] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9971" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:440254 reslen:45 440ms
Sun Dec 23 05:36:33.407 [conn90] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9983" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:439516 reslen:45 439ms
Sun Dec 23 05:36:33.938 [conn90] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9995" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:439727 reslen:45 439ms
Sun Dec 23 05:36:34.000 [conn90] build index test.system.js { _id: 1 }
Sun Dec 23 05:36:34.000 [conn90] build index done.  scanned 0 total records. 0.001 secs
Sun Dec 23 05:36:35.856 [conn90] *** unhandled exception 0x80000003 at 0x000000013FF6287A, terminating
Sun Dec 23 05:36:35.856 [conn90] *** stack trace for unhandled exception:
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\v8\src\checks.cc(99)                         API_Fatal+0x6a
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\v8\src\api.cc(133)                           v8::DefaultFatalErrorHandler+0x65
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\v8\src\api.cc(213)                           v8::internal::V8::FatalProcessOutOfMemory+0x1f2
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\v8\src\execution.cc(137)                     v8::internal::Invoke+0x211
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\v8\src\execution.cc(179)                     v8::internal::Execution::Call+0x1f3
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\v8\src\api.cc(3660)                          v8::Function::Call+0x19b
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\scripting\engine_v8.cpp(855)                       mongo::V8Scope::invoke+0x4b3
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\scripting\engine.cpp(434)                          mongo::PooledScope::invoke+0x3e
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\dbeval.cpp(86)                                  mongo::dbEval+0x3d3
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\dbeval.cpp(132)                                 mongo::CmdEval::run+0x169
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1822)                            mongo::_execCommand+0x5e
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1931)                            mongo::Command::execCommand+0x93d
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\dbcommands.cpp(2023)                            mongo::_runCommands+0x3fd
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\ops\query.cpp(46)                               mongo::runCommands+0x46
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\ops\query.cpp(944)                              mongo::runQuery+0x43f
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\instance.cpp(260)                               mongo::receivedQuery+0x35a
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\instance.cpp(405)                               mongo::assembleResponse+0x2f0
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\db\db.cpp(193)                                     mongo::MyMessageHandler::process+0xfa
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\mongo\util\net\message_server_port.cpp(206)              mongo::PortMessageServer::handleIncomingMsg+0x56a
Sun Dec 23 05:36:37.229 [conn90] mongod.exe  ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(180)  boost::`anonymous namespace'::thread_start_function+0x21
Sun Dec 23 05:36:37.229 [conn90] writing minidump diagnostic file mongo.dmp
Processing memoryTest0
Sun Dec 23 05:36:38.072 [conn90] *** immediate exit due to unhandled exception
Processing memoryTest1000
Processing memoryTest2000
Processing memoryTest3000
Processing memoryTest4000
Processing memoryTest5000
Processing memoryTest6000
Processing memoryTest7000
Processing memoryTest8000
Processing memoryTest9000
Sun Dec 23 05:36:38.072 Socket recv() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
Sun Dec 23 05:36:38.072 SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [1] server [127.0.0.1:27999] 
Sun Dec 23 05:36:38.072 DBClientCursor::init call() failed
Sun Dec 23 05:36:38.072 trying reconnect to 127.0.0.1:27999
Sun Dec 23 05:36:39.117 reconnect 127.0.0.1:27999 failed couldn't connect to server 127.0.0.1:27999
Sun Dec 23 05:36:39.117 exec error: src/mongo/shell/collection.js:51 socket error on query
dify( { update : ... , remove : bool [, query: {}, sort: {}, 'new': false] } )
                                                                    ^
failed to load: D:\slave\Windows_64bit_2008+_Nightly\mongo\jstests\slowNightly\memory.js

The "unhandled exception" that it hits is a breakpoint exception (80000003) in the V8 API_Fatal routine.



 Comments   
Comment by auto [ 24/Dec/12 ]

Author:

{u'date': u'2012-12-23T22:40:47Z', u'email': u'ben.becker@10gen.com', u'name': u'Ben Becker'}

Message: SERVER-8021: on v8 OOM, throw instead of abort
Branch: master
https://github.com/mongodb/mongo/commit/a7169fdbfb82f228afba1aad96080e40687571e1

Comment by Tad Marshall [ 23/Dec/12 ]

http://buildlogs.mongodb.org/Linux%2064-bit%20DUR%20OFF/builds/2083/test/recent%20failures/memory.js

Sun Dec 23 03:59:43.218 [conn2] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9959" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:427480 reslen:45 427ms
Sun Dec 23 03:59:43.739 [conn2] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9971" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:427171 reslen:45 427ms
Sun Dec 23 03:59:44.261 [conn2] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9983" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:427504 reslen:45 427ms
Sun Dec 23 03:59:44.783 [conn2] command test.$cmd command: { $eval: function (col) { for (var i = 0; i < 100; ++i) {db[col + "_" + i].find();} }, args: [ "memoryTest9995" ] } ntoreturn:1 keyUpdates:0 locks(micros) W:426805 reslen:45 426ms
Sun Dec 23 03:59:44.845 [FileAllocator] allocating new datafile /data/db/sconsTests/test.ns, filling with zeroes...
Sun Dec 23 03:59:44.846 [FileAllocator] done allocating datafile /data/db/sconsTests/test.ns, size: 16MB,  took 0 secs
Sun Dec 23 03:59:44.846 [FileAllocator] allocating new datafile /data/db/sconsTests/test.0, filling with zeroes...
Sun Dec 23 03:59:44.846 [FileAllocator] done allocating datafile /data/db/sconsTests/test.0, size: 64MB,  took 0 secs
Sun Dec 23 03:59:44.847 [FileAllocator] allocating new datafile /data/db/sconsTests/test.1, filling with zeroes...
Sun Dec 23 03:59:44.848 [FileAllocator] done allocating datafile /data/db/sconsTests/test.1, size: 128MB,  took 0 secs
Sun Dec 23 03:59:44.851 [conn2] build index test.system.js { _id: 1 }
Sun Dec 23 03:59:44.853 [conn2] build index done.  scanned 0 total records. 0.001 secs
 
Processing memoryTest0
#
# Fatal error in JS
# Allocation failed - process out of memory
#
 
Processing memoryTest1000
Processing memoryTest2000
Processing memoryTest3000
Processing memoryTest4000
Processing memoryTest5000
Processing memoryTest6000
Processing memoryTest7000
Processing memoryTest8000
Processing memoryTest9000
Sun Dec 23 03:59:47.375 DBClientCursor::init call() failed
Sun Dec 23 03:59:47.375 trying reconnect to 127.0.0.1:27999
Sun Dec 23 03:59:47.375 reconnect 127.0.0.1:27999 failed couldn't connect to server 127.0.0.1:27999
Sun Dec 23 03:59:47.376 exec error: src/mongo/shell/collection.js:51 socket error on query
dify( { update : ... , remove : bool [, query: {}, sort: {}, 'new': false] } )
                                                                    ^
 
failed to load: /home/yellow/buildslave/Linux_64bit_DUR_OFF/mongo/jstests/slowNightly/memory.js

Looks like the same event, but this one has V8 error test and no stack trace, probably because the breakpoint fires before the V8 text is output and nothing catches it on Linux.

Generated at Thu Feb 08 03:16:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.