[SERVER-23888] Unoptimized builds on OSX overflow the stack Created: 22/Apr/16  Updated: 08/Jan/24

Status: Backlog
Project: Core Server
Component/s: JavaScript
Affects Version/s: 3.3.5
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Kamran K. Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 1
Labels: move-sa, platforms-re-triaged
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-24469 jstests/core/and.js fails with SEGFAU... Closed
is duplicated by SERVER-25112 mapReduce segmentation fault on v3.3.... Closed
Assigned Teams:
Query Execution
Backwards Compatibility: Fully Compatible
Participants:

 Description   

1 - My shell was built with SpiderMonkey 45.0.2:

$ ./mongo --version
MongoDB shell version: 3.3.4-319-g64258ac

2 - I built with optimizations off on OS X 10.11.4:

scons -j5 core --opt=off
 
$ clang --version
Apple LLVM version 7.3.0 (clang-703.0.29)

3 - I removed the build/ directory and rebuilt and can still repro the crash:

frame #0: 0x0000000107ceb46d mongo`js::detail::HashTable<js::AtomStateEntry const, js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::SetOps, js::SystemAllocPolicy>::all(this=0x0000000000000000) const + 29 at HashTable.h:1578
   1575	    Range all() const
   1576	    {
   1577	        MOZ_ASSERT(table);
-> 1578	        return Range(*this, table, table + capacity());
   1579	    }
   1580	
   1581	    bool empty() const
(lldb) fr v
(const js::detail::HashTable<const js::AtomStateEntry, js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::SetOps, js::SystemAllocPolicy> *) this = 0x0000000000000000
(lldb) bt
* thread #3: tid = 0x0002, 0x0000000107ceb46d mongo`js::detail::HashTable<js::AtomStateEntry const, js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::SetOps, js::SystemAllocPolicy>::all(this=0x0000000000000000) const + 29 at HashTable.h:1578, stop reason = signal SIGSTOP
  * frame #0: 0x0000000107ceb46d mongo`js::detail::HashTable<js::AtomStateEntry const, js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::SetOps, js::SystemAllocPolicy>::all(this=0x0000000000000000) const + 29 at HashTable.h:1578
    frame #1: 0x0000000107ceb431 mongo`js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::all(this=0x0000000000000000) const + 17 at HashTable.h:402
    frame #2: 0x0000000107ced571 mongo`js::detail::HashTable<js::AtomStateEntry const, js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::SetOps, js::SystemAllocPolicy>::Enum::Enum<js::GCHashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy, js::DefaultGCPolicy<js::AtomStateEntry> > >(this=0x0000700000102bc0, map=0x0000000000000000) + 33 at HashTable.h:977
    frame #3: 0x0000000107ce9e4d mongo`js::detail::HashTable<js::AtomStateEntry const, js::HashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy>::SetOps, js::SystemAllocPolicy>::Enum::Enum<js::GCHashSet<js::AtomStateEntry, js::AtomHasher, js::SystemAllocPolicy, js::DefaultGCPolicy<js::AtomStateEntry> > >(this=0x0000700000102bc0, map=0x0000000000000000) + 29 at HashTable.h:977
    frame #4: 0x0000000107ce7e8e mongo`js::MarkAtoms(trc=0x00007fcc6b05a238) + 46 at jsatom.cpp:197
    frame #5: 0x00000001085a51e2 mongo`js::gc::GCRuntime::markRuntime(this=0x00007fcc6b0583f0, trc=0x00007fcc6b05a238, traceOrMark=MarkRuntime) + 994 at RootMarking.cpp:298
    frame #6: 0x00000001081d81dd mongo`js::gc::GCRuntime::beginMarkPhase(this=0x00007fcc6b0583f0, reason=DESTROY_RUNTIME) + 2765 at jsgc.cpp:4044
    frame #7: 0x00000001081df0f3 mongo`js::gc::GCRuntime::incrementalCollectSlice(this=0x00007fcc6b0583f0, budget=0x0000700000103390, reason=DESTROY_RUNTIME) + 387 at jsgc.cpp:6024
    frame #8: 0x00000001081dfb90 mongo`js::gc::GCRuntime::gcCycle(this=0x00007fcc6b0583f0, nonincrementalByAPI=true, budget=0x0000700000103390, reason=DESTROY_RUNTIME) + 704 at jsgc.cpp:6278
    frame #9: 0x00000001081e0261 mongo`js::gc::GCRuntime::collect(this=0x00007fcc6b0583f0, nonincrementalByAPI=true, budget=SliceBudget @ 0x0000700000103390, reason=DESTROY_RUNTIME) + 497 at jsgc.cpp:6384
    frame #10: 0x00000001081e0746 mongo`js::gc::GCRuntime::gc(this=0x00007fcc6b0583f0, gckind=GC_NORMAL, reason=DESTROY_RUNTIME) + 102 at jsgc.cpp:6442
    frame #11: 0x0000000108434dad mongo`JSRuntime::~JSRuntime(this=0x00007fcc6b058000) + 893 at Runtime.cpp:412
    frame #12: 0x0000000108435455 mongo`JSRuntime::~JSRuntime(this=0x00007fcc6b058000) + 21 at Runtime.cpp:361
    frame #13: 0x000000010815ec18 mongo`JS_DestroyRuntime(JSRuntime*) [inlined] void js_delete<JSRuntime>(p=0x00007fcc6b058000) + 20 at Utility.h:370
    frame #14: 0x000000010815ec04 mongo`JS_DestroyRuntime(rt=0x00007fcc6b058000) + 20 at jsapi.cpp:480
    frame #15: 0x00000001076ccabe mongo`mongo::mozjs::MozJSImplScope::MozRuntime::~MozRuntime(this=0x00007fcc6b055468) + 46 at implscope.cpp:280
    frame #16: 0x00000001076ccb85 mongo`mongo::mozjs::MozJSImplScope::MozRuntime::~MozRuntime(this=0x00007fcc6b055468) + 21 at implscope.cpp:278
    frame #17: 0x00000001076cdbde mongo`mongo::mozjs::MozJSImplScope::MozJSImplScope(this=0x00007fcc6b055400, engine=0x00007fcc6ac0c7b0) + 4174 at implscope.cpp:365
    frame #18: 0x00000001076cea1d mongo`mongo::mozjs::MozJSImplScope::MozJSImplScope(this=0x00007fcc6b055400, engine=0x00007fcc6ac0c7b0) + 29 at implscope.cpp:332
    frame #19: 0x0000000107714848 mongo`mongo::mozjs::MozJSProxyScope::implThread(arg=0x00007fcc6ae005c0) + 264 at proxyscope.cpp:330
    frame #20: 0x00000001076a75d6 mongo`nspr::Thread::ThreadRoutine(arg=0x00007fcc6ae006f0) + 54 at PosixNSPR.cpp:56
    frame #21: 0x00000001076aa16d mongo`void* std::__1::__thread_proxy<std::__1::tuple<void* (*)(void*), nspr::Thread*> >(void*) [inlined] decltype(__f=0x00007fcc6ae00710, __args=0x00007fcc6ae00718)(void*)>(fp)(std::__1::forward<nspr::Thread*>(fp0))) std::__1::__invoke<void* (*)(void*), nspr::Thread*>(void* (*&&)(void*), nspr::Thread*&&) + 24 at __functional_base:416
    frame #22: 0x00000001076aa155 mongo`void* std::__1::__thread_proxy<std::__1::tuple<void* (*)(void*), nspr::Thread*> >(void*) [inlined] void std::__1::__thread_execute<void* (*)(void*), nspr::Thread*, 1ul>(__t=0x00007fcc6ae00710)(void*), nspr::Thread*>&, std::__1::__tuple_indices<1ul>) + 40 at thread:337
    frame #23: 0x00000001076aa12d mongo`void* std::__1::__thread_proxy<std::__1::tuple<void* (*)(void*), nspr::Thread*> >(__vp=0x00007fcc6ae00710) + 365 at thread:347
    frame #24: 0x00007fff8ee0599d libsystem_pthread.dylib`_pthread_body + 131
    frame #25: 0x00007fff8ee0591a libsystem_pthread.dylib`_pthread_start + 168
    frame #26: 0x00007fff8ee03351 libsystem_pthread.dylib`thread_start + 13
(lldb) 



 Comments   
Comment by Jonathan Reams [ 13/Dec/16 ]

Since mongo/mongod now run and pass almost all of jscore, I'm going to reassign this to the storage team to figure out why/fix keystring overflowing the stack on deeply nested objects.

Comment by Jonathan Reams [ 13/Dec/16 ]

So now mongod/mongo build just fine and pass all of jsCore except for the nestedarr1.js/nestedobj1.js tests. The failing trace is:

ireCount: { r: 1, w: 1 } }, Database: { acquireCount: { W: 1 } }, Collection: { acquireCount: { w: 1 } } } protocol:op_command 111ms
Process 16775 stopped
* thread #2: tid = 0x48e0f, 0x0000000100008e3f mongod`mongo::_BufBuilder<mongo::SharedBufferAllocator>::grow(this=0x0000700001978da8, by=1) + 31 at builder.h:281, name = 'conn1', stop reason = EXC_BAD_ACCESS (code=2, address=0x7000018feffc)
    frame #0: 0x0000000100008e3f mongod`mongo::_BufBuilder<mongo::SharedBufferAllocator>::grow(this=0x0000700001978da8, by=1) + 31 at builder.h:281
   278 	    /* returns the pre-grow write position */
   279 	    inline char* grow(int by) {
   280 	        int oldlen = l;
-> 281 	        int newLen = l + by;
   282 	        int minSize = newLen + reservedBytes;
   283 	        if (minSize > size) {
   284 	            grow_reallocate(minSize);
(lldb) bt
warning: could not load any Objective-C class information. This will significantly reduce the quality of type information available.
* thread #2: tid = 0x48e0f, 0x0000000100008e3f mongod`mongo::_BufBuilder<mongo::SharedBufferAllocator>::grow(this=0x0000700001978da8, by=1) + 31 at builder.h:281, name = 'conn1', stop reason = EXC_BAD_ACCESS (code=2, address=0x7000018feffc)
  * frame #0: 0x0000000100008e3f mongod`mongo::_BufBuilder<mongo::SharedBufferAllocator>::grow(this=0x0000700001978da8, by=1) + 31 at builder.h:281
    frame #1: 0x000000010001bfde mongod`void mongo::_BufBuilder<mongo::SharedBufferAllocator>::appendNumImpl<char>(this=0x0000700001978da8, t='\x04') + 46 at builder.h:319
    frame #2: 0x000000010001becf mongod`mongo::_BufBuilder<mongo::SharedBufferAllocator>::appendNum(this=0x0000700001978da8, j='\x04') + 31 at builder.h:205
    frame #3: 0x00000001000862b9 mongod`mongo::BSONObjBuilder::subarrayStart(this=0x00007000019017a0, fieldName=(_data = "a", _size = 1)) + 57 at bsonobjbuilder.h:217
    frame #4: 0x000000010008624a mongod`mongo::BSONObjBuilderValueStream::subarrayStart(this=0x00007000019017c8) + 138 at bsonmisc.cpp:95
...
    frame #150: 0x00000001011b7036 mongod`mongo::(anonymous namespace)::toBson(reader=0x0000700001978d88, typeBits=0x0000700001978d78, inverted=false, version=V1, builder=0x0000700001978ba0) + 422 at key_string.cpp:1103
    frame #151: 0x00000001011b3c47 mongod`mongo::(anonymous namespace)::toBsonValue(ctype='F', reader=0x0000700001978d88, typeBits=0x0000700001978d78, inverted=false, version=V1, stream=0x0000700001978dc8) + 7351 at key_string.cpp:1276
    frame #152: 0x00000001011b1e28 mongod`mongo::KeyString::toBson(buffer="FPa", len=348, ord=(bits = 0), typeBits=0x000000010787df00) + 568 at key_string.cpp:1707
    frame #153: 0x00000001012f9cd4 mongod`mongo::(anonymous namespace)::WiredTigerIndexCursorBase::curr(this=0x000000010787dc00, parts=kKeyAndLoc) const + 244 at wiredtiger_index.cpp:785
    frame #154: 0x00000001012f8245 mongod`mongo::(anonymous namespace)::WiredTigerIndexCursorBase::next(this=0x000000010787dc00, parts=kKeyAndLoc) + 117 at wiredtiger_index.cpp:674
    frame #155: 0x0000000100600d9a mongod`mongo::IndexScan::doWork(this=0x0000000105f2a030, out=0x0000700001979f58) + 378 at index_scan.cpp:140
    frame #156: 0x000000010062b8fa mongod`mongo::PlanStage::work(this=0x0000000105f2a030, out=0x0000700001979f58) + 170 at plan_stage.cpp:46
    frame #157: 0x00000001005da833 mongod`mongo::FetchStage::doWork(this=0x0000000105f2a2d0, out=0x000070000197a870) + 179 at fetch.cpp:86
    frame #158: 0x000000010062b8fa mongod`mongo::PlanStage::work(this=0x0000000105f2a2d0, out=0x000070000197a870) + 170 at plan_stage.cpp:46
    frame #159: 0x0000000100c5dde2 mongod`mongo::PlanExecutor::getNextImpl(this=0x0000000105f2a3a0, objOut=0x000070000197aa50, dlOut=0x0000000000000000) + 4258 at plan_executor.cpp:425
    frame #160: 0x0000000100c5cc72 mongod`mongo::PlanExecutor::getNext(this=0x0000000105f2a3a0, objOut=0x000070000197b778, dlOut=0x0000000000000000) + 114 at plan_executor.cpp:345
    frame #161: 0x000000010045f229 mongod`mongo::FindCmd::run(this=0x00000001036b96c8, txn=0x0000000105f27590, dbname="test", cmdObj=0x000070000197c108, options=0, errmsg="", result=0x000070000197c208) + 7097 at find_cmd.cpp:354
    frame #162: 0x000000010042aa08 mongod`mongo::Command::run(this=0x00000001036b96c8, txn=0x0000000105f27590, request=0x000070000197d268, replyBuilder=0x000070000197d2d8) + 2584 at dbcommands.cpp:1547
    frame #163: 0x000000010042931b mongod`mongo::Command::execCommand(txn=0x0000000105f27590, command=0x00000001036b96c8, request=0x000070000197d268, replyBuilder=0x000070000197d2d8) + 6875 at dbcommands.cpp:1461
    frame #164: 0x0000000101014c00 mongod`mongo::runCommands(txn=0x0000000105f27590, request=0x000070000197d268, replyBuilder=0x000070000197d2d8) + 2320 at run_commands.cpp:73
    frame #165: 0x00000001007f78b8 mongod`mongo::(anonymous namespace)::receivedRpc(txn=0x0000000105f27590, client=0x0000000105f1ed40, dbResponse=0x000070000197e510, message=0x000070000197e560) + 616 at instance.cpp:271
    frame #166: 0x00000001007f4840 mongod`mongo::assembleResponse(txn=0x0000000105f27590, m=0x000070000197e560, dbresponse=0x000070000197e510, remote=0x0000000106c004d8) + 2400 at instance.cpp:618
    frame #167: 0x0000000100023719 mongod`mongo::ServiceEntryPointMongod::_sessionLoop(this=0x0000000105f007d0, session=std::__1::shared_ptr<mongo::transport::Session>::element_type @ 0x0000000106c00490 strong=2 weak=3) + 761 at service_entry_point_mongod.cpp:135
    frame #168: 0x0000000100025f79 mongod`mongo::ServiceEntryPointMongod::startSession(this=0x0000000105f1ed18, session=std::__1::shared_ptr<mongo::transport::Session>::element_type @ 0x0000000106c00490 strong=2 weak=3)::$_0::operator()(std::__1::shared_ptr<mongo::transport::Session> const&) const + 105 at service_entry_point_mongod.cpp:103
    frame #169: 0x0000000100025efd mongod`void std::__1::__invoke_void_return_wrapper<void>::__call<mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0&, std::__1::shared_ptr<mongo::transport::Session> const&>(mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0&&&, std::__1::shared_ptr<mongo::transport::Session> const&&&) [inlined] decltype(__f=0x0000000105f1ed18, __args=std::__1::shared_ptr<mongo::transport::Session>::element_type @ 0x0000000106c00490 strong=2 weak=3)::$_0&>(fp)(std::__1::forward<std::__1::shared_ptr<mongo::transport::Session> const&>(fp0))) std::__1::__invoke<mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0&, std::__1::shared_ptr<mongo::transport::Session> const&>(mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0&&&, std::__1::shared_ptr<mongo::transport::Session> const&&&) + 77 at __functional_base:416
    frame #170: 0x0000000100025ee0 mongod`void std::__1::__invoke_void_return_wrapper<void>::__call<mongo::ServiceEntryPointMongod::startSession(__args=0x0000000105f1ed18, __args=std::__1::shared_ptr<mongo::transport::Session>::element_type @ 0x0000000106c00490 strong=2 weak=3)::$_0&, std::__1::shared_ptr<mongo::transport::Session> const&>(mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0&&&, std::__1::shared_ptr<mongo::transport::Session> const&&&) + 48 at __functional_base:468
    frame #171: 0x0000000100025da9 mongod`std::__1::__function::__func<mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0, std::__1::allocator<mongo::ServiceEntryPointMongod::startSession(std::__1::shared_ptr<mongo::transport::Session>)::$_0>, void (std::__1::shared_ptr<mongo::transport::Session> const&)>::operator(this=0x0000000105f1ed10, __arg=std::__1::shared_ptr<mongo::transport::Session>::element_type @ 0x0000000106c00490 strong=2 weak=3)(std::__1::shared_ptr<mongo::transport::Session> const&) + 57 at functional:1437
    frame #172: 0x000000010010cde1 mongod`std::__1::function<void (mongo::DBClientCursorBatchIterator&)>::operator(this=0x0000000105f1ed10, __arg=0x0000000105f1ed00)(mongo::DBClientCursorBatchIterator&) const + 145 at functional:1817
    frame #173: 0x0000000101725f68 mongod`mongo::(anonymous namespace)::runFunc(ptr=0x0000000105f1ed00) + 824 at service_entry_point_utils.cpp:78
    frame #174: 0x0000000101727475 mongod`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(void*) [inlined] decltype(__f=0x0000000105f1cf50, __args=0x0000000105f1cf58)(void*)>(fp)(std::__1::forward<mongo::(anonymous namespace)::Context*&>(fp0))) std::__1::__invoke<void* (*&)(void*), mongo::(anonymous namespace)::Context*&>(void* (*&&&)(void*), mongo::(anonymous namespace)::Context*&&&) + 24 at __functional_base:416
    frame #175: 0x000000010172745d mongod`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(void*) [inlined] std::__1::__bind_return<void* (*)(void*), std::__1::tuple<mongo::(anonymous namespace)::Context*>, std::__1::tuple<>, __is_valid_bind_return<void* (*)(void*), std::__1::tuple<mongo::(anonymous namespace)::Context*>, std::__1::tuple<> >::value>::type std::__1::__apply_functor<void* (__f=0x0000000105f1cf50, __bound_args=0x0000000105f1cf58, __args=0x000070000197eee0)(void*), std::__1::tuple<mongo::(anonymous namespace)::Context*>, 0ul, std::__1::tuple<> >(void* (*&)(void*), std::__1::tuple<mongo::(anonymous namespace)::Context*>&, std::__1::__tuple_indices<0ul>, std::__1::tuple<>&&) + 46 at functional:2097
    frame #176: 0x000000010172742f mongod`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(void*) [inlined] std::__1::__bind_return<void* (*)(void*), std::__1::tuple<mongo::(anonymous namespace)::Context*>, std::__1::tuple<>, __is_valid_bind_return<void* (*)(void*), std::__1::tuple<mongo::(anonymous namespace)::Context*>, std::__1::tuple<> >::value>::type std::__1::__bind<void* (this=0x0000000105f1cf50)(void*), mongo::(anonymous namespace)::Context*>::operator()<>() + 47 at functional:2160
    frame #177: 0x0000000101727400 mongod`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(void*) [inlined] decltype(__f=0x0000000105f1cf50)(void*), mongo::(anonymous namespace)::Context*> >(fp)(std::__1::forward<>(fp0))) std::__1::__invoke<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> >(std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*>&&) + 14 at __functional_base:416
    frame #178: 0x00000001017273f2 mongod`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(void*) [inlined] void std::__1::__thread_execute<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> >(__t=0x0000000105f1cf50)(void*), mongo::(anonymous namespace)::Context*> >&, std::__1::__tuple_indices<>) + 25 at thread:347
    frame #179: 0x00000001017273d9 mongod`void* std::__1::__thread_proxy<std::__1::tuple<std::__1::__bind<void* (&)(void*), mongo::(anonymous namespace)::Context*> > >(__vp=0x0000000105f1cf50) + 361 at thread:357
    frame #180: 0x00007fffbc933aab libsystem_pthread.dylib`_pthread_body + 180
    frame #181: 0x00007fffbc9339f7 libsystem_pthread.dylib`_pthread_start + 286
    frame #182: 0x00007fffbc933221 libsystem_pthread.dylib`thread_start + 13

Comment by Andrew Morrow (Inactive) [ 24/Aug/16 ]

I tried out XCode 8 Beta 6 and the issue still reproduces.

Comment by Geert Bosch [ 24/Aug/16 ]

I consistently hit this. Using latest XCode version:
Apple LLVM version 7.3.0 (clang-703.0.31)

This makes development hard on OS X.

Generated at Thu Feb 08 04:04:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.