[SERVER-11181] parallel basic.js failing on Windows 64 DEBUG (or6.js?) Created: 14/Oct/13  Updated: 11/Jul/16  Resolved: 16/Oct/13

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 2.5.3

Type: Bug Priority: Major - P3
Reporter: Matt Kangas Assignee: J Rassi
Resolution: Done Votes: 0
Labels: buildbot
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-11179 or6.js failed on Windows 64-bit DEBUG Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

486efd8f13fc Windows 64-bit DEBUG parallel (logfile)
486efd8f13fc Windows 64-bit 2008R2+ DEBUG parallel (logfile)

On both of these Windows DEBUG builders, the first detected failure is 34b8897 SERVER-9063 Add new query operator $text"

34b88976c91f Windows 64-bit DEBUG parallel
34b88976c91f Windows 64-bit 2008R2+ DEBUG parallel



 Comments   
Comment by Matt Kangas [ 16/Oct/13 ]

Green as of 4850a6ae1060

4850a6ae1060 Windows 64-bit DEBUG parallel
4850a6ae1060 Windows 64-bit 2008R2+ DEBUG parallel

Comment by hari.khalsa@10gen.com [ 16/Oct/13 ]

I probably fixed this with http://codereview.10gen.com/13240003/

rassi@10gen.com kangas

Comment by J Rassi [ 15/Oct/13 ]

regex5.js and or6.js pass on my Windows dev box as of 1ea7e56c (today). Running the whole parallel/basic suite...

Comment by J Rassi [ 15/Oct/13 ]

Of the last 11 failures, 8 of them involve or6.js (SERVER-11179, already resolved). The other 3 have regex5.js in common, which hasn't been run in solo on this builder since or6.js has been failing (since or6.js is run before regex5.js in the js suite). I'm thus under the suspicion that or6.js and regex5.js are both contributing to the parallel build failure – compiling debug on a Windows dev machine to test them in isolation.

Comment by Matt Kangas [ 14/Oct/13 ]

Ok, here are two recent failures that look a lot like the failure in SERVER-11179.

"test.jstests_or6 Fatal Assertion 17006"

9b1589d125be Windows 64-bit DEBUG parallel: http://buildlogs.mongodb.org/mci_0.9_windows_64_debug/builds/1322/test/parallel_0/basic.js

         S2 Test : jstests/or6.js ...
2013-10-13T06:35:52.748+0000 [conn29] CMD: drop test.jstests_or6
2013-10-13T06:35:52.751+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.jstests_or6", "name" : "_id_" }
2013-10-13T06:35:52.752+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.752+0000 [conn29] info: creating collection test.jstests_or6 on add index
2013-10-13T06:35:52.752+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "a" : 1 }, "ns" : "test.jstests_or6", "name" : "a_1" }
2013-10-13T06:35:52.753+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.811+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "b" : 1 }, "ns" : "test.jstests_or6", "name" : "b_1" }
2013-10-13T06:35:52.812+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.817+0000 [conn32] remove test.removetest query: { x: 3.0 } ndeleted:500 keyUpdates:0 numYields:9 locks(micros) w:237761 142ms
2013-10-13T06:35:52.823+0000 [conn29] CMD: drop test.jstests_or6
2013-10-13T06:35:52.833+0000 [conn32] CMD: validate test.removetest
2013-10-13T06:35:52.834+0000 [conn32] validating index 0: test.removetest.$_id_
2013-10-13T06:35:52.835+0000 [conn32] validating index 1: test.removetest.$x_1
2013-10-13T06:35:52.835+0000 [conn32] validating index 2: test.removetest.$x_-1
2013-10-13T06:35:52.836+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.jstests_or6", "name" : "_id_" }
2013-10-13T06:35:52.837+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.837+0000 [conn29] info: creating collection test.jstests_or6 on add index
2013-10-13T06:35:52.837+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "a" : 1, "b" : 1 }, "ns" : "test.jstests_or6", "name" : "a_1_b_1" }
2013-10-13T06:35:52.838+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.838+0000 [conn32] CMD: validate test.removetest
2013-10-13T06:35:52.839+0000 [conn32] validating index 0: test.removetest.$_id_
2013-10-13T06:35:52.839+0000 [conn32] validating index 1: test.removetest.$x_1
2013-10-13T06:35:52.839+0000 [conn32] validating index 2: test.removetest.$x_-1
         S1 Test : jstests/remove.js 1511ms
         S1 Test : jstests/geo_haystack3.js ...
2013-10-13T06:35:52.843+0000 [conn32] CMD: drop test.geo_haystack3
2013-10-13T06:35:52.845+0000 [conn32] build index on: test.geo_haystack3 properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.geo_haystack3", "name" : "_id_" }
2013-10-13T06:35:52.846+0000 [conn32] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.850+0000 [conn32] build index on: test.geo_haystack3 properties: { "v" : 1, "key" : { "pos" : "geoHaystack", "type" : 1 }, "ns" : "test.geo_haystack3", "name" : "pos_geoHaystack_type_1", "bucketSize" : 1 }
2013-10-13T06:35:52.850+0000 [conn32] build index done.  scanned 4 total records. 0 secs
2013-10-13T06:35:52.878+0000 [conn29] *** C runtime error: d:\Microsoft Visual Studio 10.0\VC\INCLUDE\algorithm(3665) : Assertion failed: invalid operator<, terminating
2013-10-13T06:35:52.878+0000 [conn29] test.jstests_or6 Fatal Assertion 17006
         S1 Test : jstests/geo_haystack3.js 50ms
         S1 Test : jstests/capped5.js ...
2013-10-13T06:35:53.477+0000 [conn29] mongod.exe  ...\src\mongo\util\stacktrace.cpp(169)                        mongo::printStackTrace+0x43
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\util\log.cpp(131)                               mongo::logContext+0xc1
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\util\assert_util.cpp(131)                       mongo::fassertFailed+0xa9
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\util\stacktrace.cpp(285)                        mongo::crtDebugCallback+0x17f
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrptt.c(605)    _VCrtDbgReportW+0x812
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(242)     _CrtDbgReportWV+0x43
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(258)     _CrtDbgReportW+0x4d
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdthrow.cpp(13)  std::_Debug_message+0x34
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\algorithm(3667)    std::_Insertion_sort1<mongo::Interval * __ptr64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64),mongo::Interval>+0x1ad
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\algorithm(3798)    std::_Sort<mongo::Interval * __ptr64,__int64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0xdc
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\algorithm(3806)    std::sort<std::_Vector_iterator<std::_Vector_val<mongo::Interval,std::allocator<mongo::Interval> > >,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0x17c
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\index_bounds_builder.cpp(481)          mongo::IndexBoundsBuilder::unionize+0x60
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\index_bounds_builder.cpp(372)          mongo::IndexBoundsBuilder::translate+0x130f
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\index_bounds_builder.cpp(194)          mongo::IndexBoundsBuilder::translateAndUnion+0xe
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(375)                 mongo::QueryPlanner::mergeWithLeafNode+0x25b
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(595)                 mongo::QueryPlanner::processIndexScans+0x39e
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(729)                 mongo::QueryPlanner::buildIndexedOr+0xa8
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(798)                 mongo::QueryPlanner::buildIndexedDataAccess+0x5e5
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(1197)                mongo::QueryPlanner::plan+0x1450
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\new_find.cpp(263)                      mongo::getRunner+0x7aa
2013-10-13T06:35:53.478+0000 [conn29] test.jstests_or6 
2013-10-13T06:35:53.506+0000 [conn29] *** unhandled exception 0x80000003 at 0x000007FEFD713CA2, terminating
2013-10-13T06:35:53.506+0000 [conn29] *** stack trace for unhandled exception:
2013-10-13T06:35:54.104+0000 [conn29] KERNELBASE.dll                                                                DebugBreak+0x2
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      ...\src\mongo\util\assert_util.cpp(132)                       mongo::fassertFailed+0xb7
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      ...\src\mongo\util\stacktrace.cpp(285)                        mongo::crtDebugCallback+0x17f
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrptt.c(605)    _VCrtDbgReportW+0x812
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(242)     _CrtDbgReportWV+0x43
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(258)     _CrtDbgReportW+0x4d
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdthrow.cpp(13)  std::_Debug_message+0x34
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\algorithm(3667)    std::_Insertion_sort1<mongo::Interval * __ptr64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64),mongo::Interval>+0x1ad
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\algorithm(3798)    std::_Sort<mongo::Interval * __ptr64,__int64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0xdc
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\algorithm(3806)    std::sort<std::_Vector_iterator<std::_Vector_val<mongo::Interval,std::allocator<mongo::Interval> > >,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0x17c
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\index_bounds_builder.cpp(481)          mongo::IndexBoundsBuilder::unionize+0x60
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\index_bounds_builder.cpp(372)          mongo::IndexBoundsBuilder::translate+0x130f
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\index_bounds_builder.cpp(194)          mongo::IndexBoundsBuilder::translateAndUnion+0xe
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(375)                 mongo::QueryPlanner::mergeWithLeafNode+0x25b
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(595)                 mongo::QueryPlanner::processIndexScans+0x39e
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(729)                 mongo::QueryPlanner::buildIndexedOr+0xa8
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(798)                 mongo::QueryPlanner::buildIndexedDataAccess+0x5e5
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(1197)                mongo::QueryPlanner::plan+0x1450
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\new_find.cpp(263)                      mongo::getRunner+0x7aa
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\new_find.cpp(568)                      mongo::newRunQuery+0x2fd
2013-10-13T06:35:54.105+0000 [conn29] 
2013-10-13T06:35:54.106+0000 [conn29] writing minidump diagnostic file mongo.dmp

rev db089e849ad5: http://buildlogs.mongodb.org/mci_0.9_windows_64_debug/builds/1320/test/parallel_0/basic.js

         S2 Test : jstests/or6.js ...
2013-10-13T06:35:52.748+0000 [conn29] CMD: drop test.jstests_or6
2013-10-13T06:35:52.751+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.jstests_or6", "name" : "_id_" }
2013-10-13T06:35:52.752+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.752+0000 [conn29] info: creating collection test.jstests_or6 on add index
2013-10-13T06:35:52.752+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "a" : 1 }, "ns" : "test.jstests_or6", "name" : "a_1" }
2013-10-13T06:35:52.753+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.811+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "b" : 1 }, "ns" : "test.jstests_or6", "name" : "b_1" }
2013-10-13T06:35:52.812+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.817+0000 [conn32] remove test.removetest query: { x: 3.0 } ndeleted:500 keyUpdates:0 numYields:9 locks(micros) w:237761 142ms
2013-10-13T06:35:52.823+0000 [conn29] CMD: drop test.jstests_or6
2013-10-13T06:35:52.833+0000 [conn32] CMD: validate test.removetest
2013-10-13T06:35:52.834+0000 [conn32] validating index 0: test.removetest.$_id_
2013-10-13T06:35:52.835+0000 [conn32] validating index 1: test.removetest.$x_1
2013-10-13T06:35:52.835+0000 [conn32] validating index 2: test.removetest.$x_-1
2013-10-13T06:35:52.836+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.jstests_or6", "name" : "_id_" }
2013-10-13T06:35:52.837+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.837+0000 [conn29] info: creating collection test.jstests_or6 on add index
2013-10-13T06:35:52.837+0000 [conn29] build index on: test.jstests_or6 properties: { "v" : 1, "key" : { "a" : 1, "b" : 1 }, "ns" : "test.jstests_or6", "name" : "a_1_b_1" }
2013-10-13T06:35:52.838+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.838+0000 [conn32] CMD: validate test.removetest
2013-10-13T06:35:52.839+0000 [conn32] validating index 0: test.removetest.$_id_
2013-10-13T06:35:52.839+0000 [conn32] validating index 1: test.removetest.$x_1
2013-10-13T06:35:52.839+0000 [conn32] validating index 2: test.removetest.$x_-1
         S1 Test : jstests/remove.js 1511ms
         S1 Test : jstests/geo_haystack3.js ...
2013-10-13T06:35:52.843+0000 [conn32] CMD: drop test.geo_haystack3
2013-10-13T06:35:52.845+0000 [conn32] build index on: test.geo_haystack3 properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.geo_haystack3", "name" : "_id_" }
2013-10-13T06:35:52.846+0000 [conn32] build index done.  scanned 0 total records. 0 secs
2013-10-13T06:35:52.850+0000 [conn32] build index on: test.geo_haystack3 properties: { "v" : 1, "key" : { "pos" : "geoHaystack", "type" : 1 }, "ns" : "test.geo_haystack3", "name" : "pos_geoHaystack_type_1", "bucketSize" : 1 }
2013-10-13T06:35:52.850+0000 [conn32] build index done.  scanned 4 total records. 0 secs
2013-10-13T06:35:52.878+0000 [conn29] *** C runtime error: d:\Microsoft Visual Studio 10.0\VC\INCLUDE\algorithm(3665) : Assertion failed: invalid operator<, terminating
2013-10-13T06:35:52.878+0000 [conn29] test.jstests_or6 Fatal Assertion 17006
         S1 Test : jstests/geo_haystack3.js 50ms
         S1 Test : jstests/capped5.js ...
2013-10-13T06:35:53.477+0000 [conn29] mongod.exe  ...\src\mongo\util\stacktrace.cpp(169)                        mongo::printStackTrace+0x43
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\util\log.cpp(131)                               mongo::logContext+0xc1
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\util\assert_util.cpp(131)                       mongo::fassertFailed+0xa9
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\util\stacktrace.cpp(285)                        mongo::crtDebugCallback+0x17f
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrptt.c(605)    _VCrtDbgReportW+0x812
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(242)     _CrtDbgReportWV+0x43
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(258)     _CrtDbgReportW+0x4d
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdthrow.cpp(13)  std::_Debug_message+0x34
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\algorithm(3667)    std::_Insertion_sort1<mongo::Interval * __ptr64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64),mongo::Interval>+0x1ad
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\algorithm(3798)    std::_Sort<mongo::Interval * __ptr64,__int64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0xdc
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\algorithm(3806)    std::sort<std::_Vector_iterator<std::_Vector_val<mongo::Interval,std::allocator<mongo::Interval> > >,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0x17c
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\index_bounds_builder.cpp(481)          mongo::IndexBoundsBuilder::unionize+0x60
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\index_bounds_builder.cpp(372)          mongo::IndexBoundsBuilder::translate+0x130f
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\index_bounds_builder.cpp(194)          mongo::IndexBoundsBuilder::translateAndUnion+0xe
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(375)                 mongo::QueryPlanner::mergeWithLeafNode+0x25b
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(595)                 mongo::QueryPlanner::processIndexScans+0x39e
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(729)                 mongo::QueryPlanner::buildIndexedOr+0xa8
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(798)                 mongo::QueryPlanner::buildIndexedDataAccess+0x5e5
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\query_planner.cpp(1197)                mongo::QueryPlanner::plan+0x1450
2013-10-13T06:35:53.478+0000 [conn29] mongod.exe  ...\src\mongo\db\query\new_find.cpp(263)                      mongo::getRunner+0x7aa
2013-10-13T06:35:53.478+0000 [conn29] test.jstests_or6 
2013-10-13T06:35:53.506+0000 [conn29] *** unhandled exception 0x80000003 at 0x000007FEFD713CA2, terminating
2013-10-13T06:35:53.506+0000 [conn29] *** stack trace for unhandled exception:
2013-10-13T06:35:54.104+0000 [conn29] KERNELBASE.dll                                                                DebugBreak+0x2
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      ...\src\mongo\util\assert_util.cpp(132)                       mongo::fassertFailed+0xb7
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      ...\src\mongo\util\stacktrace.cpp(285)                        mongo::crtDebugCallback+0x17f
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrptt.c(605)    _VCrtDbgReportW+0x812
2013-10-13T06:35:54.104+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(242)     _CrtDbgReportWV+0x43
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(258)     _CrtDbgReportW+0x4d
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdthrow.cpp(13)  std::_Debug_message+0x34
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\algorithm(3667)    std::_Insertion_sort1<mongo::Interval * __ptr64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64),mongo::Interval>+0x1ad
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\algorithm(3798)    std::_Sort<mongo::Interval * __ptr64,__int64,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0xdc
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\algorithm(3806)    std::sort<std::_Vector_iterator<std::_Vector_val<mongo::Interval,std::allocator<mongo::Interval> > >,bool (__cdecl*)(mongo::Interval const & __ptr64,mongo::Interval const & __ptr64)>+0x17c
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\index_bounds_builder.cpp(481)          mongo::IndexBoundsBuilder::unionize+0x60
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\index_bounds_builder.cpp(372)          mongo::IndexBoundsBuilder::translate+0x130f
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\index_bounds_builder.cpp(194)          mongo::IndexBoundsBuilder::translateAndUnion+0xe
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(375)                 mongo::QueryPlanner::mergeWithLeafNode+0x25b
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(595)                 mongo::QueryPlanner::processIndexScans+0x39e
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(729)                 mongo::QueryPlanner::buildIndexedOr+0xa8
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(798)                 mongo::QueryPlanner::buildIndexedDataAccess+0x5e5
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\query_planner.cpp(1197)                mongo::QueryPlanner::plan+0x1450
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\new_find.cpp(263)                      mongo::getRunner+0x7aa
2013-10-13T06:35:54.105+0000 [conn29] mongod.exe      ...\src\mongo\db\query\new_find.cpp(568)                      mongo::newRunQuery+0x2fd
2013-10-13T06:35:54.105+0000 [conn29] 
2013-10-13T06:35:54.106+0000 [conn29] writing minidump diagnostic file mongo.dmp
2013-10-13T06:35:55.030+0000 [conn29] *** immediate exit due to unhandled exception

Comment by J Rassi [ 14/Oct/13 ]

The fts_querylang.js and stages_text.js failures were fixed by 3a2776f0d66a03cd2c8de51117e14a0afffb716e.

Comment by Matt Kangas [ 14/Oct/13 ]

I dug through a few more logs and regex5.js is not a recurring pattern. But occasionally there is a stack trace.

http://buildlogs.mongodb.org/mci_0.9_windows_64_debug/builds/1292/test/parallel_0/basic.js

         S2 Test : jstests/fts_querylang.js ...
2013-10-12T07:40:59.289+0000 [conn29] CMD: drop test.fts_querylang
2013-10-12T07:40:59.291+0000 [conn29] build index on: test.fts_querylang properties: { "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.fts_querylang", "name" : "_id_" }
2013-10-12T07:40:59.292+0000 [conn29] build index done.  scanned 0 total records. 0 secs
2013-10-12T07:40:59.302+0000 [conn29] build index on: test.fts_querylang properties: { "v" : 1, "key" : { "_fts" : "text", "_ftsx" : 1 }, "ns" : "test.fts_querylang", "name" : "a_text", "weights" : { "a" : 1 }, "default_language" : "english", "language_override" : "language", "textIndexVersion" : 1 }
2013-10-12T07:40:59.303+0000 [conn29] build index done.  scanned 3 total records. 0 secs
2013-10-12T07:40:59.674+0000 [conn29] test.fts_querylang Assertion failure NULL != rawRunner src\mongo\db\query\new_find.cpp 551
2013-10-12T07:41:00.245+0000 [conn29] mongod.exe    ...\src\mongo\util\stacktrace.cpp(169)                           mongo::printStackTrace+0x43
2013-10-12T07:41:00.245+0000 [conn29] mongod.exe    ...\src\mongo\util\log.cpp(131)                                  mongo::logContext+0xc1
2013-10-12T07:41:00.245+0000 [conn29] mongod.exe    ...\src\mongo\util\assert_util.cpp(115)                          mongo::verifyFailed+0x190
2013-10-12T07:41:00.245+0000 [conn29] mongod.exe    ...\src\mongo\db\query\new_find.cpp(551)                         mongo::newRunQuery+0x352
2013-10-12T07:41:00.245+0000 [conn29] mongod.exe    ...\src\mongo\db\ops\query.cpp(1105)                             mongo::runQuery+0xb07
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    ...\src\mongo\db\instance.cpp(280)                               mongo::receivedQuery+0x2f8
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    ...\src\mongo\db\instance.cpp(443)                               mongo::assembleResponse+0x308
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    ...\src\mongo\db\db.cpp(197)                                     mongo::MyMessageHandler::process+0x158
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    ...\src\mongo\util\net\message_server_port.cpp(210)              mongo::PortMessageServer::handleIncomingMsg+0x7b9
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(185)  boost::`anonymous namespace'::thread_start_function+0x21
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314)      _callthreadstartex+0x25
2013-10-12T07:41:00.246+0000 [conn29] mongod.exe    f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(297)      _threadstartex+0xb4
2013-10-12T07:41:00.246+0000 [conn29] kernel32.dll                                                                   BaseThreadInitThunk+0xd
2013-10-12T07:41:00.246+0000 [conn29] test.fts_querylang 
2013-10-12T07:41:00.255+0000 [conn29] *** unhandled exception 0x80000003 at 0x000007FEFD7E3CA2, terminating
2013-10-12T07:41:00.255+0000 [conn29] *** stack trace for unhandled exception:
2013-10-12T07:41:00.818+0000 [conn29] KERNELBASE.dll                                                                   DebugBreak+0x2
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\util\assert_util.cpp(122)                          mongo::verifyFailed+0x3b6
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\db\query\new_find.cpp(551)                         mongo::newRunQuery+0x352
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\db\ops\query.cpp(1105)                             mongo::runQuery+0xb07
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\db\instance.cpp(280)                               mongo::receivedQuery+0x2f8
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\db\instance.cpp(443)                               mongo::assembleResponse+0x308
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\db\db.cpp(197)                                     mongo::MyMessageHandler::process+0x158
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\mongo\util\net\message_server_port.cpp(210)              mongo::PortMessageServer::handleIncomingMsg+0x7b9
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      ...\src\third_party\boost\libs\thread\src\win32\thread.cpp(185)  boost::`anonymous namespace'::thread_start_function+0x21
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(314)      _callthreadstartex+0x25
2013-10-12T07:41:00.819+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c(297)      _threadstartex+0xb4
2013-10-12T07:41:00.819+0000 [conn29] kernel32.dll                                                                     BaseThreadInitThunk+0xd
2013-10-12T07:41:00.819+0000 [conn29] 
2013-10-12T07:41:00.819+0000 [conn29] writing minidump diagnostic file mongo.dmp
2013-10-12T07:41:01.579+0000 [conn29] *** immediate exit due to unhandled exception

http://buildlogs.mongodb.org/mci_0.9_windows_64_debug/builds/1294/test/parallel_0/basic.js

         S1 Test : jstests/stages_text.js ...
2013-10-12T05:24:44.344+0000 [conn29] build index on: test.stages_text properties: { "v" : 1, "key" : { "_fts" : "text", "_ftsx" : 1 }, "ns" : "test.stages_text", "name" : "x_text", "weights" : { "x" : 1 }, "default_language" : "english", "language_override" : "language", "textIndexVersion" : 1 }
2013-10-12T05:24:44.345+0000 [conn29] build index done.  scanned 1 total records. 0 secs
2013-10-12T05:24:44.366+0000 [conn29] *** C runtime error: d:\Microsoft Visual Studio 10.0\VC\INCLUDE\vector(932) : Assertion failed: vector subscript out of range, terminating
2013-10-12T05:24:44.366+0000 [conn29] test Fatal Assertion 17006
2013-10-12T05:24:44.949+0000 [conn29] mongod.exe  ...\src\mongo\util\stacktrace.cpp(169)                        mongo::printStackTrace+0x43
2013-10-12T05:24:44.949+0000 [conn29] mongod.exe  ...\src\mongo\util\log.cpp(131)                               mongo::logContext+0xc1
2013-10-12T05:24:44.949+0000 [conn29] mongod.exe  ...\src\mongo\util\assert_util.cpp(131)                       mongo::fassertFailed+0xa9
2013-10-12T05:24:44.949+0000 [conn29] mongod.exe  ...\src\mongo\util\stacktrace.cpp(285)                        mongo::crtDebugCallback+0x17f
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrptt.c(605)    _VCrtDbgReportW+0x812
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(242)     _CrtDbgReportWV+0x43
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(258)     _CrtDbgReportW+0x4d
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdthrow.cpp(13)  std::_Debug_message+0x34
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  d:\microsoft visual studio 10.0\vc\include\vector(933)        std::vector<mongo::TextStage::ScoredLocation,std::allocator<mongo::TextStage::ScoredLocation> >::operator[]+0x3a
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\exec\text.cpp(72)                            mongo::TextStage::work+0x83
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\exec\fetch.cpp(83)                           mongo::FetchStage::work+0xdd
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\query\plan_executor.cpp(85)                  mongo::PlanExecutor::getNext+0x93
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\exec\stagedebug_cmd.cpp(127)                 mongo::StageDebugCmd::run+0x344
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1850)                         mongo::_execCommand+0x5e
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1998)                         mongo::Command::execCommand+0x14ca
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\dbcommands.cpp(2083)                         mongo::_runCommands+0x836
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\ops\query.cpp(69)                            mongo::runCommands+0x42
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\ops\query.cpp(1064)                          mongo::runQuery+0x576
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\instance.cpp(280)                            mongo::receivedQuery+0x2f8
2013-10-12T05:24:44.950+0000 [conn29] mongod.exe  ...\src\mongo\db\instance.cpp(443)                            mongo::assembleResponse+0x308
2013-10-12T05:24:44.950+0000 [conn29] test 
2013-10-12T05:24:44.970+0000 [conn29] *** unhandled exception 0x80000003 at 0x000007FEFCDE3CA2, terminating
2013-10-12T05:24:44.970+0000 [conn29] *** stack trace for unhandled exception:
2013-10-12T05:24:45.553+0000 [conn29] KERNELBASE.dll                                                                DebugBreak+0x2
2013-10-12T05:24:45.553+0000 [conn29] mongod.exe      ...\src\mongo\util\assert_util.cpp(132)                       mongo::fassertFailed+0xb7
2013-10-12T05:24:45.553+0000 [conn29] mongod.exe      ...\src\mongo\util\stacktrace.cpp(285)                        mongo::crtDebugCallback+0x17f
2013-10-12T05:24:45.553+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrptt.c(605)    _VCrtDbgReportW+0x812
2013-10-12T05:24:45.553+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(242)     _CrtDbgReportWV+0x43
2013-10-12T05:24:45.553+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\dbgrpt.c(258)     _CrtDbgReportW+0x4d
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      f:\dd\vctools\crt_bld\self_64_amd64\crt\src\stdthrow.cpp(13)  std::_Debug_message+0x34
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      d:\microsoft visual studio 10.0\vc\include\vector(933)        std::vector<mongo::TextStage::ScoredLocation,std::allocator<mongo::TextStage::ScoredLocation> >::operator[]+0x3a
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\exec\text.cpp(72)                            mongo::TextStage::work+0x83
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\exec\fetch.cpp(83)                           mongo::FetchStage::work+0xdd
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\query\plan_executor.cpp(85)                  mongo::PlanExecutor::getNext+0x93
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\exec\stagedebug_cmd.cpp(127)                 mongo::StageDebugCmd::run+0x344
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\dbcommands.cpp(1850)                         mongo::_execCommand+0x5e
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\dbcommands.cpp(1998)                         mongo::Command::execCommand+0x14ca
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\dbcommands.cpp(2083)                         mongo::_runCommands+0x836
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\ops\query.cpp(69)                            mongo::runCommands+0x42
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\ops\query.cpp(1064)                          mongo::runQuery+0x576
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\instance.cpp(280)                            mongo::receivedQuery+0x2f8
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\instance.cpp(443)                            mongo::assembleResponse+0x308
2013-10-12T05:24:45.554+0000 [conn29] mongod.exe      ...\src\mongo\db\db.cpp(197)                                  mongo::MyMessageHandler::process+0x158
2013-10-12T05:24:45.554+0000 [conn29] 
2013-10-12T05:24:45.555+0000 [conn29] writing minidump diagnostic file mongo.dmp
2013-10-12T05:24:46.057+0000 [conn29] *** immediate exit due to unhandled exception

Comment by Matt Kangas [ 14/Oct/13 ]

Rassi, is this yours?

From these two logs, it seems like the server is abruptly exiting. In both cases regex5.js is running at the time of failure.

http://buildlogs.mongodb.org/mci_0.9_windows_64_debug/builds/1327/test/parallel_0/basic.js

         S2 Test : jstests/regex5.js ...
now indexed
2013-10-14 01:32:08 EDT	
2013-10-14T05:32:08.929+0000 Socket recv() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T05:32:08.929+0000 Socket say send() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T05:32:08.931+0000 Error: socket exception [SEND_ERROR] for 127.0.0.1:27999 at src/mongo/shell/collection.js:202
********** Parallel Test FAILED: Error: error loading js file: jstests/orm.js
2013-10-14T05:32:08.935+0000 SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 
2013-10-14T05:32:08.935+0000 DBClientCursor::init call() failed
2013-10-14T05:32:08.936+0000 Socket say send() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T05:32:08.936+0000 Error: error doing query: failed at src/mongo/shell/query.js:81
********** Parallel Test FAILED: Error: error loading js file: jstests/regex5.js
2013-10-14T05:32:08.937+0000 Error: socket exception [SEND_ERROR] for 127.0.0.1:27999 at src/mongo/shell/collection.js:202
********** Parallel Test FAILED: Error: error loading js file: jstests/use_power_of_2.js
2013-10-14T05:32:08.950+0000 Socket recv() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T05:32:08.950+0000 SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 
2013-10-14T05:32:08.950+0000 DBClientCursor::init call() failed
2013-10-14T05:32:08.952+0000 Error: error doing query: failed at src/mongo/shell/collection.js:54
********** Parallel Test FAILED: Error: error loading js file: jstests/mr1.js
assert: [0] != [4] are not equal : one or more tests failed

http://buildlogs.mongodb.org/mci_0.9_windows_64_2k8_debug/builds/718/test/parallel_0/basic.js

         S2 Test : jstests/regex5.js ...
now indexed
2013-10-14 14:44:20 EDT	
2013-10-14T18:44:20.710+0000 Socket say send() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T18:44:20.710+0000 Socket recv() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T18:44:20.712+0000 Error: socket exception [SEND_ERROR] for 127.0.0.1:27999 at src/mongo/shell/collection.js:202
********** Parallel Test FAILED: Error: error loading js file: jstests/splitvector.js
2013-10-14T18:44:20.715+0000 Socket say send() errno:10054 An existing connection was forcibly closed by the remote host. 127.0.0.1:27999
2013-10-14T18:44:20.717+0000 Error: socket exception [SEND_ERROR] for 127.0.0.1:27999 at src/mongo/shell/collection.js:202
********** Parallel Test FAILED: Error: error loading js file: jstests/removeb.js
2013-10-14T18:44:20.730+0000 SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 
2013-10-14T18:44:20.730+0000 DBClientCursor::init call() failed
2013-10-14T18:44:20.733+0000 Error: error doing query: failed at src/mongo/shell/query.js:81
********** Parallel Test FAILED: Error: error loading js file: jstests/regex5.js
assert: [0] != [3] are not equal : one or more tests failed

Generated at Thu Feb 08 03:25:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.