[SERVER-10609] BtreeTests1::NoMoveAtLowWaterMarkLeft failure on Win32 Created: 23/Aug/13  Updated: 28/Oct/13  Resolved: 27/Sep/13

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: 2.5.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Matt Kangas Assignee: Unassigned
Resolution: Done Votes: 0
Labels: buildbot
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File windows_32bit_build_6153_core.txt    
Participants:

 Description   

Frickin' weird. What are the odds we can repro this? Not great, I suspect.

Builder Windows 32-bit Build #6153 Thu Aug 22, rev e71ca210ed9

http://buildlogs.mongodb.org/Windows%2032-bit/builds/6153/test/core/test.exe

2013-08-22T23:29:46.093+0000 [testsuite] going to run suite: btree1
...
2013-08-22T23:29:46.186+0000 [testsuite] 	 going to run test: class BtreeTests1::NoMoveAtLowWaterMarkRight
2013-08-22T23:29:46.186+0000 [testsuite] 	 going to run test: class BtreeTests1::MoveBelowLowWaterMarkRight
2013-08-22T23:29:46.186+0000 [testsuite] 	 going to run test: class BtreeTests1::NoMoveAtLowWaterMarkLeft
2013-08-22T23:29:46.280+0000 [testsuite] *** unhandled exception (access violation) at 0x01544668, terminating
2013-08-22T23:29:46.280+0000 [testsuite] *** access violation was a write to 0x07863020
2013-08-22T23:29:46.280+0000 [testsuite] *** stack trace for unhandled exception:
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\db\btree.cpp(363)                     mongo::BucketBasics<mongo::BtreeData_V1>::_pushBack+0x208
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\dbtests\btreetests.inl(502)           BtreeTests1::ArtificialTree::push+0x101
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\dbtests\btreetests.inl(861)           BtreeTests1::MergeSizeBase::run+0xf2
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.h(297)              mongo::unittest::Suite::runTestObject<BtreeTests1::NoMoveAtLowWaterMarkLeft>+0x5d
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(164)            mongo::unittest::Suite::run+0x5d2
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(228)            mongo::unittest::Suite::run+0x4fd
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\dbtests\framework.cpp(269)            mongo::dbtests::runDbTests+0x1243
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\dbtests\dbtests.cpp(40)               dbtestsMain+0x114
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      ...\src\mongo\dbtests\dbtests.cpp(51)               wmain+0x4f
2013-08-22T23:29:46.997+0000 [testsuite] test.exe      f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c(278)  __tmainCRTStartup+0x10b
2013-08-22T23:29:46.997+0000 [testsuite] kernel32.dll                                                      BaseThreadInitThunk+0x12
2013-08-22T23:29:46.997+0000 [testsuite] ntdll.dll                                                         RtlInitializeExceptionChain+0x63
2013-08-22T23:29:46.997+0000 [testsuite] 
2013-08-22T23:29:46.997+0000 [testsuite] writing minidump diagnostic file mongo.dmp
2013-08-22T23:29:47.653+0000 [testsuite] *** immediate exit due to unhandled exception

The only slightly fishy thing I see in this test preceding this access violation are a few occurrences of this Assertion failure. Unrelated?

2013-08-22T23:29:13.332+0000 [testsuite] 	 going to run test: EmptyArrayTags
2013-08-22T23:29:13.332+0000 [testsuite]  Assertion failure !_isExhausted src\mongo\client\dbclient_rs.cpp 2122
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\util\log.cpp(131)                           mongo::logContext+0xa7
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\util\assert_util.cpp(113)                   mongo::verifyFailed+0x166
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\client\dbclient_rs.cpp(2124)                mongo::TagSet::getCurrentTag+0x1c
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\dbtests\replica_set_monitor_test.cpp(1514)  mongo_test::UnitTest__TagSet__EmptyArrayTags::_doTest+0xeb
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(119)                  mongo::unittest::Test::run+0x47
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.h(297)                    mongo::unittest::Suite::runTestObject<mongo_test::UnitTest__TagSet__EmptyArrayTags>+0x41
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(164)                  mongo::unittest::Suite::run+0x5d2
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(228)                  mongo::unittest::Suite::run+0x4fd
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\dbtests\framework.cpp(269)                  mongo::dbtests::runDbTests+0x1243
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\dbtests\dbtests.cpp(40)                     dbtestsMain+0x114
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      ...\src\mongo\dbtests\dbtests.cpp(51)                     wmain+0x4f
2013-08-22T23:29:13.972+0000 [testsuite] test.exe      f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c(278)        __tmainCRTStartup+0x10b
2013-08-22T23:29:13.972+0000 [testsuite] kernel32.dll                                                            BaseThreadInitThunk+0x12
2013-08-22T23:29:13.972+0000 [testsuite] ntdll.dll                                                               RtlInitializeExceptionChain+0x63
2013-08-22T23:29:13.972+0000 [testsuite]  



 Comments   
Comment by Matt Kangas [ 27/Aug/13 ]

Hm, another one.

Nightly Windows 32-bit Build #1406 Tue Aug 27 (rev 820d678f3)

http://buildlogs.mongodb.org/Nightly%20Windows%2032-bit/builds/1406/test/core/test.exe

2013-08-27T15:08:52.177+0000 [testsuite] going to run suite: btree1
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::Create
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::SimpleInsertDelete
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::SplitRightHeavyBucket
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::SplitLeftHeavyBucket
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::MissingLocate
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::MissingLocateMultiBucket
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::SERVER983
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::DontReuseUnused
2013-08-27T15:08:52.177+0000 [testsuite] 	 going to run test: class BtreeTests1::PackUnused
2013-08-27T15:08:52.208+0000 [testsuite] 	 going to run test: class BtreeTests1::DontDropReferenceKey
2013-08-27T15:08:52.208+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeBucketsLeft
2013-08-27T15:08:52.208+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeBucketsRight
2013-08-27T15:08:52.208+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeBucketsDontReplaceHead
2013-08-27T15:08:52.208+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeBucketsDelInternal
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeBucketsRightNull
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::DontMergeSingleBucket
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::ParentMergeNonRightToLeft
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::ParentMergeNonRightToRight
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::CantMergeRightNoMerge
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::CantMergeLeftNoMerge
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeOption
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::ForceMergeLeft
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::ForceMergeRight
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::RecursiveMerge
2013-08-27T15:08:52.223+0000 [testsuite] 	 going to run test: class BtreeTests1::RecursiveMergeRightBucket
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::RecursiveMergeDoubleRightBucket
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeSizeJustRightRight
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeSizeJustRightLeft
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeSizeRight
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeSizeLeft
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::NoMergeBelowMarkRight
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::NoMergeBelowMarkLeft
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeSizeRightTooBig
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::MergeSizeLeftTooBig
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceOneLeftToRight
2013-08-27T15:08:52.239+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceOneRightToLeft
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceThreeLeftToRight
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceThreeRightToLeft
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceSingleParentKey
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::PackEmpty
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::PackedDataSizeEmpty
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceSingleParentKeyPackParent
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::BalanceSplitParent
2013-08-27T15:08:52.255+0000 [testsuite] 	 going to run test: class BtreeTests1::EvenRebalanceLeft
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::EvenRebalanceLeftCusp
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::EvenRebalanceRight
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::EvenRebalanceRightCusp
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::EvenRebalanceCenter
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::OddRebalanceLeft
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::OddRebalanceRight
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::OddRebalanceCenter
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::RebalanceEmptyRight
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::RebalanceEmptyLeft
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::NoMoveAtLowWaterMarkRight
2013-08-27T15:08:52.270+0000 [testsuite] 	 going to run test: class BtreeTests1::MoveBelowLowWaterMarkRight
2013-08-27T15:08:52.286+0000 [testsuite] 	 going to run test: class BtreeTests1::NoMoveAtLowWaterMarkLeft
2013-08-27T15:08:52.333+0000 [testsuite] *** unhandled exception (access violation) at 0x00DA37E8, terminating
2013-08-27T15:08:52.333+0000 [testsuite] *** access violation was a write to 0x078B3020
2013-08-27T15:08:52.333+0000 [testsuite] *** stack trace for unhandled exception:
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\db\btree.cpp(363)                     mongo::BucketBasics<mongo::BtreeData_V1>::_pushBack+0x208
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\dbtests\btreetests.inl(502)           BtreeTests1::ArtificialTree::push+0x101
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\dbtests\btreetests.inl(861)           BtreeTests1::MergeSizeBase::run+0xf2
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.h(297)              mongo::unittest::Suite::runTestObject<BtreeTests1::NoMoveAtLowWaterMarkLeft>+0x5d
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(164)            mongo::unittest::Suite::run+0x5d2
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\unittest\unittest.cpp(228)            mongo::unittest::Suite::run+0x4fd
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\dbtests\framework.cpp(269)            mongo::dbtests::runDbTests+0x1243
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\dbtests\dbtests.cpp(40)               dbtestsMain+0x114
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      ...\src\mongo\dbtests\dbtests.cpp(51)               wmain+0x4f
2013-08-27T15:08:53.144+0000 [testsuite] test.exe      f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c(278)  __tmainCRTStartup+0x10b
2013-08-27T15:08:53.144+0000 [testsuite] kernel32.dll                                                      BaseThreadInitThunk+0x12
2013-08-27T15:08:53.144+0000 [testsuite] ntdll.dll                                                         RtlInitializeExceptionChain+0x63
2013-08-27T15:08:53.144+0000 [testsuite] 
2013-08-27T15:08:53.144+0000 [testsuite] writing minidump diagnostic file mongo.dmp
2013-08-27T15:08:53.783+0000 [testsuite] *** immediate exit due to unhandled exception

Comment by Matt Kangas [ 26/Aug/13 ]

For this issue we now have two data points on one builder, a MBTF of 6 weeks, and no way to reproduce. Closing until we have more data or a better angle of attack.

Comment by Matt Kangas [ 23/Aug/13 ]

Output of http://buildlogs.mongodb.org/Windows%2032-bit/builds/6153/test/core/test.exe?mode=raw

Comment by Matt Kangas [ 23/Aug/13 ]

Next build ran the unit tests cleanly

http://buildbot.mongodb.org:8081/builders/Windows%2032-bit/builds/6154

Will attach complete output of the failed test run for posterity's sake.

Comment by Matt Kangas [ 23/Aug/13 ]

Looks like this is almost exactly the same thing as SERVER-10157, except that failed on `BtreeTests1::MoveBelowLowWaterMarkRight`. This is Left.

Hari burned a whole day trying to repro that one with no luck.

Generated at Thu Feb 08 03:23:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.