[SERVER-18052] Race condition in collection_cloner_test Created: 14/Apr/15  Updated: 25/Jan/17  Resolved: 15/Apr/15

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 3.1.2

Type: Bug Priority: Major - P3
Reporter: Andy Schwerin Assignee: Benety Goh
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-17894 Add Collection Cloner to Data Replicator Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:
  1. Build collection_cloner_test with --opt=off and --dbg=on
  2. Run the following shell script:

    i=0; while ./build/linux2/dbg/mongo/db/repl/collection_cloner_test ; do (( ++i )); done; echo $i
    

Sprint: RPL 2 04/24/15
Participants:

 Description   

There is a race condition in collection_cloner_test. The BeginCollectionCallbackCanceled test fails sometimes at line 385 of collection_cloner_test.cpp expecting CallbackCanceled but seeing InternalError.

It also sometimes fails in InsertDocumentsCallbackCanceled, at line 583 expecting CallbackCanceled but getting OK.



 Comments   
Comment by Githook User [ 15/Apr/15 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-18052 fixed race condition in CollectionClonerTest
Branch: master
https://github.com/mongodb/mongo/commit/d4a9b806ddb4b05cb9729afd78a24d1c4934ee44

Comment by Andy Schwerin [ 14/Apr/15 ]

Sample output from reproduction steps:

2015-04-14T14:38:40.894-0400 I -        going to run suite: CollectionClonerTest
2015-04-14T14:38:40.894-0400 I -        	 going to run test: InvalidConstruction
2015-04-14T14:38:40.895-0400 I -        	 going to run test: GetDiagnosticString
2015-04-14T14:38:40.895-0400 I -        	 going to run test: IsActiveAfterStart
2015-04-14T14:38:40.895-0400 I -        	 going to run test: StartWhenActive
2015-04-14T14:38:40.895-0400 I -        	 going to run test: CancelWithoutStart
2015-04-14T14:38:40.896-0400 I -        	 going to run test: WaitWithoutStart
2015-04-14T14:38:40.896-0400 I -        	 going to run test: ShutdownBeforeStart
2015-04-14T14:38:40.896-0400 I -        	 going to run test: StartAndCancel
2015-04-14T14:38:40.896-0400 I -        	 going to run test: StartButShutdown
2015-04-14T14:38:40.896-0400 I -        	 going to run test: FirstRemoteCommand
2015-04-14T14:38:40.897-0400 I -        	 going to run test: RemoteCollectionMissing
2015-04-14T14:38:40.897-0400 I -        	 going to run test: ListIndexesReturnedNoIndexes
2015-04-14T14:38:40.897-0400 W REPL     [ReplicationExecutor] No indexes found for collection db.coll while cloning from localhost:27017
2015-04-14T14:38:40.897-0400 I -        	 going to run test: BeginCollectionScheduleDbWorkFailed
2015-04-14T14:38:40.897-0400 I -        	 going to run test: BeginCollectionCallbackCanceled
2015-04-14T14:38:40.898-0400 I -        FAIL: BeginCollectionCallbackCanceled	Expected ErrorCodes::CallbackCanceled == status.code() (CallbackCanceled == InternalError) @src/mongo/db/repl/collection_cloner_test.cpp:385
2015-04-14T14:38:40.898-0400 I -        	 going to run test: BeginCollectionFailed
2015-04-14T14:38:40.898-0400 I -        	 going to run test: BeginCollection
2015-04-14T14:38:40.899-0400 I -        	 going to run test: FindFetcherScheduleFailed
2015-04-14T14:38:40.899-0400 I -        	 going to run test: FindCommandAfterBeginCollection
2015-04-14T14:38:40.899-0400 I -        	 going to run test: FindCommandFailed
2015-04-14T14:38:40.900-0400 I -        	 going to run test: FindCommandCanceled
2015-04-14T14:38:40.900-0400 I -        	 going to run test: InsertDocumentsScheduleDbWorkFailed
2015-04-14T14:38:40.900-0400 I -        	 going to run test: InsertDocumentsCallbackCanceled
2015-04-14T14:38:40.901-0400 I -        	 going to run test: InsertDocumentsFailed
2015-04-14T14:38:40.901-0400 I -        	 going to run test: InsertDocumentsSingleBatch
2015-04-14T14:38:40.901-0400 I -        	 going to run test: InsertDocumentsMultipleBatches
2015-04-14T14:38:40.902-0400 I -        	 DONE running tests
2015-04-14T14:38:40.902-0400 I -        **************************************************
2015-04-14T14:38:40.902-0400 I -        CollectionClonerTest           | tests:   25 | fails:    1 | assert calls:          0 | time secs:  0.007
	BeginCollectionCallbackCanceled	Expected ErrorCodes::CallbackCanceled == status.code() (CallbackCanceled == InternalError) @src/mongo/db/repl/collection_cloner_test.cpp:385
2015-04-14T14:38:40.902-0400 I -        TOTALS                         | tests:   25 | fails:    1 | assert calls:          0 | time secs:  0.007
2015-04-14T14:38:40.902-0400 I -        Failing tests:
2015-04-14T14:38:40.902-0400 I -        	 CollectionClonerTest/BeginCollectionCallbackCanceled Failed
2015-04-14T14:38:40.902-0400 I -        FAILURE - 1 tests in 1 suites failed
23

and

2015-04-14T14:39:57.586-0400 I -        going to run suite: CollectionClonerTest
2015-04-14T14:39:57.586-0400 I -        	 going to run test: InvalidConstruction
2015-04-14T14:39:57.587-0400 I -        	 going to run test: GetDiagnosticString
2015-04-14T14:39:57.587-0400 I -        	 going to run test: IsActiveAfterStart
2015-04-14T14:39:57.587-0400 I -        	 going to run test: StartWhenActive
2015-04-14T14:39:57.587-0400 I -        	 going to run test: CancelWithoutStart
2015-04-14T14:39:57.588-0400 I -        	 going to run test: WaitWithoutStart
2015-04-14T14:39:57.588-0400 I -        	 going to run test: ShutdownBeforeStart
2015-04-14T14:39:57.588-0400 I -        	 going to run test: StartAndCancel
2015-04-14T14:39:57.588-0400 I -        	 going to run test: StartButShutdown
2015-04-14T14:39:57.588-0400 I -        	 going to run test: FirstRemoteCommand
2015-04-14T14:39:57.589-0400 I -        	 going to run test: RemoteCollectionMissing
2015-04-14T14:39:57.589-0400 I -        	 going to run test: ListIndexesReturnedNoIndexes
2015-04-14T14:39:57.589-0400 W REPL     [ReplicationExecutor] No indexes found for collection db.coll while cloning from localhost:27017
2015-04-14T14:39:57.589-0400 I -        	 going to run test: BeginCollectionScheduleDbWorkFailed
2015-04-14T14:39:57.590-0400 I -        	 going to run test: BeginCollectionCallbackCanceled
2015-04-14T14:39:57.590-0400 I -        	 going to run test: BeginCollectionFailed
2015-04-14T14:39:57.590-0400 I -        	 going to run test: BeginCollection
2015-04-14T14:39:57.591-0400 I -        	 going to run test: FindFetcherScheduleFailed
2015-04-14T14:39:57.591-0400 I -        	 going to run test: FindCommandAfterBeginCollection
2015-04-14T14:39:57.591-0400 I -        	 going to run test: FindCommandFailed
2015-04-14T14:39:57.592-0400 I -        	 going to run test: FindCommandCanceled
2015-04-14T14:39:57.592-0400 I -        	 going to run test: InsertDocumentsScheduleDbWorkFailed
2015-04-14T14:39:57.593-0400 I -        	 going to run test: InsertDocumentsCallbackCanceled
2015-04-14T14:39:57.593-0400 I -        FAIL: InsertDocumentsCallbackCanceled	Expected ErrorCodes::CallbackCanceled == status.code() (CallbackCanceled == OK) @src/mongo/db/repl/collection_cloner_test.cpp:583
2015-04-14T14:39:57.593-0400 I -        	 going to run test: InsertDocumentsFailed
2015-04-14T14:39:57.594-0400 I -        	 going to run test: InsertDocumentsSingleBatch
2015-04-14T14:39:57.594-0400 I -        	 going to run test: InsertDocumentsMultipleBatches
2015-04-14T14:39:57.595-0400 I -        	 DONE running tests
2015-04-14T14:39:57.595-0400 I -        **************************************************
2015-04-14T14:39:57.595-0400 I -        CollectionClonerTest           | tests:   25 | fails:    1 | assert calls:          0 | time secs:  0.008
	InsertDocumentsCallbackCanceled	Expected ErrorCodes::CallbackCanceled == status.code() (CallbackCanceled == OK) @src/mongo/db/repl/collection_cloner_test.cpp:583
2015-04-14T14:39:57.595-0400 I -        TOTALS                         | tests:   25 | fails:    1 | assert calls:          0 | time secs:  0.008
2015-04-14T14:39:57.595-0400 I -        Failing tests:
2015-04-14T14:39:57.595-0400 I -        	 CollectionClonerTest/InsertDocumentsCallbackCanceled Failed
2015-04-14T14:39:57.595-0400 I -        FAILURE - 1 tests in 1 suites failed
49

Generated at Thu Feb 08 03:46:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.