[SERVER-28500] Implicitly sharding dropped collection can fail if test is running with parallel shell Created: 24/Mar/17  Updated: 06/Dec/17  Resolved: 30/Mar/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.5.6

Type: Task Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Eddie Louie
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-26315 sharded_collections_jscore_passthroug... Closed
Backwards Compatibility: Fully Compatible
Sprint: TIG 2017-04-17
Participants:
Linked BF Score: 0

 Description   

The "shardCollection" command may fail with an ConflictingOperationInProgress error response if a parallel shell started by the test runs the "drop" command. This is known to happen when running the queryoptimizer3.js test as part of the sharded_collections_jscore_passthrough suite.

[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 2017-03-16T22:27:54.977+0000 E QUERY    [thread1] Error: command failed: {
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"code" : 117,
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"ok" : 0,
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"errmsg" : "Collection was successfully written as sharded but got dropped before it could be evenly distributed",
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"logicalTime" : {
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 		"clusterTime" : Timestamp(1489703274, 47),
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 		"signature" : BinData(0,"BzSDcnuxcSxhtKKsU3yXUkRM+kw=")
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	}
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 } : sharding 'test.jstests_queryoptimizer3' with a hashed _id key failed :
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 _getErrorWithCode@src/mongo/shell/utils.js:25:13
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 doassert@src/mongo/shell/assert.js:16:14
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 assert.commandWorked@src/mongo/shell/assert.js:387:5
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 shardCollection@jstests/libs/override_methods/implicitly_shard_accessed_collections.js:47:9
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 DBCollection.prototype.drop@jstests/libs/override_methods/implicitly_shard_accessed_collections.js:69:9
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 @jstests\core\queryoptimizer3.js:10:5
[js_test:queryoptimizer3] 2017-03-16T22:27:54.982+0000 failed to load: jstests\core\queryoptimizer3.js

We can override the startParallelShell() function so that it sets a variable in an outer scope that controls whether or not we should swallow an ConflictingOperationInProgress error response. This way we can still run these tests as part of the sharded_collections_jscore_passthrough suite and are much less likely to mask the ConflictingOperationInProgress error response being returned in other contexts for the "shardCollection" command.

var testMayRunDropInParallel = false;
var originalStartParallelShell = startParallelShell;
 
function shardCollection(collection) {
    ...
    res = db.adminCommand({shardCollection: fullName, key: {_id: 'hashed'}});
    if (res.ok === 0 && testMayRunDropInParallel) {
        assert.commandFailedWithCode(res, ErrorCodes.ConflictingOperationInProgress);
    } else {
        assert.commandWorked(res, "sharding '" + fullName + "' with a hashed _id key failed");
    }
}
 
startParallelShell = function() {
    testMayRunDropInParallel = true;
    return originalStartParallelShell.apply(this, arguments);
}



 Comments   
Comment by Githook User [ 29/Mar/17 ]

Author:

{u'username': u'elouie99', u'name': u'Eddie Louie', u'email': u'eddie.louie@mongodb.com'}

Message: SERVER-28500 Absorb a ConflictingOperationInProgress error during a parallel implicit sharding drop collection command
Branch: master
https://github.com/mongodb/mongo/commit/fee13bbec0b177a20fc994439c630b142a815fd1

Comment by Eddie Louie [ 29/Mar/17 ]

Thanks max.hirschhorn for setting this up. I've put the code together as you've suggested.

Generated at Thu Feb 08 04:18:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.