Implicitly sharding dropped collection can fail if test is running with parallel shell

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Done
    • Priority: Major - P3
    • 3.5.6
    • Affects Version/s: None
    • Component/s: Testing Infrastructure
    • None
    • Fully Compatible
    • TIG 2017-04-17
    • 0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The "shardCollection" command may fail with an ConflictingOperationInProgress error response if a parallel shell started by the test runs the "drop" command. This is known to happen when running the queryoptimizer3.js test as part of the sharded_collections_jscore_passthrough suite.

      [js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 2017-03-16T22:27:54.977+0000 E QUERY    [thread1] Error: command failed: {
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"code" : 117,
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"ok" : 0,
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"errmsg" : "Collection was successfully written as sharded but got dropped before it could be evenly distributed",
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	"logicalTime" : {
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 		"clusterTime" : Timestamp(1489703274, 47),
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 		"signature" : BinData(0,"BzSDcnuxcSxhtKKsU3yXUkRM+kw=")
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 	}
[js_test:queryoptimizer3] 2017-03-16T22:27:54.979+0000 } : sharding 'test.jstests_queryoptimizer3' with a hashed _id key failed :
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 _getErrorWithCode@src/mongo/shell/utils.js:25:13
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 doassert@src/mongo/shell/assert.js:16:14
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 assert.commandWorked@src/mongo/shell/assert.js:387:5
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 shardCollection@jstests/libs/override_methods/implicitly_shard_accessed_collections.js:47:9
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 DBCollection.prototype.drop@jstests/libs/override_methods/implicitly_shard_accessed_collections.js:69:9
[js_test:queryoptimizer3] 2017-03-16T22:27:54.980+0000 @jstests\core\queryoptimizer3.js:10:5
[js_test:queryoptimizer3] 2017-03-16T22:27:54.982+0000 failed to load: jstests\core\queryoptimizer3.js

      We can override the startParallelShell() function so that it sets a variable in an outer scope that controls whether or not we should swallow an ConflictingOperationInProgress error response. This way we can still run these tests as part of the sharded_collections_jscore_passthrough suite and are much less likely to mask the ConflictingOperationInProgress error response being returned in other contexts for the "shardCollection" command.

      var testMayRunDropInParallel = false;
var originalStartParallelShell = startParallelShell;

function shardCollection(collection) {
    ...
    res = db.adminCommand({shardCollection: fullName, key: {_id: 'hashed'}});
    if (res.ok === 0 && testMayRunDropInParallel) {
        assert.commandFailedWithCode(res, ErrorCodes.ConflictingOperationInProgress);
    } else {
        assert.commandWorked(res, "sharding '" + fullName + "' with a hashed _id key failed");
    }
}

startParallelShell = function() {
    testMayRunDropInParallel = true;
    return originalStartParallelShell.apply(this, arguments);
}

            Assignee:
            Eddie Louie (Inactive)
            Reporter:
            Max Hirschhorn
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: