-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Catalog and Routing
-
CAR Team 2024-05-13
SERVER-89182 adds a concurrency suite with random migrations and config shard transitions, and several tests can fail with LockBusy errors taking DDL locks for DDL operations or in the background metadata consistency checker. This seems most likely on slower variants, but not exclusively on them, and seems to only involve tests that do many DDL operations.
This ticket tracks diagnosing this failure and either fixing it or officially disabling the affected tests in the config shard transition suite.
The disabled tests:
- jstests/concurrency/fsm_workloads/agg_sort.js
- jstests/concurrency/fsm_workloads/map_reduce_interrupt.js
- jstests/concurrency/fsm_workloads/map_reduce_replace.js
- jstests/concurrency/fsm_workloads/map_reduce_replace_remove.js
- jstests/concurrency/fsm_workloads/map_reduce_replace_nonexistent.js
- jstests/concurrency/fsm_workloads/random_DDL_operations.js
- jstests/concurrency/fsm_workloads/rename_sharded_collection.js
Example failure:
2024-04-23T22:51:12.605Z assert: command failed: {
"ok" : 0,
"errmsg" : "Failed to acquire DDL lock for 'test30_fsmdb0' in mode S after 300000 ms that is currently locked by '{\"{ operation: \"{ namespace: \"test30_fsmdb0.tmp.agg_out.dfdbc260-8704-4fe2-8936-2723c8937651\", operationType: \"renameCollection_V2\" }\", lock mode: \"IX\" }\", \"{ operation: \"{ namespace: \"test30_fsmdb0.tmp.agg_out.dc87d861-1937-4991-8f53-16cc5375b11e\", operationType: \"renameCollection_V2\" }\", lock mode: \"IX\" }\", \"{ operation: \"{ namespace: \"test30_fsmdb0.tmp.agg_out.ea14da7e-1710-444b-9420-4fb390ed055e\", operationType: \"renameCollection_V2\" }\", lock mode: \"IX\" }\", \"{ operation: \"{ namespace: \"test30_fsmdb0.tmp.agg_out.9ade3ec4-31f3-4733-82a6-1c2cfe1fdc09\", operationType: \"renameCollection_V2\" }\", lock mode: \"IX\" }\", \"{ operation: \"{ namespace: \"test30_fsmdb0.tmp.agg_out.88e82a10-0b4d-4dc3-8b33-f2d8874e07fa\", operationType: \"renameCollection_V2\" }\", lock mode: \"IX\" }\", \"{ operation: \"{ namespace: \"test30_fsmdb0.map_reduce_replace3\", operationType: \"createCollection_V4\" }\", lock mode: \"IX\" }\"}'",
"code" : 46,
"codeName" : "LockBusy",
"$clusterTime" : {
"clusterTime" : Timestamp(1713912672, 20),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1713912672, 20)
} with original command request: {
"checkMetadataConsistency" : 1,
"lsid" : {
"id" : UUID("acceb5ff-7ffc-468a-8ea7-ecd7871a1b8f")
},
"$clusterTime" : {
"clusterTime" : Timestamp(1713912372, 34),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
} on connection: connection to localhost:20009,localhost:20010
_getErrorWithCode@src/mongo/shell/utils.js:24:13
doassert@src/mongo/shell/assert.js:18:14
_assertCommandWorked@src/mongo/shell/assert.js:761:25
assert.commandWorked@src/mongo/shell/assert.js:855:16
DB.prototype.checkMetadataConsistency@src/mongo/shell/db.js:1727:24
checkMetadataConsistency@/data/mci/99663d3669209ec618205300ac878a78/src/jstests/libs/check_metadata_consistency_helpers.js:52:45
run@/data/mci/99663d3669209ec618205300ac878a78/src/jstests/libs/check_metadata_consistency_helpers.js:61:13
@jstests/hooks/run_check_metadata_consistency.js:33:28
- related to
-
SERVER-88188 Check metadata consistency hook should not fail on LockBusy error
-
- Closed
-