-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Integration
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Problem
jstests/libs/override_methods/network_error_and_txn_override.js with wrapCRUDinTransactions:true only commits the currently-buffered wrapped transaction when a subsequent non-txn-compatible command arrives (see setupTransactionCommand, lines ~636--664: the else branch commits only when ops.length > 0 AND a non-CRUD command comes next). There is no end-of-test / shell-teardown hook that flushes a still-open wrapped txn.
For mochalite tests (see jstests/libs/mochalite.js) whose last it() body ends with a wrapped CRUD op, the final transaction is therefore never committed. The abandoned txn remains active on the primary for transactionLifetimeLimitSeconds (default 86400 in the multi-stmt-txn suites), and CheckReplOplogs / CheckReplDBHash then hang on the following fixture — observed as a task-timed-out with hang analyzer SIGKILL.
Concrete evidence
Reproduced locally on v8.2-staging with replica_sets_multi_stmt_txn_jscore_passthrough + jstests/core/index/geo/geo_s2stale_polygon_parse.js (3 mochalite iterations):
- Primary log shows New transaction started for txnNumber:0, :1, :2; only txnNumber:0 and :1 receive a Received commitTransaction. :2 is started, its insert/find succeed, and then no commit ever arrives.
- Immediately after, CheckReplOplogs freezes the secondary, fsync-locks the primary, and hangs indefinitely; WT stable timestamp is pinned at the last committed opTime and never advances.
- Test-shell side, the last override log line is the response to the third-iteration find. There is no setupTransactionCommand :: Committing transaction NumberLong(2) following it, matching the flow: after mochalite's final it() body returns, no further server command is issued, so the override's commit trigger never fires.
Analogous production symptoms in the sharded/replset multi-stmt-txn passthroughs on the same test:
- BF-44810 (multi_shard_multi_stmt_txn_jscore_passthrough / others, CheckReplDBHash timeout, {{CMD fsync
{lock:true}
}} in the log)
- BF-44833 (replica_sets_multi_stmt_txn_jscore_passthrough, CheckReplOplogs timeout, mongod SIGKILL from hang analyzer)
Prior art / related:
- SERVER-128348 — same override, same class of bug (txnRunCommandOverride retaining dangling txn state across mocha-style hooks in implicit_change_stream_v2.js).
- BF-43660 — production symptom of the same shape on disagg_change_streams_multi_stmt_txn_mongos_passthrough: commitTransaction@network_error_and_txn_override.js:538 frames with "Transaction was aborted / NoSuchTransaction."
SERVER-108224— canonical "mochalite bypasses passthrough runners" (fixed, runner-level only, does not touch txn flushing).- BF-38850 — earlier mochalite × override interaction closed with a revert.
Proposed fix
Install an end-of-test flush in network_error_and_txn_override.js that commits the currently-buffered wrapped txn if one is still open. Two triggers, so it covers both mochalite and non-mochalite tests:
- Wrap globalThis.__mochalite_closer at assignment time so that after mochalite's runTests returns, commitTransaction(lastConn, lastLsid, txnOptions.txnNumber) is invoked if ops.length > 0. The override never imports mochalite — intercepting the global assignment sidesteps the import boundary cleanly.
- Optionally, register a mongo-shell exit hook (if a suitable API exists) as a belt-and-suspenders path for non-mochalite tests that finish inside a wrapped txn.
Sketch:
function flushPendingWrappedTxn() { if (!TestData.networkErrorAndTxnOverrideConfig?.wrapCRUDinTransactions) return; if (ops.length === 0) return; logMsgFull('flushPendingWrappedTxn', `Committing dangling txn ${txnOptions.txnNumber} on session ` + `${tojsononeline(lastTxnLsid)} at test teardown`); try { commitTransaction(lastTxnConn, lastTxnLsid, txnOptions.txnNumber); } catch (e) { jsTest.log.info("flushPendingWrappedTxn: commit failed; aborting", {error: e}); try { abortTransaction(lastTxnConn, lastTxnLsid, txnOptions.txnNumber); } catch (_) {} } } let _closer; Object.defineProperty(globalThis, '__mochalite_closer', { configurable: true, get() { return _closer; }, set(fn) { _closer = async () => { try { await fn(); } finally { flushPendingWrappedTxn(); } }; }, });
Requires threading lastTxnConn/lastTxnLsid through continueTransaction. If the abandoned txn was already aborted server-side (SERVER-128348's failure mode), the commit will fail with NoSuchTransaction; the flusher should tolerate that as a no-op.
An alternative cleaner boundary: have mochalite.js invoke a TestData.txnOverrideFlushHook if present, and the override registers into it. Touches two files instead of one but avoids the Object.defineProperty intercept.
Scope
- Fix targets master; this ticket is not intended as a v8.2 backport candidate.
- v8.2-staging BF-44810 / BF-44833 will be mitigated separately by adding an after(...) hook to geo_s2stale_polygon_parse.js that issues a non-CRUD command (e.g. {{db.adminCommand(
{ping:1}
)}}), which triggers the override's existing commit path.
- Should also close out the class of failures represented by BF-43660 once landed.
Not addressed here
The interaction with fixture-level parameters (writePeriodicNoops=false, transactionLifetimeLimitSeconds=86400, disableLogicalSessionCacheRefresh=true) that turns an abandoned txn from "annoying" into "hangs for the full task timeout" is orthogonal — worth a separate look, but with this hook in place the abandoned-txn state should not exist in the first place.
- is related to
-
SERVER-108224 Mochalite tests bypass test runners defined by passthrough suites
-
- Closed
-
-
SERVER-128348 Make combination of implicit_change_stream_v2.js and txn overrides more robust
-
- Needs Scheduling
-
- related to
-
SERVER-131806 v8.2: Robustify geo_s2stale_polygon_parse.js
-
- Closed
-
-
SERVER-128348 Make combination of implicit_change_stream_v2.js and txn overrides more robust
-
- Needs Scheduling
-