-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
DB Integration & Observability
-
QI 2025-02-17
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Problem
jstests/sharding/query/change_streams/express_id_lookup_missing_index_race.js fails flakily on mainline multiversion sharding tasks (e.g. sharding_last_lts_2-linux-enterprise), in the subtest "updateLookup succeeds via the aggregation fallback on a relocated post-image":
assert.eq\(delta\[engine\].notHandled, 1\) \-> expected 0 to equal 1 \(engine: "sbe"\)
The lookup itself succeeds via the aggregation fallback (aggregation.found = 1); only the SBE primary's decline counter never moves.
Root cause
- The subtest relies on the default of featureFlagChangeStreamOptimizedUpdateLookup and does not pin it (unlike its sibling subtests). It assumes the flag is default-on everywhere (see the comment in jstests/libs/query/change_stream_metrics_util.js).
- On v9.0 binaries the flag defaults to off (disabled for 9.0.2 by SERVER-134970; re-enablement tracked in SERVER-135788). With the flag off, buildUpdateLookupExecutor() (src/mongo/db/exec/agg/change_stream_add_post_image_stage_fn.cpp) builds only the aggregation executor, so SBE is never constructed and its counters stay at 0.
- The multiversion exclusion for this test previously hung off SERVER-130823, and self-lifted when SERVER-130823 was backported to v9.0 (PR #64184, carrying both the test and its backports_required_for_multiversion_tests.yml entries). The master test file now runs in sharding_last_lts against 9.0 binaries.
- It is flaky rather than deterministic because shard nodes are single-node replica sets with randomly selected binaries (last-lts vs latest) — it passes whenever a shard happens to pick a master binary.
Fix
Add the pair to etc/backports_required_for_multiversion_tests.yml under all three branch keys, next to the existing SERVER-130823 entries:
\- test\_file: jstests/sharding/query/change\_streams/express\_id\_lookup\_missing\_index\_race.js ticket: SERVER\-135788
- last-patch, last-continuous, and last-lts sections (last-patch 9.0.x binaries also have the flag off).
- Re-tagging under SERVER-135788 restores the exclusion and self-lifts exactly when the flag-enablement commit merges to v9.0 — the SERVER-135788 PR must carry this pair into v9.0's copy of the yml (like its other entries).
Also correct the now-false comment in jstests/libs/query/change_stream_metrics_util.js claiming the flag "is assumed enabled everywhere these tests run".
Related
- SERVER-135788 — [9.0] Enable featureFlagChangeStreamOptimizedUpdateLookup (exclusion lifts with its enablement PR)
- SERVER-134970 — Disable featureFlagChangeStreamOptimizedUpdateLookup (why the flag is off on 9.0)
- SERVER-130823 — the backport (#64184) that lifted the original exclusion
- related to
-
SERVER-134970 Disable featureFlagChangeStreamOptimizedUpdateLookup
-
- Closed
-
-
SERVER-130823 Change stream with update lookup can trip server assertion
-
- Closed
-
-
SERVER-135788 [9.0] Enable featureFlagChangeStreamOptimizedUpdateLookup
-
- Open
-