-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: 8.0.0, 8.3.0, 9.1.0-rc0, 9.0.0
-
Component/s: None
-
None
-
Catalog and Routing
-
CAR Team 2026-09-14
-
0
-
None
-
None
-
None
-
None
-
None
-
None
-
None
[Test only]
Problem
The *_with_balancer_and_config_transitions_and_add_remove_shard test suites have dynamics that lead to slow shard removal and eventually timeout in slow variants:
- Random migrations continuously create more chunks for config.system.sessions.
- config.system.sessions isn't cleaned between tests --> Ever growing chunk count.
- removeShard needs to drain that ever growing pile of chunks out.
- Previous removeShard drainings also tend to leave the chunks unevenly balanced.
- The balancer in "random migrations" test-only mode prefers moveCollection (resharding) over draining 50% of the time, by default.
Potential solutions and mitigations
- Avoid random migration for config.system.sessions, or limit the number of chunk splits.
- Drop config.system.sessions in background or between tests.
- Merge config.system.sessions chunks before removeShard (note: may be problematic since chunk merges discarding the chunk history).
- Adjust skipMoveCollectionThreshold so the balancer favors draining.
- Change the balancer random migrations mode so it always prefers draining over moveCollection.
(2) seems the best long-term solution, but needs SERVER-133099 to avoid generating much tech debt.
(4) seems the simplest short term mitigation but skipMoveCollectionThreshold is only respected since SERVER-133663.
- is related to
-
SERVER-133663 [test-only] skipMoveCollectionThreshold field is ignored
-
- Closed
-
-
SERVER-133099 Add server parameter and allow to drop system.session for a sharded cluster
-
- In Code Review
-
-
SERVER-134588 Remove misleading ignore for sharded collections in configDB in move unsharded balancer policy
-
- In Code Review
-