-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines, Storage Engines - Transactions
-
194.769
-
SE Persistence backlog
-
None
Background
Design: WRITING-39975. Epic: SPM-4886.
Change
Build a Grafana dashboard to watch the safety of clusters with the feature on, and run a staged rollout: internal dev clusters, then production clusters in widening steps. Each step runs long enough to include step-ups. Move on only when the dashboard is clean. If it shows a problem, turn the kill switch off.
Metrics:
- layered_curs_blind_remove_follower rate per node: above zero on every follower where the feature is on.
- Orphan removes per cluster (layered_curs_blind_remove_ingest_deleted + disagg_step_up_ingest_drain_orphan_remove): near zero, alert when rising.
- layered_curs_blind_remove_leader: zero.
- Validation failures and index errors: zero, alert on any.
- Server asserts, crashes and restarts: no increase compared with clusters where the feature is off.
Acceptance criteria
- Dashboard and alerts are live. Each rollout step is recorded as clean before the next.
Depends on
MongoDB testing ticket (proctor override removal).
- depends on
-
SERVER-137755 Remove the testing proctor override and run tests with blind remove enabled
-
- Backlog
-