-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Workload Resilience
-
None
-
None
-
None
-
None
-
None
-
None
-
None
SPM-4263 introduces a low priority queue for expensive operations so that they are de-prioritized compared to high-priority/short-running operations, limiting how many of them can be running at the same time with a ticket pool.
However it doesn't put a cap on the number of operations that can be "active" (i.e. have started execution and are holding onto some resources) which means we can easily consume a lot of resources (specially memory) if many of these expensive/long-running operations are admitted.
Limiting the number of operations that can queue in the low priority queue has the undesirable effect of potentially killing operations that had already done a lot of work and where close to finish.
This ticket proposes adding a concurrency limiter at admission time that will reject new expensive/long running queries before they consume significant resources.