-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Execution
-
None
-
None
-
None
-
None
-
None
-
None
-
None
This ticket decomposes pipelined-apply latency into per-stage counters under serverStatus().repl.pipelinedApply.
This ticket should include:
- Micros counters: batcherWaitMicros, dispatchMicros, backpressureMicros, drainMicros, inlineApplyMicros, publishWaitMicros, workerApplyMicros, workerIdleMicros, advancerPublishMicros, advancerMultikeyMicros.
- Backpressure trigger counter: backpressure.inflight
- A workerIdleMicros hook in the PipelinedApplierWorkerPool — the pool has no idle callback today, so one must be added.
edit: updated list of counters:
dispatcher loop:
- request batch (batcherWaitMicros)
- if must serialize:
- drain (see below)
- dispatch to one worker (serializedApplyMicros, see below)
- drain again (see below)
- else, if can parallelize:
- dispatch to workers (parallelApplyMicros, see below)
drain:
- start timer for drainWorkers
- wait for worker pool to become idle
- wait for advancer to become idle
- (every time the advancer waits for a new batch from the tracker, it updates publishWaitMicros)
- (every time the advancer publishes, it updates advancerPublishMicros and/or advancerMultikeyMicros)
dispatch:
- start dispatch timer for dispatchMicros
- pause timer
- add batch to batch tracker
- check if backpressure necessary (backpressureMicros)
- assign batches to workers (see below)
- resume timer
worker loop:
- wait for a batch
- process the batch (workerApplyMicros)