Today, if we detect a source is idle, we will immediately cause the executor loop to sleep for 100 ms. With this behavior, we are likely seeing latency hits in workloads where we do not have a constant stream of input docs.

We can instead make this behavior more smoother by sleeping only if the source is idle for the last X millis or for the last X tries etc. The sleep duration can also start off small and increase for every consecutively idle period till it hits the max of 100ms.

So, something like this:

sleepMs := 0;
sourceLastActiveAt := 0;
while(true) {
sourceIdle := runLoopOnce()
if(!sourceIdle) {
  sourceLastActiveAt := now();
  sleepMs := 0;
} else {
  int intervals = (now() - sourceLastActiveAt)/10ms - 1;
  if(intervals >= 0) {
    sleepMs := min(2^^intervals , 100ms);
  }
}
sleepFor(sleepMs);
}