Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32013

Ensure consistent getMore polling behaviour for sharded $changeStream

    • Query Execution
    • Fully Compatible

      Sharded $changeStream requires a regular stream of updates from each shard in order to return results to the client as soon as possible. To achieve this, we poll each shard with getMores until it returns a batch of data. If we request a getMore for a particular shard and it returns an empty batch after timing out, the ARM will automatically reschedule additional getMores until a valid batch is received.

      However, there are two circumstances in which polling of shards 'stalls':

      • If we are in kGetMoreWithAtLeastOneResultInBatch context and we hit !_arm.ready(), we return immediately without kicking off a new round of getMores on the shard(s) whose buffers have been exhausted.
      • For kInitialFind we always return an empty batch immediately regardless of maxTimeMS or batchSize, and do not schedule any shard getMores until the following client getMore is received.

      Previously we attempted to extend the ARM's existing auto-reschedule behaviour for these scenarios, but this required scheduling getMores in the ARM constructor and in nextReady, neither of which was appropriate. However, the same behaviour can be appropriately (and trivially) achieved in RouterStageMerge, by simply scheduling and stashing an ARM event just before we return EOF from kInitialFind or kGetMoreWithAtLeastOneResultInBatch context. We already use this approach when we time out while waiting for results in kGetMoreNoResultsYet context.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            bernard.gorman@mongodb.com Bernard Gorman
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: