Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-4313

sharded systems should throttle writes when chunk migrations cannot keep up

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: Sharding
    • Labels:
      None

      Description

      In some cases data may be arriving at a shard faster than it can be migrated elsewhere. The typical situation where this is seen is on a brand new collection which does not have many chunks yet, to which a very large bulk import is sent. In this situation, it is necessary to throttle the injection at the shard; if injest rate is, for a long period of time, higher than migration output rate, the node will "overflow" with either data or workload (often the latter). Presplitting is a good technique to avoid this situation currently.

      Thus this ticket is to throttle writes s.t. migrations do not fall behind.

      For "phase one" inserts are the only thing that needs throttling. We can make a phase 2 ticket after that.

        Attachments

          Activity

            People

            Assignee:
            backlog-server-sharding Backlog - Sharding Team
            Reporter:
            dwight_10gen Dwight Merriman
            Participants:
            Votes:
            3 Vote for this issue
            Watchers:
            10 Start watching this issue

              Dates

              Created:
              Updated: