Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-34447

Deduplicate concurrent chunk split requests for the same chunk

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Duplicate
    • Icon: Major - P3 Major - P3
    • None
    • None
    • Sharding
    • None
    • Sharding 2018-06-04, Sharding 2018-06-18, Sharding 2018-07-02

    Description

      With the new autosplitting design, a thread pool is used to asynchronously execute chunk splits. This can lead to scenarios where, for example, a request to split one chunk is submitted, more writes come in, and another request to split the same chunk is submitted before the first is executed. Those two tasks could then either be executed concurrently or sequentially. This can lead to:

      • Potentially duplicated work, if two threads concurrently process conflicting work corresponding to the same chunk, and only one succeeds
      • If you’re loading a lot into a chunk that’s currently being split you could end up with the task list growing large until the chunk is split, followed by a bunch of useless work when the chunk is finally split
      • Almost unbounded growth of the task queue, if your splitting can't keep up with writes

      One solution to this is to keep track of which chunks have split tasks currently pending/executing, and when a new task is submitted it should be checked against the list of pending tasks. If it corresponds to the same chunk as a currently executing task, it should return silently without doing any work.

      Attachments

        Activity

          People

            matthew.saltz@mongodb.com Matthew Saltz (Inactive)
            matthew.saltz@mongodb.com Matthew Saltz (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: