Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29817

Optimize incremental update performance of ChunkManager and CollectionMetadata

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.7, 3.5.11
    • Component/s: Sharding
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v3.4
    • Case:

      Description

      The ChunkManager and CollectionMetadata types are effectively in-memory routing tables used by map shard key values to chunks and shards. These data structures are read only, and updated copies must be made after chunk migrations. Because the data structures are updated by copy, their update time is proportional to the total number of chunks in a given collection. When the number of chunks in a collection is large (10s of thousands), the update time is significant, and increases the latency of chunk migration commit.

      This ticket tracks improvements to the constant factors affecting update time. Prototyping indicates that a speedup of >4x is possible by replacing woCompare with KeyString comparison, and by replacing all but one instance of std::map with instances of std::vector that are constructed in sorted order.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                3 Vote for this issue
                Watchers:
                21 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: