Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12886

Make non-fastmod updates perform work proportional to the update size

    XMLWordPrintable

    Details

    • Case:

      Description

      Currently non-fastmod updates generate I/O proportional to the size of the document (which is pathological for large documents and multi-updates). See the linked and other related tickets to high write I/O cause by update amplification.

      A proposed fix is two-fold:

      1. Only write back dirty pages (i.e. implement SERVER-66)
      2. Move Object / Array types which change size to the end of their parent Document
      1. should ensure that we minimize I/O at little cost to clients
      2. should ensure that we don't constantly re-write unchanged fields in a document.

      I believe 2 is OK, because:

      • drivers use HashMaps (unless using an explicit SON type) which doesn't guarantee order
      • Mongo re-orders fields when moving an expanded document

      This is related, but different, to:
      SERVER-12732
      SERVER-12578

      I've attempted implementations of 1 + 2 which seem to give 2 orders of magnitude improvement to performance of the benchmarks on the linked ticket:

      1. https://github.com/jamesblackburn/mongo/commit/54b5f6b56374630e4a872021e424e6c476c02cff
      2. https://github.com/jamesblackburn/mongo/commit/54b5f6b56374630e4a872021e424e6c476c02cff

        Attachments

        1. mongostat.py
          3 kB
        2. perf-after.txt
          5 kB
        3. perf-before.txt
          5 kB
        4. version_store.py
          6 kB

          Activity

            People

            Assignee:
            backlog-server-execution Backlog - Storage Execution Team
            Reporter:
            jblackburn James Blackburn
            Participants:
            Votes:
            2 Vote for this issue
            Watchers:
            15 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: