Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2647

Improve page-out behavior for OpLog

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      The current heuristic for append-only tables (such as the mongodb oplog) causes recently written pages to be evicted from memory in order to avoid polluting the cache. This means that when the page is read it must be read from disk (or OS cache) then decompressed before results can be returned. On a system with high write load, this increases the work done in the critical path to getting oplog data to downstream nodes in a replica set.

      One solution we discussed in person would be for the uncompressed page images for the oplog to be held in cache after they've been written to disk. MongoDB could then periodically inform WT of the lowest position that is still "interesting" so that older pages can still be rapidly evicted. "Interesting" can be loosely defined as the position of the furthest behind replica that is still up, possibly also considering the position of any oplog-tailing cursors created by users.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                backlog-server-storage Backlog - Storage NY Team
                Reporter:
                redbeard0531 Mathias Stearn
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: