Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1560

log server could fsync and close old log files

    • Type: Icon: Task Task
    • Resolution: Done
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      It would be good to move handling of fsync and close of old log files out of the path of the thread that happens to open a new one and move it into the log server thread. Otherwise random threads may encounter longer latencies. This issue arises from WT-1545 and WT-1555.

      Here are random thoughts about this:

      • Should there be a no-sync path to avoid fsync before close? Currently there is not. The problem is that, if someone has all the sync options turned off, but then wants to force a sync via transaction_begin, it requires us to open/fsync/close all earlier log files that exist. That is a major amount of time for that sync.
      • Would we want two (or more) log_server threads then? If the single log_server thread is doing a long fsync, then log file pre-allocation does not happen (nor archiving, but that is off the critical path anyway).
      • We could signal the log file close in a couple places. First when we notice and create a new log file, but that has the disadvantage of requiring the log server thread to wait for earlier writes to complete via write_lsn. We could keep similar logic as we have today in _log_newfile and _log_release but instead of performing the fsync and close, it gives the log server the old fh and signals it. That way the log server can assume (and assert) the write_lsn is later.
      • Currently log_newfile waits for earlier log files to complete closing before moving to a new one if we've wrapped an entire log file before the previous one finished (i.e if we're moving from log 3 to log 4, we make sure log 2 is closed). We may need to add a chain of log files to close for the log server instead because we don't want to make log_newfile wait on other work the log_server thread is doing. (Maybe that bolsters the case for another, dedicated thread.)

            Assignee:
            sue.loverso@mongodb.com Susan LoVerso
            Reporter:
            sue.loverso@mongodb.com Susan LoVerso
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: