Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2670

Inefficient I/O when read full DB (poor readahead)

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: WT2.8.0
    • Fix Version/s: 3.2.12, WT2.9.1, 3.5.1, 3.4.2
    • Labels:
      None
    • Sprint:
      Storage 2016-10-31, Storage 2016-11-21, Storage 2016-12-12

      Description

      We see inefficiency in disk I/O when read full database (either by primary or secondary key). Our investigation shows that WT opens database and immediately sets POSIX_FADV_RANDOM for db and index files. POSIX_FADV_WILLNEED is set only for a small number of pages (as we understand for internal pages only). A quick test with preloading an empty posix_faddvise() shows almost 2x speedup in full database read. For 3.9 GB database stored on spinning disk we have:

      # regular posix_faddvise():
      PK total time: 45.1785 seconds, 17530462 records
      # empty posix_faddvise():
      PK total time: 25.8627 seconds, 17530462 records
      

      It is a common scenario for us to do full database scan. Is it possible to optimize this use case?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sue.loverso Sue LoVerso
                Reporter:
                eiva Eugene Ivanov
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: