Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2670

Inefficient I/O when read full DB (poor readahead)

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • WT2.8.0
    • WT2.9.1, 3.2.12, 3.4.2, 3.5.1
    • None
    • None
    • Storage 2016-10-31, Storage 2016-11-21, Storage 2016-12-12

    Description

      We see inefficiency in disk I/O when read full database (either by primary or secondary key). Our investigation shows that WT opens database and immediately sets POSIX_FADV_RANDOM for db and index files. POSIX_FADV_WILLNEED is set only for a small number of pages (as we understand for internal pages only). A quick test with preloading an empty posix_faddvise() shows almost 2x speedup in full database read. For 3.9 GB database stored on spinning disk we have:

      # regular posix_faddvise():
      PK total time: 45.1785 seconds, 17530462 records
      # empty posix_faddvise():
      PK total time: 25.8627 seconds, 17530462 records
      

      It is a common scenario for us to do full database scan. Is it possible to optimize this use case?

      Attachments

        Issue Links

          Activity

            People

              sue.loverso@mongodb.com Susan LoVerso
              eiva Eugene Ivanov
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: