We see inefficiency in disk I/O when read full database (either by primary or secondary key). Our investigation shows that WT opens database and immediately sets POSIX_FADV_RANDOM for db and index files. POSIX_FADV_WILLNEED is set only for a small number of pages (as we understand for internal pages only). A quick test with preloading an empty posix_faddvise() shows almost 2x speedup in full database read. For 3.9 GB database stored on spinning disk we have:
# regular posix_faddvise(): PK total time: 45.1785 seconds, 17530462 records # empty posix_faddvise(): PK total time: 25.8627 seconds, 17530462 records
It is a common scenario for us to do full database scan. Is it possible to optimize this use case?
- is duplicated by
-
SERVER-31177 MongoDB consumes all free memory, leading to throttled replication
-
- Closed
-
-
WT-3226 Investigate impact of FILE_FLAG_RANDOM_ACCESS flag on Windows
-
- Closed
-
- is related to
-
-
- Closed
-