- Create a collection consisting of 10 M documents each 1 kB in size
- Start mongod with a cache size of 10 GB, a little too small to hold the data set (incl index and overhead)
- Start reading documents at random
- At point A we have read most of the index into the cache - "pages read into cache" falls to near 0; whereas we are continuing to read collection pages into cache all the way through B. This reflects the fact that index pages contain many more entries than collection pages, so we hit all the index pages much more quickly than we hit the collection pages.
- At the point B where the cache fills up we see collection pages being evicted at a rate of about 1k/s, and the rate of reading collection pages into the cache does not change when the cache fills: lru is essentially random in this case, and evicting a random collection page will not change the chance of a cache miss on the next random read, so cache miss rate states constant after the cache fills up
- However we see index pages being evicted at nearly 4 k/s, about 3-4x the rate of collection pages, in spite of the fact that index pages are much "hotter" than collection pages so should be kept in cache by lru, and as a result rate of reads of index into the cache goes very high.