[SERVER-19818] Cache usage exceeds configured maximum during index builds under WiredTiger Created: 07/Aug/15  Updated: 03/Oct/15  Resolved: 03/Oct/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Bruce Lucas (Inactive)
Resolution: Duplicate Votes: 0
Labels: RF, WTmem
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: HTML File index-build-exceeds-cache.html     PNG File index-build-exceeds-cache.png    
Issue Links:
Depends
Duplicate
duplicates SERVER-20159 Out of memory on index build during i... Closed
Related
related to SERVER-20159 Out of memory on index build during i... Closed
Operating System: ALL
Participants:

 Description   

Note: this issue is similar to SERVER-18829, but the fix there was verified on the repro on that ticket and the particulars are different in this case, so I believe this is a different issue.

During an _id index build of a collection with about 5G documents in an initial sync of a secondary cache usage grew to exceed the specified maximum. Some stats:

  • The index build starts at A. At this point (and for the duration of the timeline) we are only reading the collection data; no index entries are being inserted yet.
  • Since the cache is full pages are being evicted as fast as they are being read, and cache usage remains bounded.
  • But at B the statistics shift. Data is still being read at the same rate, but evictions are not being done in the same way, and don't keep up with the rate at which data is being read into the cache, so cache usage grows unbounded.


 Comments   
Comment by Michael Cahill (Inactive) [ 07/Aug/15 ]

bruce.lucas@mongodb.com, I can't figure this one out from the stats, sorry – it is very odd that eviction worker threads stop evicting pages, and that the number of pages walked jumps so high, but I don't yet understand why.

Generated at Thu Feb 08 03:52:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.