[SERVER-30339] Enabling profiling and read concern majority with a workload of reads and no writes causes accumulation of dirty data Created: 26/Jul/17 Updated: 06/Dec/22 Resolved: 10/Nov/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Write Ops |
| Affects Version/s: | 3.2.15, 3.4.6, 3.5.5 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Bruce Lucas (Inactive) | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
Start a replica set with these options:
Then start a workload of reads but no writes:
Range of pinned ids and dirty data grows, and instance hangs when dirty data in cache reaches 20%:
|
| Comments |
| Comment by Bruce Lucas (Inactive) [ 26/Jul/17 ] |
|
The write every 10 seconds was introduced by |
| Comment by Bruce Lucas (Inactive) [ 26/Jul/17 ] |
|
Looks like the behavior is better in 3.4 and 3.5:
We write an entry to the oplog every 10 seconds even if there are no writes, and that is sufficient to bound the range of pinned ids and dirty data to 10 seconds worth of profile entries. That may be a sufficient fix since this only occurs when there are no other writes generating dirty data. |
| Comment by Bruce Lucas (Inactive) [ 26/Jul/17 ] |
|
It appears that the dirty data is due to entries written to the profile, but because there are no db writes that advance the oplog a named snapshot remains in place for an extended period, causing the accumulation of pinned transactions and dirty data. |