[SERVER-46826] Instantiate the JournalFlusher thread for ephemeral engines and when non-durable (nojournal=true) Created: 12/Mar/20 Updated: 29/Oct/23 Resolved: 25/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.7.0, 4.4.5 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Dianna Hohensee (Inactive) | Assignee: | Dianna Hohensee (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Backport Requested: |
v4.4
|
||||||||||||||||||||||||
| Sprint: | Execution Team 2020-03-23, Execution Team 2020-04-06, Execution Team 2020-04-20, Execution Team 2020-05-04, Execution Team 2020-05-18, Execution Team 2020-06-01, Execution Team 2020-06-15, Execution Team 2020-06-29, Execution Team 2020-09-07 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Linked BF Score: | 38 | ||||||||||||||||||||||||
| Description |
|
Always create the JournalFlusher, regardless of ephemeral or non-durable storage engine settings, because using it to wait for write concern should be a performance gain for all settings and also simplifies the logic. Today we skip creating the JournalFlusher thread for both ephemeral or non-durable (nojournal=true) storage engine settings. Originally, the JournalFlusher was purely a periodically running task. However, recently, that changed. The JournalFlusher is now a periodic task that can also be pinged for an immediate run. Moving the write_concern.cpp waitUntilDurable calls onto the JournalFlusher thread via waitForJournalFlush has been a big performance win. Currently, we use logic in waitForJournalFlush to skip the JournalFlusher thread when it doesn’t exist and instead call waitUntilDurable directly. We would want to override the periodical functionality of the JournalFlusher thread when ephemeral or !durable, I think, and make it purely run when requested. Taking checkpoints so frequently for non-durable (nojournal) might slow down the system; and ephemeral only updates the JournalListener (durable timestamp). |
| Comments |
| Comment by Githook User [ 23/Feb/21 ] | |||||||||||||||||||||||||
|
Author: {'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@mongodb.com', 'username': 'DiannaHohensee'}Message: Non-durable engines will not run the JournalFlusher logic periodically, but only upon request, (cherry picked from commit 4609f3ebfb178f37153bc04678176af722b0d304) | |||||||||||||||||||||||||
| Comment by Githook User [ 25/Aug/20 ] | |||||||||||||||||||||||||
|
Author: {'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@mongodb.com', 'username': 'DiannaHohensee'}Message: Non-durable engines will not run the JournalFlusher logic periodically, but only upon request, | |||||||||||||||||||||||||
| Comment by Dianna Hohensee (Inactive) [ 24/Aug/20 ] | |||||||||||||||||||||||||
|
Code review url: https://mongodbcr.appspot.com/662400001/ | |||||||||||||||||||||||||
| Comment by Dianna Hohensee (Inactive) [ 08/Jun/20 ] | |||||||||||||||||||||||||
|
Well, that idea completely doesn't work. Per this code, the NetworkInterfaceMock unique_ptr ownership is given to the ThreadPoolTaskExecutor, whose ownership is in turn given to the ReplicationCoordinatorImpl instance. Given this kind of ownership, there's no way to extend the lifetime of the NetworkInterfaceMock past the ReplCoordTest lifetime... | |||||||||||||||||||||||||
| Comment by Dianna Hohensee (Inactive) [ 08/Jun/20 ] | |||||||||||||||||||||||||
|
Ran into an interesting problem:
It seems like I'd have to set up a NetworkInterfaceMock in unit tests running the JournalFlusher. Then higher level test fixtures can use that NetworkInterfaceMock, and the NetworkInterfaceMock will remain valid when the JournalFlusher needs to access it through the clock source. |