[SERVER-45847] Pull the JournalFlusher out of the storage engine layer and place it above the storage engine Created: 29/Jan/20  Updated: 29/Oct/23  Resolved: 17/Mar/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.7.0, 4.4.5

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Dianna Hohensee (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-46826 Instantiate the JournalFlusher thread... Closed
is depended on by SERVER-46870 Generalize the WiredTigerRecordStore:... Closed
Duplicate
is duplicated by SERVER-22575 Make periodic write flushing a servic... Closed
Related
related to SERVER-46524 ephemeralForTest cannot run new prima... Closed
related to SERVER-45665 Make JournalFlusher flush on command ... Closed
related to SERVER-47385 Complete TODO listed in SERVER-45847 Closed
is related to SERVER-55374 [v4.4] Backport original SERVER-50045... Closed
is related to SERVER-46191 Make waiting for the truncate point i... Backlog
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4
Sprint: Execution Team 2020-03-23
Participants:
Linked BF Score: 0

 Description   

This requires moving the WiredTigerJournalFlush above the storage layer, and establishing a new code layer – perhaps StorageControl. This new code layer should have things like all the oplog collection specific code currently splattered across the generic WiredTigerRecordStore class.



 Comments   
Comment by Githook User [ 23/Feb/21 ]

Author:

{'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@mongodb.com', 'username': 'DiannaHohensee'}

Message: SERVER-45847 Fix incorrect error code in unit test
Branch: v4.4
https://github.com/mongodb/mongo/commit/d9eb255ba3d295131874b4d4588be133c5026772

Comment by Githook User [ 22/Feb/21 ]

Author:

{'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@mongodb.com', 'username': 'DiannaHohensee'}

Message: SERVER-45847 Pull JournalFlusher thread out of WT and place it above the storage engine layer.

Creating a new storage/control/ directory that contains StorageControl to handle manipulation of
independent processes performing actions against the storage engine for higher level code layers.

create mode 100644 src/mongo/db/storage/control/journal_flusher.cpp
create mode 100644 src/mongo/db/storage/control/journal_flusher.h
create mode 100644 src/mongo/db/storage/control/storage_control.cpp
create mode 100644 src/mongo/db/storage/control/storage_control.h

(cherry picked from commit 6af5a08234f59cd00650a8e3b141a56b6a5ebf14)
Branch: v4.4
https://github.com/mongodb/mongo/commit/0cec2cc63f1279fa75f0867e04cf804bdc7a378e

Comment by Githook User [ 17/Mar/20 ]

Author:

{'email': 'dianna.hohensee@mongodb.com', 'name': 'Dianna Hohensee', 'username': 'DiannaHohensee'}

Message: SERVER-45847 Pull JournalFlusher thread out of WT and place it above the storage engine layer.

Creating a new storage/control/ directory that contains StorageControl to handle manipulation of
independent processes performing actions against the storage engine for higher level code layers.

create mode 100644 src/mongo/db/storage/control/journal_flusher.cpp
create mode 100644 src/mongo/db/storage/control/journal_flusher.h
create mode 100644 src/mongo/db/storage/control/storage_control.cpp
create mode 100644 src/mongo/db/storage/control/storage_control.h
Branch: master
https://github.com/mongodb/mongo/commit/6af5a08234f59cd00650a8e3b141a56b6a5ebf14

Comment by Dianna Hohensee (Inactive) [ 02/Mar/20 ]

Adding a todo reference in SERVER-46524 to this task to clean up a temporary hack for which this work will eliminate the need.

Comment by Dianna Hohensee (Inactive) [ 29/Jan/20 ]

I wonder if we could either eliminate the JournalListener, or at least move it up to wrap around the JournalFlusher's – whatever we rename it too – waitUntilDurable call. This would get the JournalListener out of the WiredTigerSessionCache:waitUntilDurable code.

Also, it would reduce future risk of racing to set the truncate point in the JournalListener with unsetting it for stepdown.

(move the journal_listener.h/cpp out of storage/, too)

Could even make the JournalFlusher (whatever we name it) stop on stepdown and start on stepup: all the other modes call waitUntilDurable manually for new data writes? Might not be 100% accurate.

Generated at Thu Feb 08 05:09:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.