[SERVER-32754] Add more verbose logging around shutdown ordering in storage engine Created: 18/Jan/18  Updated: 30/Oct/23  Resolved: 11/Jun/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.1.1

Type: Improvement Priority: Major - P3
Reporter: Alexander Gorrod Assignee: Sean Tao
Resolution: Fixed Votes: 0
Labels: neweng, nyc
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Sprint: Storage NYC 2018-06-04, Storage NYC 2018-06-18
Participants:
Linked BF Score: 27

 Description   

It would help diagnose some bugs if more messages were output during shutdown, specifically to identify races when closing the storage engine. Since all storage engine users need to be finished before the engine itself is closed.

A particular case that would be interesting is ensuring the checkpoint thread is finished prior to closing the storage engine.



 Comments   
Comment by Githook User [ 11/Jun/18 ]

Author:

{'username': 'jameswahlin', 'name': 'James Wahlin', 'email': 'james@mongodb.com'}

Message: SERVER-35044 Remove maxScan query option

SERVER-32754 Add more verbose logging around shutdown ordering in storage engine
Branch: master
https://github.com/mongodb/mongo/commit/b07dea7210fbe8ee16669ae0d2f54f929201ec2d

Comment by Sean Tao [ 05/Jun/18 ]

Sounds good. Thanks Alex!

Comment by Alexander Gorrod [ 05/Jun/18 ]

OK sean.tao If there isn't a way for the oplog maintenance thread to still be running on shutdown, it doesn't need a message.

Comment by Sean Tao [ 04/Jun/18 ]

Alex, nvm, I just found where it is called. However, if I understand the code correctly, it appears that this thread is particular to a namespace for a database and therefore not associated directly with the wired_tiger_kv_engine and not a member of this class. If this is not correct, please let me know, and I'll fix it! Thanks

Comment by Sean Tao [ 04/Jun/18 ]

Hi Alex. I added logging for the first two, but it appears as if the WiredTigerRecordStoreThread is not called anywhere, so I didn't add logging for it.

Comment by Alexander Gorrod [ 03/Jun/18 ]

sean.tao You asked in another forum if there are cases other than the checkpoint thread that it would be worthwhile adding logging for. I think any place that is using a BackgroundJob in the storage engine is a good candidate.

Searching shows me three interesting uses:

$ grep -rnI "public BackgroundJob" src/mongo/db/storage/
src/mongo/db/storage/mmap_v1/data_file_sync.h:39:class DataFileSync : public BackgroundJob, public ServerStatusSection {
src/mongo/db/storage/wiredtiger/wiredtiger_record_store_mongod.cpp:60:class WiredTigerRecordStoreThread : public BackgroundJob {
src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:206:class WiredTigerKVEngine::WiredTigerJournalFlusher : public BackgroundJob {
src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:250:class WiredTigerKVEngine::WiredTigerCheckpointThread : public BackgroundJob {

There is one that looks like a generic case WiredTigerRecordStoreThread - which is only used by initRsOplogBackgroundThread.

That is a long way to say: it seems as though it would be worthwhile to add logging for WiredTigerCheckpointThread, WiredTigerJournalFlusher and initRsOplogBackgroundThread.

Generated at Thu Feb 08 04:31:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.