[SERVER-84357] Exclude the oplog when auto compaction is enabled by default Created: 20/Dec/23  Updated: 10/Jan/24

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Etienne Petrel Assignee: Backlog - Storage Engines Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-83548 Exclude the oplog when executing the ... Closed
is related to SERVER-83734 Create a server parameter to enable b... Closed
Assigned Teams:
Storage Engines
Sprint: StorEng - Refinement Pipeline, 2024-01-09 - I Grew Tired
Participants:
Story Points: 3

 Description   

SERVER-83734 made it possible to enable background compaction by default when gEnableAutoCompaction is set to true.
In SERVER-83548, we decided to exclude the oplog by default when the autoCompact command is executed.

We should exclude the oplog as well when background compaction is enabled through gEnableAutoCompaction.

This ticket should ensure the oplog is excluded whether background compaction is enabled through the autoCompact command or gEnableAutoCompaction. Furthermore, we should avoid duplicating code. If possible, we should create a function that would definite the collections that are excluded by default.



 Comments   
Comment by Etienne Petrel [ 09/Jan/24 ]

With the following test:

buildscripts/resmoke.py run --suites=replica_sets --continueOnFailure --excludeWithAnyTags=incompatible_with_amazon_linux,requires_external_data_source --jobs=2 --shuffle --runAllFeatureFlagTests --storageEngineCacheSizeGB=1 jstests/replsets/clean_shutdown_oplog_state.js

I found that in notifyStartupComplete, both LocalOplogInfo::get(opCtx)>getCollection() and _oplogRecordStore are null. However, when startOplogManager is called, _oplogRecordStore is assigned to the newly created oplog but LocalOplogInfo::get(opCtx)>getCollection() remains null.

Question:

  • Should we rely on _oplogRecordStore instead?
  • Should we move the start of background compaction in startOplogManager from notifyStartupComplete?
    • However, if there is no oplog, that's probably not the right function as background compaction would never be enabled. The caller is WiredTigerRecordStore::postConstructorInit, would that be a better place? It looks like it is called a lot, so probably not.

cc'ing you gregory.wlodarek@mongodb.com as you may have more context regarding background compaction and what we added in notifyStartupComplete.

Comment by Etienne Petrel [ 08/Jan/24 ]

louis.williams@mongodb.com, I have also noticed the field _oplogRecordStore, would it be ok to use this one instead of what you suggested?

Generated at Thu Feb 08 06:54:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.