[SERVER-52950] recoverOplogAsStandalone mode must not start oplog truncater thread Created: 19/Nov/20  Updated: 29/Oct/23  Resolved: 07/Dec/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.9.0, 4.4.3, 4.2.12, 4.0.23

Type: Bug Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4, v4.2, v4.0
Sprint: Execution Team 2020-12-14
Participants:
Case:

 Description   

This is observed on 4.0, but I imagine applies to 4.4 and 4.2.

The oplog truncater thread checks for readonly, but not for recoverFromOplogAsStandalone. At startup with recoverFromOplogAsStandalone, the node will open WT in read+write mode and perform replication recovery. After replication recovery, the readOnly flag gets flipped to prevent external users from doing writes.

Because the readonly flag is flipped after the oplog truncater thread is started up, the truncater thread can attempt to perform a write that will fail.



 Comments   
Comment by Githook User [ 09/Jan/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-52950 Add jstest tag for check_for_oplog_cap_maintainer_thread.js
Branch: v4.0
https://github.com/mongodb/mongo/commit/2516e9546b1b5df6e15d2f93e526e7218fa953f0

Comment by Githook User [ 09/Jan/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-52950 recoverFromOplogAsStandalone mode must not start oplog truncater thread

(cherry picked from commit 3417215850d7a577452552499ea55d1872199d74)
Branch: v4.0
https://github.com/mongodb/mongo/commit/8495ff679afa2c3a13d3456e3f3f1742d8431c85

Comment by Githook User [ 18/Dec/20 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-52950 recoverFromOplogAsStandalone mode must not start oplog truncater thread
Branch: v4.2
https://github.com/mongodb/mongo/commit/3417215850d7a577452552499ea55d1872199d74

Comment by Githook User [ 14/Dec/20 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-52950 Avoid calculating oplog stones when recoverFromOplogAsStandalone=true as the OplogCapMaintainerThread is not running

(cherry picked from commit 3be0c64532e121780dd11788fa3db4a9089b008c)
Branch: v4.4
https://github.com/mongodb/mongo/commit/2146f93edb068f8020666cb0c495fc555a49415c

Comment by Githook User [ 05/Dec/20 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-52950 Avoid calculating oplog stones when recoverFromOplogAsStandalone=true as the OplogCapMaintainerThread is not running
Branch: master
https://github.com/mongodb/mongo/commit/3be0c64532e121780dd11788fa3db4a9089b008c

Comment by Gregory Wlodarek [ 01/Dec/20 ]

The change in SERVER-40168 which resides in v4.4+ refactored the startup logic of the background thread. So the work will be slightly different when backporting this change to v4.2 and v4.0.

In versions >= v4.4:

  • initializeStorageControlsForReplication(), which initializes and starts the OplogCapMaintainerThread is only called when using --replSet and recoverFromOplogAsStandalone is false.
    • --replSet cannot be used in conjunction with --queryableBackupMode: "Cannot specify both queryable backup mode and replication.replSet".
    • --repair terminates the process before the replication coordinator starts up.
  • To avoid calculating oplog stones during startup when recoverFromOplogAsStandalone is true, an optimization can be made here. This will avoid unnecessary work and speed up startup.

In versions < v4.4:

  • During startup, initRsOplogBackgroundThread() is checked here. This function is responsible for starting the OplogTruncaterThread if it isn't already running only when we're not in repair or read-only mode.
  • All that's needed here is to check whether the recoverFromOplogAsStandalone flag is set to true.

 

tldr; versions >= v4.4 don't start the background thread but still calculate the oplog stones, and versions < v4.4 need to stop starting the background thread when recoverFromOplogAsStandalone=true.

Generated at Thu Feb 08 05:29:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.