[SERVER-39321] Re-enable the CheckReplDBHashInBackground hook Created: 31/Jan/19  Updated: 29/Oct/23  Resolved: 25/Apr/19

Status: Closed
Project: Core Server
Component/s: Replication, Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.1.11

Type: Task Priority: Major - P3
Reporter: Vesselina Ratcheva (Inactive) Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: open_todo_in_code, prepare_testing, txn_storage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-39169 Add testing-only support for doing sn... Closed
is depended on by SERVER-40086 Extend CheckReplDBHashInBackground to... Closed
Related
related to SERVER-39372 Make secondary lock acquisition for D... Closed
is related to SERVER-39096 Prepared transactions and DDL operati... Closed
is related to SERVER-39139 Remove testing support for secondary ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage NYC 2019-04-08, Storage NYC 2019-04-22, Storage NYC 2019-05-06
Participants:

 Description   

All usages of the hook were temporarily removed as part of SERVER-39139. This ticket serves as a placeholder for the work of re-enabling those usages, as well as to more easily track where they should be put back.



 Comments   
Comment by Githook User [ 25/Apr/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-39321 Add testing around the semnatics of the find and dbHash commands when using the option in the presence of prepared transactions
Branch: master
https://github.com/mongodb/mongo/commit/db0e16980ad20eeb9d4c7cc7e07f64409fc3edb9

Comment by Githook User [ 25/Apr/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-39321 Re-enable the CheckReplDBHashInBackground hook
Branch: master
https://github.com/mongodb/mongo/commit/5ae506cb8fc73074bb46d6be9667ed1825e98d69

Comment by Githook User [ 25/Apr/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-39321 add setIgnorePrepared(false) to dbHash, find and getMore
Branch: master
https://github.com/mongodb/mongo/commit/935157a3cf54ca802419cd8d92ff4b137fbe2949

Comment by Max Hirschhorn [ 08/Apr/19 ]

I'd like to discuss with Max the specifics of the new tests he proposed.

milkie, I had added calls to opCtx->recoveryUnit()->setIgnorePrepared(false) without adding testing around the semantics of the find, getMore, and dbHash commands when using the $_internalReadAtClusterTime option in the presence of prepared transactions. Since the $_internalReadAtClusterTime option is effectively simulating one-shot reads with {readConcern: {level: "snapshot", atClusterTime: ...}}, it must wait for decision of the prepared transaction before returning data back to the client if the commit/abort timestamp would fall within the window of the specified cluster time. However, the requirement for the $_internalReadAtClusterTime option is that it must not prevent oplog application while doing so.

To more explicitly enumerate the cases for testing I have in mind:

  1. Perform a write in prepared transaction, followed by a write outside of a transaction. It should be possible to specify $_internalReadAtClusterTime as the timestamp of the second write without error, but it should block until the prepared transaction is committed or aborted. Finally, commit the prepared transaction.
    • Does SERVER-40105 give enough of an indication for the find, getMore, or dbHash operation that it is blocked on a prepare conflict?
  2. Do #1 but instead read from a secondary. Additionally also perform a variety of DDL operations that are compatible with the locks taken by the primary (these can likely be lifted from the apply_ops_DDL_operation_does_not_take_global_X.js test) before committing the prepared transaction.
    • Assert that the data returned and/or the hash returned by the secondary matches that of the primary. Repeat the reads on both the primary and secondary after committing the prepared transaction and assert that the data returned and/or the hash returned the second time matches that of the first time. The dbhash_read_at_cluster_time.js test does both already but not in the presence of prepared transactions.
Comment by Eric Milkie [ 03/Apr/19 ]

The priority for this ticket work will be to re-enable the hook and investigate any failures. I'd like to discuss with Max the specifics of the new tests he proposed.

Comment by Max Hirschhorn [ 05/Feb/19 ]

Work on this ticket should include adding a test that (1) ensures the find, getMore, and dbHash commands block when attempting to read prepared data, and (2) ensure these operations being blocked doesn't prevent oplog application from proceeding. #2 requires one of SERVER-39096 or SERVER-39372 to be completed.

Comment by Vesselina Ratcheva (Inactive) [ 04/Feb/19 ]

The commit for the removal left some TODOs, which are meant to be addressed here.

Generated at Thu Feb 08 04:51:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.