[SERVER-66609] Avoid global lock upgrade in storage_timestamp_test.cpp Created: 20/May/22  Updated: 29/Oct/23  Resolved: 03/Jun/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Yujin Kang Park Assignee: Yujin Kang Park
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File SERVER-66609_backtrace.txt     Text File SERVER-66609_print_stack_on_global_lock.log     Text File backtrace.log    
Issue Links:
Depends
depends on SERVER-61733 Determine if global lock upgrade is n... Closed
is depended on by SERVER-60621 Investigate if we can ban upgrading t... Closed
Related
is related to SERVER-61788 Determine if we can avoid global lock... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2022-06-13
Participants:

 Description   

Investigation to ban global lock upgrades is ongoing in SERVER-60621. While working on dependent SERVER-61788, avoid the lock upgrade in StorageTimestampTest.SecondaryInsertTimes, we have found that MultiDocumentTransactionTest also fails due to a global lock upgrade from MODE_IS to MODE_IX in the constructor of MongoDOperationContextSession.

This issue does not appear to be with the test itself, but something general.

Attached backtrace.log.
Comment from Dan Larkin-York.



 Comments   
Comment by Githook User [ 03/Jun/22 ]

Author:

{'name': 'Yu Jin Kang Park', 'email': 'yujin.kang@mongodb.com', 'username': 'ykangpark'}

Message: SERVER-66609: Avoid global lock upgrade within storage_timestamp_test.cpp
Branch: master
https://github.com/mongodb/mongo/commit/9aafbc13112e03e9aa6889b0fb754adaac8d1e84

Comment by Yujin Kang Park [ 01/Jun/22 ]

As Haley pointed out assertOplogDocumentExistsAtTimestamp causes a lock upgrade in the locations where it is called with an AutoGetCollection with MODE_IS in scope.

Additionally, similar to SERVER-61788 there are instances where a lock upgrade is caused by calling doNonAtomicApply or doAtomicApplyOps. 

I have renamed this ticket to address these.

Comment by Haley Connelly [ 01/Jun/22 ]

yujin.kang@mongodb.com I've found this is still an issue despite preventing a lock upgrade in fetchActiveTransactionHistory. As Yujin pointed out offline, we can confirm the lock is upgraded by adding the following invariant to lock_manager.cpp

invariant(resId != resourceIdGlobal, fmt::format("Global lock upgrade {} -> {}", modeName(request->mode), modeName(newMode)));

Running the storage_timestamp_test with the addition of the invariant, on top of changes made in SERVER-61733, still causes the server to crash.

One issue seems to be that while setting up the MultiDocumentTransactionTest, we call assertOplogDocumentExistsAtTimestamp after acquiring a MODE_IS lock. We've already set the opCtx to be in a multi-document transaction, which means the AutoGetCollectionForRead passed into Helpers::findOne() eventually tries to upgrade the lock to MODE_IX.

Going to re-open since this is slightly different than SERVER-61733.

Comment by Yujin Kang Park [ 31/May/22 ]

Closed as duplicate of SERVER-61733

Comment by Yujin Kang Park [ 31/May/22 ]

This is caused by fetchActiveTransactionHistory taking global lock IS and then using DBClientBase::findOne, which ends up taking global lock IX due to getLockModeForQuery returning IX for reads with snapshot readConcern.

Dump of lock upgrade:

SERVER-66609_backtrace.txt

Print of stack trace each time LockManager::lock is called with resourceIdGlobal:

SERVER-66609_print_stack_on_global_lock.log

Comment by Yujin Kang Park [ 23/May/22 ]

kaloian.manassiev@mongodb.com Intended to point to the place where the constructor is called, not the constructor itself. But that might have been confusing. I changed it to point to the constructor and also added a link to the test making the call. Thanks!

Comment by Kaloian Manassiev [ 23/May/22 ]

yujin.kang@mongodb.com: In the description you have included a link to a test file, not the constructor of MongoDOperationContextSession. Can you fix it?

Generated at Thu Feb 08 06:05:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.