[SERVER-50710] Lock acquired in FindCmd run() method is not yielded by SBE Created: 02/Sep/20  Updated: 29/Oct/23  Resolved: 03/Mar/21

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Task Priority: Major - P3
Reporter: David Storch Assignee: David Storch
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-51629 Support lock-free reads in SBE Closed
related to SERVER-49385 SBE should fail cleanly if needed ind... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query 2020-09-07, Query Execution 2021-02-22, Query Execution 2021-03-08
Participants:
Linked BF Score: 126

 Description   

The FindCmd currently acquires an AutoGetCollectionForReadCommand early in its code path:

https://github.com/mongodb/mongo/blob/a8bf9e88b3cd2c0036b3ae3e484dbbc933fc4614/src/mongo/db/commands/find_cmd.cpp#L359-L362

This is held while constructing the PlanExecutor via getExecutorFind(), and it continues to be held while actually running the PlanExecutor to produce the first batch of results. Since getExecutorFind() may invoke runtime planning mechanism to choose a plan, query execution may occur both when calling this method and when constructing the first batch.

This lock must be held for the classic query execution engine, as it uses the kLockExternally model. This means that the caller is responsible for acquiring the necessary locks. SBE, on the other hand, uses the kLocksInternally model. It acquires its own locks as needed. The fact that the FindCmd also holds the lock is problematic for yielding, since the SBE yielding code has no means to relinquish this lock. As a result, not all locks held by the query are released during a yield unless that yield is happening during a getMore. This means that when the system is configured to use SBE, an operation requiring a strong lock on the collection could block for a long time behind a find command.

I discovered this while attempting to adapt commands_handle_kill.js to run a test case with SBE enabled. Doing so would be a good way to test that this ticket has been addressed adequately.



 Comments   
Comment by Githook User [ 03/Mar/21 ]

Author:

{'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}

Message: SERVER-50710 Consolidate SBE and classic yielding paths

SBE will now, like the classic engine, use
saveLockStateAndUnlock() in order to yield any locks held
higher on the stack. Additionally, this patch enables the
'concurrency' suite in the SBE build variant in order to
provide additional yielding-related test coverage.
Branch: master
https://github.com/mongodb/mongo/commit/f83d51c0ffe9420f8dfd68d477de228dc78f9527

Generated at Thu Feb 08 05:23:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.