[SERVER-68380] Investigate MongoDB code that can use bounded WT cursors instead of search_near Created: 28/Jul/22  Updated: 01/Feb/24  Resolved: 16/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.3.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Louis Williams Assignee: Louis Williams
Resolution: Fixed Votes: 0
Labels: storex-perf
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on WT-11911 Fix use-after-free with bounded curso... Closed
depends on WT-11957 Calling next() is not idempotent with... Closed
Problem/Incident
Related
related to SERVER-78365 Investigate removing ignore_prepare=f... Open
is related to SERVER-84748 Index cursors no longer need callers ... Open
is related to SERVER-65527 Use range bounded cursors in place of... Closed
is related to SERVER-69093 Use range bounded cursors for restori... Closed
Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2023-12-25, Execution Team 2024-01-08, Execution Team 2024-01-22
Participants:
Linked BF Score: 20

 Description   

The new WT_CURSOR bound feature is more efficient than a search_near if we set a lower bound and call next() or set an upper bound and call prev().

The important cases we know about are already tracked in other tickets, but there is at least one case in seekNear that could take advantage of a bounded cursor.



 Comments   
Comment by Githook User [ 16/Jan/24 ]

Author:

{'name': 'Louis Williams', 'email': 'louiswilliams@users.noreply.github.com', 'username': 'louiswilliams'}

Message: SERVER-68380 Always use bounded cursors (#16754)

SERVER-68380 Always use WT bounded cursors

GitOrigin-RevId: 96960e245961a02f77cfa67717e6a689351bebb5
Branch: master
https://github.com/mongodb/mongo/commit/1fa8459ee823b6f829564e16dce608ee78485b07

Comment by Jie Chen [ 09/Nov/23 ]

Hello from the storage engines team! I was part of the project that help created the bound cursors feature. I would like to help in this investigation as much as i can. Some things I would like to point out for the bound cursor feature. When a cursor is unpositioned and we call at next() with a lower bound set, we perform a small modification of cursor->search_near() function to position at the right spot.
In cursor->search_near() in the case that we are not able to find a record on the search spot, we would bias and find a record in the forward direction using cursor->next() and if nothing is found, we would use cursor->prev() to find a record in the reverse direction. The small modification that cursor bounds does in this scenario is that we would stop early without the need to perform cursor->next() or cursor->prev() if the upper bound or lower bound is set.

Another improvement that this could do internally is that we would be able to stop the next() call early due to deleted or invisible records. We would also read the deleted or invisible records to exit as early as possible, which creates a huge improvement in mass delete scenarios.

Comment by Louis Williams [ 09/Nov/23 ]

I found at least 2 bugs in bounded cursors, WT-11957 and WT-11911. The perf results are promising, but I'm going to pause this work until those are addressed.

Comment by Louis Williams [ 01/Nov/23 ]

This would actually be valuable to avoid needing to use ignore_prepare=false on secondaries. This would workaround the bugs described in SERVER-40167 and SERVER-40176.

Generated at Thu Feb 08 06:10:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.