[SERVER-60334] Avoid caching the cursor and session in WiredTigerSizeStorer Created: 30/Sep/21  Updated: 29/Oct/23  Resolved: 05/Nov/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.2.0, 5.0.6, 4.4.16, 4.2.22

Type: Improvement Priority: Major - P3
Reporter: Josef Ahmad Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-67514 SizeStorer load() can get stuck in pa... Closed
Related
related to SERVER-67514 SizeStorer load() can get stuck in pa... Closed
is related to SERVER-61097 SizeStorer can cause deadlocks with c... Closed
is related to SERVER-58311 MongoDB calls WT RTS API while holdin... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.0, v4.4, v4.2
Sprint: Execution Team 2021-11-15
Participants:

 Description   

Note: this ticket fixes the bug described in SERVER-67514.

Original description:
SERVER-58311 discovered that mongod calls into WiredTiger Rollback To Stable (RTS) while holding the sizeStorer.wt dhandle. This is not expected as WiredTiger usually requires RTS to have exclusive access to a dhandle. This is observed when building the server with these non-default flags.

E  STORAGE  22435   [BackgroundSync] "WiredTiger error","attr":{"error":16,"message":"[1625743985:510865][6205:0x7f6a41a84700], file:sizeStorer.wt, txn rollback_to_stable: __rollback_to_stable_btree_apply, 1522: file:sizeStorer.wt: unable to open handle, error indicates handle is unavailable due to concurrent use: Device or resource busy"}

Add a pause/resume API to WiredTigerSizeStorer. Pause the WiredTigerSizeStorer after syncing the sizeInfo and resume it after completing the RTS.



 Comments   
Comment by Githook User [ 08/Jul/22 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Avoid caching the cursor and session in WiredTigerSizeStorer

(cherry picked from commit 6da66799d036cfb42e18a75ca9bd2272574a814d)
(cherry picked from commit 1dee7f9d89fe66befef3b30d9cc12ad432d2067d)
(cherry picked from commit 5229b86161b1588df34fbee304b525068081a860)
Branch: v4.2
https://github.com/mongodb/mongo/commit/2956f0a8a744e31ec3efeeff7e43f9c306d16bb2

Comment by Githook User [ 01/Jul/22 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Avoid caching the cursor and session in WiredTigerSizeStorer

(cherry picked from commit 6da66799d036cfb42e18a75ca9bd2272574a814d)
(cherry picked from commit 1dee7f9d89fe66befef3b30d9cc12ad432d2067d)
Branch: v4.4
https://github.com/mongodb/mongo/commit/5229b86161b1588df34fbee304b525068081a860

Comment by Githook User [ 01/Jul/22 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Remove unused KVEngine::okToRename() function

(cherry picked from commit e5cb44d46d108c3cf3784621264bb04cd3c2f503)
(cherry picked from commit 0422d0708a8ffad3bc9376dd83b88cbd3ceb12ef)
Branch: v4.4
https://github.com/mongodb/mongo/commit/32a4e62789019cd086f33b4ede0ced7ef2217d52

Comment by Sulabh Mahajan [ 01/Jul/22 ]

This change fixes cursor mutex deadlock when sizeStorer is forced into eviction by WiredTiger, hence requesting backport to 4.4 and 4.2

Comment by Githook User [ 15/Dec/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Avoid caching the cursor and session in WiredTigerSizeStorer

(cherry picked from commit 6da66799d036cfb42e18a75ca9bd2272574a814d)
Branch: v5.0
https://github.com/mongodb/mongo/commit/1dee7f9d89fe66befef3b30d9cc12ad432d2067d

Comment by Githook User [ 15/Dec/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Remove unused KVEngine::okToRename() function

(cherry picked from commit e5cb44d46d108c3cf3784621264bb04cd3c2f503)
Branch: v5.0
https://github.com/mongodb/mongo/commit/0422d0708a8ffad3bc9376dd83b88cbd3ceb12ef

Comment by Githook User [ 05/Nov/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Avoid caching the cursor and session in WiredTigerSizeStorer
Branch: master
https://github.com/mongodb/mongo/commit/6da66799d036cfb42e18a75ca9bd2272574a814d

Comment by Githook User [ 05/Nov/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60334 Remove unused KVEngine::okToRename() function
Branch: master
https://github.com/mongodb/mongo/commit/e5cb44d46d108c3cf3784621264bb04cd3c2f503

Comment by Louis Williams [ 29/Oct/21 ]

gregory.wlodarek, this is related to SERVER-61097. I think one solution we should consider, which would solve both problems, is to stop caching this cursor and session, if possible. This does not appear to be a hot enough code path to warrant such an optimization.

Generated at Thu Feb 08 05:49:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.