I am creating this ticket to investigate an inefficient lock contention between dropping tables and the sweep server.

Dropping a table involves schema lock and the table write lock. Likely handle list lock is also needed to locate the handles associated with the table and underlying files. On dropping both the table and the file, the sweep server is woken up to process the handles.

Each time the sweep server wakes up, it goes through all handles to process each one of them. Sweep server does a lot of things, marking handles time of death, checking if they have expired, closing tables, then btrees, clearing them from memory, etc. Again it needs the table write lock if processing a table, and handle write lock when processing handles.

Imagine MongoDB dropping a collection with several indexes. So, in one shot there will be a bunch of drops issued to WiredTiger. Each drop is at least a drop on the "table:" URI and the "file:" URI. Each one of those holds these above-mentioned locks, and trigger sweep server. Then sweep server goes through all the above activity, while contending for the same locks.

This process doesn't seem very efficient. There is some evidence seen in the customer cases. Especially if the collections being dropped have a lot of data, even if clean, it must be cleared.

I am not exactly sure what the sequence of operations in this interaction is. So for this ticket, we should investigate more. Ideally, create a reproducer that demonstrates this inefficiency. Once we have a reproducer we can take a call on if and how to improve the situation.

See the case and FTDC attached for more information.