Hierarchical locking is a well-known technique for avoiding deadlocks. 

We have a hierarchy for WiredTiger's major locks. For example, the checkpoint lock must be acquired before the schema lock.

This ticket proposes changing our lock implementation to enforce this hierarchy in diagnostic mode. I.e., each session would keep track of the "level" of the previous lock it acquired and assert that new lock acquisitions are at a lower "level". It would also update the session lock level for each lock acquisition or release. (N.B., this is only a sketch of an implementation.)

Currently some WT lock macros include assertions to make sure other locks aren't held. But this isn't a general solution. 

Locking issues can be hard to detect, especially when a locking violation seldom leads to actual failures. Having this functionality would make it easier to detect and fix such issues in the future.

Items for consideration if we decide to pursue this (I don't know the answers):

• WT does not require that a session hold all locks in the hierarchy. So any solution would need to make sure locks are acquired and released in order, but not require that all locks in that ordering be held.
• We'll probably want to support both mutexes and reader-writer locks. I believe that a RW lock should have a single position in the hierarchy regardless of whether it is acquired for reading or writing.
• I think we allow some locks to be acquired when a session already holds the lock. So we'll need to support this. We'll also need to figure out whether this is universally acceptable or something that varies depending on the lock or circumstance.
• We have a lot of smaller locks that are used for very local purposes. I'm guessing it would be more trouble than it's worth to assign them all levels in the locking hierarchy. So we may want to support locks that aren't part of the defined hierarchy. I.e., if a session acquires a lock that is in the hierarchy, we enforce the hierarchy. Other locks we just acquire and release without checking the hierarchy.