Looking at alexander.gorrod's change in https://github.com/wiredtiger/wiredtiger/commit/31058f483e0c34d9c141b83352481a18ae337eb8 for SERVER-19522, I think there's a problem in the use of the WT_PAGE_SCANNING lock vs. the full WT_PAGE_LOCK in reconciliation.

The code is:

/*

 * The compaction process looks at the page's modification information;

 * if compaction is running, lock the page down.

 *

 * Otherwise, flip on the scanning flag: obsolete updates cannot be

 * freed while reconciliation is in progress.

 */

locked = 0;

if (conn->compact_in_memory_pass) {

        locked = 1;

        WT_PAGE_LOCK(session, page);

} else

        for (;;) {

                F_CAS_ATOMIC(page, WT_PAGE_SCANNING, ret);

                if (ret == 0)

                        break;

                __wt_yield();

        }

That is, the assumption is if you're holding the page lock for any reason, you don't need the WT_PAGE_SCANNING lock to avoid scanning obsolete updates.

I don't think that's correct: __wt_update_serial, the function that sets/clears WT_PAGE_SCANNING, doesn't acquire the page's spin lock when traversing the update list.

__wt_update_serial used to take a page lock, that changed here: https://github.com/wiredtiger/wiredtiger/commit/af42bbbb27907e681469d35375a41e83500169b3#diff-8ac2f2ee2ad6dc20ed78ce706cab5f47