I have written a reproducer in BF-41629-repro.

Repro line:

env WT_BUILDDIR=$(pwd) python3 ../test/suite/run.py -v 4 -p schema09

Problem
The issue happens when we crash in drop(). If we crash after dropping the file: metadata entry, we still have colgroup: and table: entry existing in the database.

This is usually okay since it wasn't reconciled to disk. The problem comes when WAL log records this removal entry. It was verified through logs that the WAL log has only the remove entry for file:.

Now performing database recovery, we replay the file: log entry, causing the table to be incomplete. When trying to go through the same logic in MongoDB inside createStorage function, we perform create/drop/create scenario to ensure we have a consistent complete table, however this fails.

WT create function will return EEXIST, since it sees the table already exists. WT drop call removes the metadata entries in order of (file:, colgroup:, table:}. When trying to remove the file: it returns WT_NOTFOUND and exits with a ENOENT to MongoDB. This means we have a stalemate where we have incomplete table in the metadata file that can't be removed.

Solution
There are two solutions we can go forward here:

• Have MongoDB call drop with force=true. This forcibly removes all metadata entries for that table. This will fix the stalemate issue.
• As of WT-14586, we added a policy that WT should remove all incomplete tables during database recovery. We should be adhering to this rule for this scenario too. Therefore we should modify the __metadata_clean_incomplete_table function to remove the table if we encounter missing colgroups or file entries.