Because transactions yield their locks on secondaries (SERVER-37199), a concurrent hybrid background index build can conflict in a way that leads to lost writes into building indexes (i.e. corruption) on secondaries.
In this example, a background index build on {a: 1} is concurrent with an insert of a document {a: 0} in a transaction while applied on a secondary.
- The background index build converts its X lock to an IX lock while collection scanning. It creates a temporary side-writes table to accept all index key insertions during the build.
- A document {a: 0}is inserted in a transaction and prepared. The key for a: 0 is inserted into the side-writes table as part of the same transaction. When applied on a secondary, it drops its IX locks.
- The background index build takes an X lock, uncontested, and drains the side-writes table. Because the insert into the side-writes table was part of a prepared, but uncommitted transaction, it is invisible to the index builder. The table is then dropped on completion.
- The transaction is finally committed, but its side-write is committed to a now-deleted table. The inserted key is now lost forever and the resulting index is corrupted.
On a primary, our locks prevent this from happening, but because an index build can complete while a prepared transaction is active, we can lose writes into building indexes.
edit: louis.williams