For uncapped collections, CollectionBulkLoaderImpl::insertDocuments inserts documents by calling CollectionImpl::insertDocumentForBulkLoader in a writeConflictRetry block. Regardless of this new batching design approach, it can lead to dangling index record entries. Consider, insertDocumentForBulkLoader() throws WriteConflictException. This leads to retrying of below operations.
- Record insertion
- Inserting <Index key, RecordId> pair into external sorter.
Since inserting <Index key, RecordId> pair into external sorter is not part of storage transaction, previously failed storage transaction attempt would leave dangling index entries pointing to invalid RecordIds.
The solution to fix this bug is that we should not wrap the _addDocumentToIndexBlocks() method in a writeConflictRetry block.And, _addDocumentToIndexBlocks (insertion into external sorter) should be called only after the batch of records got successfully committed in the storage