Partial indexes in MongoDB only include documents that match a specific filter expression. As a result, they are incompatible with SBE $lookup operations because SBE requires the entire collection to be indexed.

Proposed Solution for SBE:

The Classic engine avoids this issue because it re-plans the lookup for each local field value. The inability of SBE to correctly handle this scenario with partial indexes represents a regression for MongoDB users.

To resolve this and allow SBE to use sparse indexes, we should adopt a strategy similar to how SBE handles index collation incompatibility (SERVER-81555):

1. If the local field value satisfies the filter expression: Use the partial index.
2. If the local field value does not satisfy the expression: Perform a collection scan.