Sparse indexes only include entries for documents that contain the indexed field, even if the value of that field is null. Documents lacking the indexed field are skipped by the index.

SBE engine cannot utilise sparse indexes during a $lookup operation. This is because $lookup is expected to match documents regardless of whether the field is missing or contains a null value, a behaviour inconsistent with sparse indexes. Classic engine avoids this issue by re-planning the lookup for each value of the localfield. Consequently, customers switching to SBE may experience a performance regression with $lookup queries that previously used sparse indexes under the Classic engine. To address this, we should prevent pushing down a $lookup if the Classic engine would have been able to leverage a sparse index.