[SERVER-24710] Optimize $sample+$project - MongoDB

XML

Word

Printable

When performing a pipeline doing a sample then project I noticed it was much slower than doing a project then sample. Could this be optimized? In a similar fashion as to: https://docs.mongodb.com/manual/core/aggregation-pipeline-optimization

For example with the MovieLens dataset ~1million documents:

Pipeline: $sample + $project _id: 76120 ms
Pipeline: $project _id + $sample: 1124 ms

duplicates

SERVER-21887 $sample takes disproportionately long time on newly created collection

SERVER-23661 $sample takes disproportionately long time on newly created collection

is related to

SPARK-64 Sampling then projecting in the MongoSamplePartitioner is slow