-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Integration
-
Fully Compatible
-
ALL
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Summary
jstests/noPassthrough/extensions/extension_accesses_host_metrics.js fails deterministically when featureFlagSearchOptimizedIdLookup is enabled (i.e. on all-feature-flags variants). This is the extension_accesses_host_metrics.js half of BF-44777, where the root cause is the batched $_internalSearchIdLookup work (SERVER-130062, SERVER-130899).
Root cause
The test runs [{$readNDocuments: {numDocs: 4{}}}], which desugars to [$produceIds, $_internalSearchIdLookup]. $produceIds (upstream source) reads the host idLookup counters via getHostMetrics() at its EOF getNext and expects them fully accumulated (see the comment at src/mongo/db/extension/test_examples/read_n_documents.cpp:43: "At EOF, all idLookup processing for previously produced IDs is complete").
That assumption only holds for batch-of-one pull ordering. With featureFlagSearchOptimizedIdLookup on, $internalSearchIdLookup inherits BatchedEnrichmentStage and batches (size internalSearchIdLookupMaxBatchSize, default 100). BatchedEnrichmentStage::doGetNext() runs fillBatch() — which drains the entire upstream, including the $produceIds EOF getNext where the metrics are read — _before enrichBatch() increments the counters (src/mongo/db/exec/agg/batched_enrichment_stage.cpp:48-58). So $produceIds observes 0/0/0 instead of 4/2/0.5.
Repro (local, all confirmed)
| Config | Result |
|---|---|
| optimized ON (batch 100), execution-control-prioritization suite | FAIL 0/0/0 |
| optimized ON (batch 100), vanilla no_passthrough | FAIL 0/0/0 |
| only featureFlagExtensionsAPI (optimized OFF, batch 1) | PASS |
| all flags ON but internalSearchIdLookupMaxBatchSize: 1 | PASS |
This is a test-only issue; the original tickets just didn't update the test behavior with the feature enabled.
Two options that I see, but I'll leave the solution to the ticket owner:
- Test fix: pin internalSearchIdLookupMaxBatchSize: 1 in the test (validated to pass), keeping it a valid getHostMetrics surfacing check without depending on pull ordering
- Feature semantics: decide whether batched idLookup should surface idLookup metrics incrementally so mid-pipeline getHostMetrics() reads stay accurate
- related to
-
SERVER-130062 Wire Express idLookup live behind featureFlagSearchOptimizedIdLookup
-
- In Code Review
-
-
SERVER-130899 Make $_internalSearchIdLookup inherit from BatchedEnrichmentStage
-
- Closed
-