extension_accesses_host_metrics.js needs to be updated for batched $_internalSearchIdLookup behavior

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Integration
    • Fully Compatible
    • ALL
    • 200
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      jstests/noPassthrough/extensions/extension_accesses_host_metrics.js fails deterministically when featureFlagSearchOptimizedIdLookup is enabled (i.e. on all-feature-flags variants). This is the extension_accesses_host_metrics.js half of BF-44777, where the root cause is the batched $_internalSearchIdLookup work (SERVER-130062, SERVER-130899).

      Root cause

      The test runs [{$readNDocuments: {numDocs: 4{}}}], which desugars to [$produceIds, $_internalSearchIdLookup]. $produceIds (upstream source) reads the host idLookup counters via getHostMetrics() at its EOF getNext and expects them fully accumulated (see the comment at src/mongo/db/extension/test_examples/read_n_documents.cpp:43: "At EOF, all idLookup processing for previously produced IDs is complete").

      That assumption only holds for batch-of-one pull ordering. With featureFlagSearchOptimizedIdLookup on, $internalSearchIdLookup inherits BatchedEnrichmentStage and batches (size internalSearchIdLookupMaxBatchSize, default 100). BatchedEnrichmentStage::doGetNext() runs fillBatch() — which drains the entire upstream, including the $produceIds EOF getNext where the metrics are read — _before enrichBatch() increments the counters (src/mongo/db/exec/agg/batched_enrichment_stage.cpp:48-58). So $produceIds observes 0/0/0 instead of 4/2/0.5.

      Repro (local, all confirmed)

      Config Result
      optimized ON (batch 100), execution-control-prioritization suite FAIL 0/0/0
      optimized ON (batch 100), vanilla no_passthrough FAIL 0/0/0
      only featureFlagExtensionsAPI (optimized OFF, batch 1) PASS
      all flags ON but internalSearchIdLookupMaxBatchSize: 1 PASS

       

      This is a test-only issue; the original tickets just didn't update the test behavior with the feature enabled.

      Two options that I see, but I'll leave the solution to the ticket owner:

      • Test fix: pin internalSearchIdLookupMaxBatchSize: 1 in the test (validated to pass), keeping it a valid getHostMetrics surfacing check without depending on pull ordering
      • Feature semantics: decide whether batched idLookup should surface idLookup metrics incrementally so mid-pipeline getHostMetrics() reads stay accurate

            Assignee:
            Chunbin Lin
            Reporter:
            Will Buerger
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: