Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-91408

DocumentSourceLookUp::_fromExpCtx has unclear ownership

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • ALL

      tl;dr: After a long investigation into DocumentSourceLookup::_fromExpCtx, I found some things that I think will guard against future bugs. My recommendations are in a comment, which unfortunately, does not have a tl;dr.

       

      Although _fromExpCtx's _memory is reference counted via intrusive_ptr, all ExpressionContext objects also need an owner that is responsible for detaching and reattaching the ExpressionContext::opCtx pointer when pipeline execution spans multiple batches (each one necessitating a new opCtx).

      Normally, this responsibility belongs to a Pipeline object, which has detachFromOperationContext and reattachToOperationContext member functions for this purpose. One-to-one PipelinetoExpressionContext ownership does not exactly work for the DocumentSourceLookUp's sub-pipeline, however, because depending on execution state, the ExpressionContexts used by the sub-pipelines can be referenced by between 0--2 Pipeline objects.

      1. In the common case, there is a DocumentSourceLookUp::resolvedIntrospectionPipeline that holds a reference to DocumentSourceLookUp::_fromExpCtx, but this pipeline's detachFromOperationContext and reattachToOperationContext methods do not get called, because the introspection pipeline is a reference-only pipeline that does not execute and _usually does not need an up-to-date opCtx. (See SERVER-91362 for exceptions.) Instead, the DocumentSourceLookUp directly performs the attach/ detach operations itself.
      2. However, while the DocumentSourceLookUp is serving the results of a subsumed $unwind operation, it maintains an active Pipeline (stored in DocumentSourceLookUp::_pipeline) that also references _fromExpCtx. During this time, _pipeline becomes the effective owner of the detach/ reattach operations.
      3. When the $lookup foreign collection is a view on a sharded collection, it may be possible for DocumentSourceLookUp::buildPipelineFromViewDefinition() to replace DocumentSourceLookUp::_fromExpCtx with a new ExpressionContext that is different from the one that _resolvedIntrospectionPipeline and _pipeline reference. This new ExpressionContext has no detach/reattach owner and may maintain a dangling reference to an old opCtx. That operation happens here.

       

      Edit: I tried super hard to format all of this nicely, but Jira keeps randomly changing the formatting around .

            Assignee:
            Unassigned Unassigned
            Reporter:
            justin.seyster@mongodb.com Justin Seyster
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: