Restrict Extension DPL to only allow issuing LogicalStages for its original stage

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 8.3.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The Extensions API DPL currently allows returning pipelines of a variant type, which may be of type LogicalStage. These LogicalStages correspond to stages which are implemented by the extension.

      Unfortunately, we discovered that some stageConstraints are required post DPL split. This means that the DocumentSourceOptimizable we construct for the merge or shards pipeline are unable to service the constraints calls, since we construct the DocumentSourceOptimizable using only a LogicalStage, while the properties for the constraint are provided by the Ast Node. 

      It's possible we may need to revisit where constraints are stored in the extensions API. We are considering taking a good look at the constraints and separating them between constraints that are required during the Ast cycle, and one required during optimization (i.e LogicalStage).

      For 8.3, we've decided to compromise, and restrict the extensions API to only allow providing LogicalStages in the DPL, if they correspond to the LogicalStage for which the DPL was created. This will be enforced in the convertDPLHandleToDocumentSources function, when parsing a LogicalStage into a DocumentSourceExtensionOptimizable.

      By imposing this restriction in 8.3, we will be able to construct a DocumentSourceExtensionOptimizable using the AstNode's properties which are already available to us. We may consider either adding a clone() method to the AstNode interface, or simply provide a constructor which accepts a LogicalStage with properties. 

      For example, if an extension Foo implements the stages $foo and $bar. 
      LogicalStage($foo)->distributedPlanLogic may only contain a LogicalStage for $foo. If a $bar stage is required in the DPL, it must be obtained via a ParseNode. 

            Assignee:
            Alyssa Clark
            Reporter:
            Santiago Roche
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: