Implement explicit desugaring step for Pipelines with Desugar Stages

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      As outlined in the technical design, we will need to introduce an explicit desugaring step that will be called from the aggregation execution path immediately after query stats registration.

      We will need to add a method getDesugaredPipeline method on DocumentSourceExtensionDesugar, which returns a std::list<intrusive_ptr<DocumentSource>>:

       std::list<intrusive_ptr<DocumentSource>> DocumentSourceExtensionDesugar::getDesugaredPipeline() {
      // returns a copy of the lazy initialized (i.e Deferred) desugared Pipeline
      }

      Implement a class Desugarer (or some other appropriate name):

      // Desugarer accepts a Pipeline* in its constructor, which is the Pipeline that needs to be desugared. 
      class Desugarer {
      public:
      
      Desugarer(Pipeline* pipeline) : m_pipeline (pipeline), m_sources(pipeline->getSources) {}
      
      void operator() {
         try {
            auto itr = m_sources.begin();
            while (itr != m_sources.end()) {
                invariant(itr.get());
                itr = _desugar(itr, *itr.get());
            }
         } catch (...) {
           
         }
      }
      
      DocumentSourceContainer::iterator _desugar(DocumentSourceContainer::iterator, DocumentSource& stage) {
         invariant(itr->get() == &stage);
         // Non-Desugar stages just advance to the next stage in the pipeline.
         return std::next(itr);
      }
      
      
      DocumentSourceContainer::iterator _desugar(DocumentSourceContainer::iterator, DocumentSourceExtensionDesugar& stage) {
          invariant(itr->get() == &stage);
         
          auto desugaredPipeline = stage.getDesugaredPipeline();
          // replace desugar stage with its desugared pipeline
          if (!desugaredPipeline.empty()) {
             auto desugarIterator = itr;
             m_sources.insert(itr, desugaredPipeline.begin(), desugaredPipeline.end());
             itr = std::next(itr);
             m_sources.erase(desugarIterator);
          }
      
          return itr; 
      }
      
      private:
      Pipeline* m_pipeline;
      DocumentSourceContainer& m_sources;
      }
      

       

            Assignee:
            Unassigned
            Reporter:
            Santiago Roche
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: