Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-51889

Introduce new stage to peek / debug a pipeline stage's output

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Minor - P4
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Aggregation Framework
    • Labels:
      None

      Description

      Compass has a feature to build an aggregation pipeline. In that interface/tab, as you build each stage of the pipeline - you can preview the transformed document samples in the adjacent part of the screen.

      A similar feature with the aggregation pipeline in the mongo shell will be useful. Suppose there is a stage called $peek, and it can be placed at any stage of the pipeline and outputs sample transformed documents (one, more than one - perhaps configurable) along with the query output.

      This will be helpful in debugging and while working with longer pipelines and for the folks who prefer using the command-line shell interface to the Compass GUI.

      There is a similar feature in Java programming language - in Java Streams API. As the data flows thru a stream's pipeline, the peek method has such a function.

      For example (as shown in the https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/stream/Stream.html#peek(java.util.function.Consumer) ):

      Stream.of("one", "two", "three", "four")
       .filter(e -> e.length() > 3)
       .peek(e -> System.out.println("Filtered value: " + e))
       .map(String::toUpperCase)
       .peek(e -> System.out.println("Mapped value: " + e))
       .collect(Collectors.toList());

      In the above Java code, the peek method outputs the elements after the filter and map operations. You get to see what is happening as the data flows thru the pipeline.

      Similarly, with the aggregation pipeline:

      db.collection.aggregate([
       { $addFields: { ... } },
       { $peek: n }, // n specifies the number of sample documents
       { $group: { ... } },
       { $peek: n },
       { $project: { ... } }
      ])

       

       

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-query-execution Backlog - Query Execution
              Reporter:
              prasadsaya@yahoo.com Prasad Saya
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: