Remove UDF undefined support

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Done
    • Priority: Major - P3
    • 2.0.0
    • Affects Version/s: None
    • Component/s: Schema
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Changes to Spark and Datasets with user defined functions mean that supporting the undefined UDF helper will no longer work as the generated pipeline $ne:null will also exclude Undefined values. Also, any fields with nullable=false declared in the schema need to be explicitly queried out in the aggregation pipeline.

      Refs: SPARK-76

              Assignee:
              Ross Lawley
              Reporter:
              Ross Lawley
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: