Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-144

Cannot lookup with boolean condition

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 2.2.0
    • Component/s: API
    • Labels:
      None
    • Environment:
      Pyspark

      Hello,

      I am using Pyspark and cannot use a pipeline like that:

      pipeline = [{‘$match’: {'enabled': True}}]
      dataframe = spark.read.format(“com.mongodb.spark.sql”).options(uri=X, collection=Y, pipeline=pipeline).load()

      'enabled' is a boolean in my collection.
      The code returns me:
      org.bson.json.JsonParseException: JSON reader was expecting a value but found 'True'.

      If I put it inside quotes like [{‘$match’: {'field1': 'True'}}] or [{‘$match’: {'field1': "True"}}] it doesn't crash but returns an empty dataframe.

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            malafp Pierre Malafosse
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: