Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-268

Cannot cast -1.0 into a BsonValue. FloatType has no matching BsonValue.

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.3.3
    • Fix Version/s: 2.4.3, 2.3.5, 2.2.9, 2.1.8, 3.0.1
    • Component/s: Schema
    • Labels:
      None
    • Environment:
      Centos 7

      Description

      I am reading a DF with the following schema: 

       

      StructType(List(StructField(transaction_id,StringType,false),StructField(date,TimestampType,true),StructField(time,TimestampType,true),StructField(customer_id,StringType,true),StructField(to_customer_id,StringType,true),StructField(mobile_number,StringType,true),StructField(to_mobile_number,StringType,true),StructField(card_number,StringType,true),StructField(service,StringType,true),StructField(reference_1,StringType,true),StructField(reference_2,StringType,true),StructField(pos_id,StringType,true),StructField(status,StringType,true),StructField(void,StringType,true),StructField(void_date,StringType,true),StructField(refund_no,StringType,true),StructField(remarks,StringType,true),StructField(amount,FloatType,true),StructField(description,StringType,true),StructField(currency_code,StringType,true),StructField(merchant_name,StringType,true),StructField(merchant_id,LongType,true),StructField(trx_district,StringType,true),StructField(city,StringType,true),StructField(lat,StringType,true),StructField(long,StringType,true),StructField(formatted_address,StringType,true),StructField(district,StringType,true)))

       

      And I am writing it back to a MongoDB. But getting the following error: 

       

      com.mongodb.spark.exceptions.MongoTypeConversionException: Cannot cast -2.0 into a BsonValue. FloatType has no matching BsonValue.

       

      It is referring to the StructField(amount,FloatType,true)

       

      Kindly help,

       

      Thanks, 

       

        Attachments

          Activity

            People

            Assignee:
            ross.lawley Ross Lawley
            Reporter:
            navmob@gmail.com Naveesh Doolhur
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: