Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-12325

[Spark] Add spark.jars.packages config to all Python config examples

    XMLWordPrintable

Details

    • Task
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • None
    • Spark Connector
    • python

    Description

      On the spark connector python guide pages, it describes how to create spark session the documentation reads:

      from pyspark.sql import SparkSession
       
      my_spark = SparkSession \
       
      .builder \
       
      .appName("myApp") \
       
      .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.coll") \
       
      .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.coll") \
       
      .getOrCreate()
      
      

      the snippet misses one more config param, which is the mongo spark connector  package

      the code should look like this

      from pyspark.sql import SparkSession
       
      my_spark = SparkSession \
       
      .builder \
       
      .appName("myApp") \
       
      .config("spark.jars.packages", "org.mongodb.spark:mongo-spark-connector_2.11:2.4.0") \
       
      .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.coll") \
       
      .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.coll") \ 
       
      .getOrCreate()
      
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            mhameed Mohammed Hameed
            Nathan Leniz Nathan Leniz
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:
              46 weeks, 5 days ago