-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Spark Connector
-
Labels:
-
Environment:python
On the spark connector python guide pages, it describes how to create spark session the documentation reads:
from pyspark.sql import SparkSession my_spark = SparkSession \ .builder \ .appName("myApp") \ .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.coll") \ .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.coll") \ .getOrCreate()
the snippet misses one more config param, which is the mongo spark connector package
the code should look like this
from pyspark.sql import SparkSession my_spark = SparkSession \ .builder \ .appName("myApp") \ .config("spark.jars.packages", "org.mongodb.spark:mongo-spark-connector_2.11:2.4.0") \ .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/test.coll") \ .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/test.coll") \ .getOrCreate()