Details
-
Improvement
-
Status: Backlog
-
Major - P3
-
Resolution: Unresolved
-
None
-
None
-
(copied to CRM)
Description
SSL configuration isn't obvious so should be documented:
The Spark Connector uses the Mongo-Java-Driver and the driver will need to be configured to work with SSL. See the ssl tutorial in the java documentation.
The SSL system properties are:
- javax.net.ssl.trustStore
- javax.net.ssl.trustStorePassword
- javax.net.ssl.keyStore
- javax.net.ssl.keyStorePassword
These are JVM wide system properties and will need to be configured on the Spark driver machine as well as each of the Spark workers.
According to the Spark configuration you can set these properties in spark via:
- spark.driver.extraJavaOptions - set properties on the spark driver
- spark.executor.extraJavaOptions - set properties on the executor
An example submitting a spark job with extra Java Options:
./bin/spark-submit --name "My app" --master local[4] --conf "spark.executor.extraJavaOptions=-Djavax.net.ssl.trustStore=path_to_trustStore -Djavax.net.ssl.trustStorePassword=trustPassword -Djavax.net.ssl.keyStore=path_to_keyStore -Djavax.net.ssl.keyStorePassword=keyPassword" sparkApplication.jar
|
A/C
As a reader, I can learn how to set my TLS/SSL configuration properties to authenticate my Spark workers from my Spark configuration file.
As a reader, I can learn how to set my TLS/SSL configuration properties to authenticate my Spark workers when launching a Spark job from the command line.