Uploaded image for project: 'Spark Connector'
  1. Spark Connector
  2. SPARK-159

Spark Connector not identifying "primary" member of replicaset

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.2.0
    • Component/s: Configuration
    • Labels:
      None
    • Environment:
      Mongodb 3.2

      While using spark-connector, we are getting the following error, while it is picking any ip other than the "primary". The database backend is MongoDB replicaset with one primary and two secondaries. The ip is being surfaced by "consul" service discovery dns, so it could be any one of the member of the given replica-set.

      "com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=REPLICA_SET, servers=[

      {address=<redacted>:27017, type=REPLICA_SET_SECONDARY, roundTripTime=1.1 ms, state=CONNECTED}

      ]

      Spark-mongo driver version:
      <dependency>
      <groupId>org.mongodb.spark</groupId>
      <artifactId>mongo-spark-connector_2.11</artifactId>
      <version>2.2.0</version>
      </dependency>

      So there is a 33% chance that secondary member gets picked up. It would be great if spark-connector is able to discover the primary member of the replica set and not fail on secondary.

            Assignee:
            ross@mongodb.com Ross Lawley
            Reporter:
            dilsingi dilsingi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: