Java Driver
  1. Java Driver
  2. JAVA-231

Failed to retrieve any result when using SlaveOK with all slaves are down

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major - P3 Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.3
    • Fix Version/s: 2.5
    • Component/s: Cluster Management
    • Labels:
      None
    • Environment:
    • Operating System:
      ALL
    • # Replies:
      10
    • Last comment by Customer:
      true

      Description

      Java driver 2.3.

      We have 3 mongo servers. Each has 48G with 24 CPUs. The servers are running with replica set.

      [joseph.wang@lpsdb1.la2 ~]$ /usr/local/mongodb-linux-x86_64-1.6.3/bin/mongo localhost:4110
      MongoDB shell version: 1.6.3
      connecting to: localhost:4110/test
      > rs.status()
      {
      "set" : "prod",
      "date" : "Sat Dec 11 2010 13:04:28 GMT-0800 (PST)",
      "myState" : 1,
      "members" : [

      { "_id" : 0, "name" : "lpsdb1.la2.estalea.net:4110", "health" : 1, "state" : 1, "self" : true }

      ,

      { "_id" : 1, "name" : "mongo-prod-mem2.lps.la2.estalea.net:4110", "health" : 1, "state" : 2, "uptime" : 11095, "lastHeartbeat" : "Sat Dec 11 2010 13:04:27 GMT-0800 (PST)" }

      ,

      { "_id" : 2, "name" : "mongo-prod-mem3.lps.la2.estalea.net:4110", "health" : 1, "state" : 2, "uptime" : 11178, "lastHeartbeat" : "Sat Dec 11 2010 13:04:26 GMT-0800 (PST)" }

      ],
      "ok" : 1
      }

      The connection pool was connecting to all three servers. We manually brought down two slaves to see if we can get query result (as part of fault tolerance testing).
      There were no update/writes. Just query. When 1 slave was down, we'd no problem getting query result. When 2 slaves were down, we got no query result.
      When a consumer query comes in, we fork out multiple threads, each responsible for fetch data from specific collection.

      MongoConnection.java shows our singleton connection pool code.
      BaseTableQueryEngine.java shows our query to one of our collections.

      As you can see, we set slaveOk at the query level.

      if (db != null) {
      DBCollection coll = db.getCollection(currentCollection);
      DBCursor cur = null;

      cur = coll.find(dbQuery).addOption(
      Bytes.QUERYOPTION_SLAVEOK);

      DBObject dbObject = db.getLastError();
      if (dbObject != null && dbObject.get("err") != null)

      { log.warn("BaseTableQueryEngine: Encounter error for query " + dbQuery.toString()); setSuccess(false); }

      if (enable_debug)

      { log.debug("BaseTableQueryEngine: Run query " + dbQuery.toString()); log.debug("BaseTableQueryEngine: Found " + cur.count() + " in " + (System.currentTimeMillis() - fStart)); }

      while (cur.hasNext()) {
      BasicDBObject dbo = (BasicDBObject) cur.next();
      BaseTableRow row = new BaseTableRow(dbo);
      if (row.isValid())

      { tuples.add(row.getTuple()); }

      long time = (timeout - (System.currentTimeMillis() - fStart));
      if (time < 0)

      { break; }

      }

      if (enable_debug)

      { log.debug("BaseTableQueryEngine: tuples " + tuples.size() + " in time " + (System.currentTimeMillis() - fStart)); }

      }

        Activity

        Hide
        Joseph Wang
        added a comment -

        if there is a way to determine that all slaves are down, i don't mind reissuing the query w/o slave_ok s.t. it will get to primary/master.

        Show
        Joseph Wang
        added a comment - if there is a way to determine that all slaves are down, i don't mind reissuing the query w/o slave_ok s.t. it will get to primary/master.
        Hide
        Scott Hernandez
        added a comment -

        SlaveOk means that queries can be sent to the slave, not they must be, IMO.

        Show
        Scott Hernandez
        added a comment - SlaveOk means that queries can be sent to the slave, not they must be, IMO.
        Hide
        Antoine Girbal
        added a comment -

        this is most likely related to bug
        http://jira.mongodb.org/browse/JAVA-225

        basically the Java driver was ignoring the slaveOk option when looking for a master.
        Joseph, could you try with latest driver from trunk (I will also attach a jar you can use).
        Note that builds from trunk should only be used for testing.

        Show
        Antoine Girbal
        added a comment - this is most likely related to bug http://jira.mongodb.org/browse/JAVA-225 basically the Java driver was ignoring the slaveOk option when looking for a master. Joseph, could you try with latest driver from trunk (I will also attach a jar you can use). Note that builds from trunk should only be used for testing.
        Hide
        Antoine Girbal
        added a comment -

        jar from trunk

        Show
        Antoine Girbal
        added a comment - jar from trunk
        Hide
        Antoine Girbal
        added a comment -

        I tested this case and was able to read fine from last slave, with 2 servers down from replica set.
        I am assuming that this issue stemmed from other bugs fixed earlier.

        Show
        Antoine Girbal
        added a comment - I tested this case and was able to read fine from last slave, with 2 servers down from replica set. I am assuming that this issue stemmed from other bugs fixed earlier.

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              3 years, 9 weeks, 2 days ago
              Date of 1st Reply: