Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-231

Failed to retrieve any result when using SlaveOK with all slaves are down

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.3
    • Fix Version/s: 2.5
    • Component/s: Cluster Management
    • Labels:
      None
    • Environment:
    • # Replies:
      10
    • Last comment by Customer:
      true

      Description

      Java driver 2.3.

      We have 3 mongo servers. Each has 48G with 24 CPUs. The servers are running with replica set.

      [joseph.wang@lpsdb1.la2 ~]$ /usr/local/mongodb-linux-x86_64-1.6.3/bin/mongo localhost:4110
      MongoDB shell version: 1.6.3
      connecting to: localhost:4110/test
      > rs.status()
      {
      "set" : "prod",
      "date" : "Sat Dec 11 2010 13:04:28 GMT-0800 (PST)",
      "myState" : 1,
      "members" : [

      { "_id" : 0, "name" : "lpsdb1.la2.estalea.net:4110", "health" : 1, "state" : 1, "self" : true }

      ,

      { "_id" : 1, "name" : "mongo-prod-mem2.lps.la2.estalea.net:4110", "health" : 1, "state" : 2, "uptime" : 11095, "lastHeartbeat" : "Sat Dec 11 2010 13:04:27 GMT-0800 (PST)" }

      ,

      { "_id" : 2, "name" : "mongo-prod-mem3.lps.la2.estalea.net:4110", "health" : 1, "state" : 2, "uptime" : 11178, "lastHeartbeat" : "Sat Dec 11 2010 13:04:26 GMT-0800 (PST)" }

      ],
      "ok" : 1
      }

      The connection pool was connecting to all three servers. We manually brought down two slaves to see if we can get query result (as part of fault tolerance testing).
      There were no update/writes. Just query. When 1 slave was down, we'd no problem getting query result. When 2 slaves were down, we got no query result.
      When a consumer query comes in, we fork out multiple threads, each responsible for fetch data from specific collection.

      MongoConnection.java shows our singleton connection pool code.
      BaseTableQueryEngine.java shows our query to one of our collections.

      As you can see, we set slaveOk at the query level.

      if (db != null) {
      DBCollection coll = db.getCollection(currentCollection);
      DBCursor cur = null;

      cur = coll.find(dbQuery).addOption(
      Bytes.QUERYOPTION_SLAVEOK);

      DBObject dbObject = db.getLastError();
      if (dbObject != null && dbObject.get("err") != null)

      { log.warn("BaseTableQueryEngine: Encounter error for query " + dbQuery.toString()); setSuccess(false); }

      if (enable_debug)

      { log.debug("BaseTableQueryEngine: Run query " + dbQuery.toString()); log.debug("BaseTableQueryEngine: Found " + cur.count() + " in " + (System.currentTimeMillis() - fStart)); }

      while (cur.hasNext()) {
      BasicDBObject dbo = (BasicDBObject) cur.next();
      BaseTableRow row = new BaseTableRow(dbo);
      if (row.isValid())

      { tuples.add(row.getTuple()); }

      long time = (timeout - (System.currentTimeMillis() - fStart));
      if (time < 0)

      { break; }

      }

      if (enable_debug)

      { log.debug("BaseTableQueryEngine: tuples " + tuples.size() + " in time " + (System.currentTimeMillis() - fStart)); }

      }

        Attachments

        1. BaseTableQueryEngine.java
          4 kB
        2. mongo.jar
          224 kB
        3. MongoConnnection.java
          3 kB

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                6 years, 31 weeks, 6 days ago
                Date of 1st Reply: