[JAVA-147] Automatically routing of queries to non-master nodes (if slave_ok set) Created: 07/Aug/10 Updated: 29/Oct/10 Resolved: 03/Oct/10 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 2.2 |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Scott Hernandez (Inactive) | Assignee: | Eliot Horowitz (Inactive) |
| Resolution: | Done | Votes: | 10 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
Add support to control how operations are handled when slave_ok is used. Extend the connection to include two pools of ports; one for the master node and one for non-master nodes. Create a configuration option to allow using non-master nodes for non-write operations (queries basically). Maybe something like this: class QueryOptions { By setting excluded/includedReplicas you can filter certain replicas in/out of the pools. By default the master will be only server used; this keeps compatibility with the existing driver. |
| Comments |
| Comment by Joseph Wang [ 10/Oct/10 ] |
|
All java files were given to Alvin as part of contract negotiation. I'll ping Alvin to forward you the program. |
| Comment by Eliot Horowitz (Inactive) [ 10/Oct/10 ] |
|
Can you provide a full test that shows that |
| Comment by Joseph Wang [ 10/Oct/10 ] |
|
Tested Java 2.2 driver on a replication set. config = {_id: 'lp', members: [ > rs.status() , , { "_id" : 2, "name" : "ip-10-166-59-166:10001", "health" : 1, "state" : 7, "uptime" : 1265, "lastHeartbeat" : "Sat Oct 09 2010 16:01:34 GMT-0400 (EDT)" } ], All queried fields are indexed. , It seems the query performance degrades significantly (w/ or w/o addOption). |
| Comment by Scott Hernandez (Inactive) [ 09/Oct/10 ] |
|
Yes, and Mongo/DB/DBCollection all have a slaveOk() method to enable this by default for queries down the line. |
| Comment by Joseph Wang [ 08/Oct/10 ] |
|
To test this, we simply do coll.find(query).addOption(Bytes.QUERYOPTION_SLAVEOK)? It will auto route to slaves if it is a query and to master if it is a write/update. |
| Comment by Eliot Horowitz (Inactive) [ 07/Oct/10 ] |
|
Right now only from priority 1. |
| Comment by Jeff Yemin (Inactive) [ 07/Oct/10 ] |
|
Eliot, did you mean that driver will only read from slaves with priority >= 1, and not from slaves with priority < 1, or did you mean that driver will only read from slaves with priority > 0 and not from slaves with priority = 0? |
| Comment by auto [ 03/Oct/10 ] |
|
Author: {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}Message: add a pair test for changes for |
| Comment by Eliot Horowitz (Inactive) [ 03/Oct/10 ] |
|
Testers would be greatly appreciated. |
| Comment by Eliot Horowitz (Inactive) [ 03/Oct/10 ] |
|
This should be working now using master. |
| Comment by auto [ 03/Oct/10 ] |
|
Author: {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}Message: when using replica set and setting SLAVE_OK automatically route reads to slaves |
| Comment by Jeff Yemin (Inactive) [ 03/Oct/10 ] |
|
Perfect. |
| Comment by Eliot Horowitz (Inactive) [ 03/Oct/10 ] |
|
Not in the way the description says. Though this will only read from priority 1 slaves, so if you make your backup slaves priority 0, then you'll be in good shape. |
| Comment by Jeff Yemin (Inactive) [ 03/Oct/10 ] |
|
Are you still planning on adding support for excluded replicas? The use case I'm thinking of is that we have a replica set where one or more nodes are being used solely for backups (and therefore might be shut down every 15 minutes to take an FS snapshot), perhaps in combination with --slavedelay, but the rest of the slaves should be used to take read load off the master. Also, one of the slaves might be in a different data center, so we don't want to read from that one either. |
| Comment by Eliot Horowitz (Inactive) [ 03/Oct/10 ] |
|
I think its ok for a slave ok to go to a salve in a request block. |
| Comment by auto [ 03/Oct/10 ] |
|
Author: {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}Message: plugging in ReplicaSetStatus to DBTCPConnector final prep for |
| Comment by Jeff Yemin (Inactive) [ 03/Oct/10 ] |
|
For our use cases, we would not be mixing write and reads, but we might have multiple reads that we want to go to the same slave. So we definitely want the ability for query in the middle of requestStart/requestDone block to be routed to a slave. What if there was some way to indicate on the requestStart method what the intention is: reads only, writes only, or mixed reads and writes. The first would go to a slave, and the second and third to the master (so reads are consistent with previous writes). |
| Comment by Joseph Wang [ 03/Oct/10 ] |
|
If it is a write/update operation, then it should use the master. In this way, we've write-to-master & read-from-slave concept apply to all situation. |
| Comment by Eliot Horowitz (Inactive) [ 03/Oct/10 ] |
|
Question: If you are in the middle of a requestStart/requestDone block, and you do a query with SLAVE_OK, should it use the master or a slave. Any thoughts? |
| Comment by Joseph Wang [ 28/Aug/10 ] |
|
Same issue was opened in http://jira.mongodb.org/browse/JAVA-149. |