[SERVER-4464] drivers and mongos should support 5 read preferences Created: 09/Dec/11  Updated: 29/Aug/14  Resolved: 19/Oct/12

Status: Closed
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: None
Fix Version/s: 2.3.1

Type: New Feature Priority: Major - P3
Reporter: Daniel Crosta Assignee: Randolph Tan
Resolution: Done Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-3550 Update C++ driver to handle replica s... Closed
is duplicated by SERVER-3358 sharding - tagging - use for reading ... Closed
Related
is related to SERVER-7423 Support commands for read preference Closed
is related to SERVER-6707 Add better messages for read preferen... Closed
is related to SERVER-6983 The shell should support ReadPreferen... Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

There are five read preferences that drivers and mongos should support when routing queries and getmores:

1. PRIMARY – all reads are directed to the replica set (or shard) primary; if there is no primary, reads are rejected
2. PRIMARY_FIRST – reads are preferentially directed to the replica set primary, but will fall back to a secondary if a primary is not available or reachable
3. SECONDARY_ONLY – all reads are directed to the replica set secondaries; if there are no secondaries, reads are rejected
4. SECONDARY – reads are preferentially directed to the replica set secondaries, but will fall back to the primary if no secondaries are available or reachable
5. ALL – reads are distributed to all replica set nodes, and a client does not know a-priori if the read will go to a primary or secondary

Drivers, and mongos, currently support option #1 (when slave_okay is false), and most drivers support option #3 (when slave_okay is true); old versions of some drivers do not direct reads to secondaries even when slave_okay is true, and so implement #2.

Both drivers and mongos should allow for setting a default read preference and overriding the read preference on a per-request basis, as is currently the case with slave_okay.

For backwards compatibility, the slave_okay bit should be set for modes 3, 4, and 5.


EDIT: Renamed "PRIMARY_ONLY" => "PRIMARY"; "PRIMARY" => "PRIMARY_FIRST"; at least Java driver and Pymongo use ReadPreference.PRIMARY to mean what I had initially called "PRIMARY_ONLY". I think the particular names are less important than the behaviors they encapsulate.



 Comments   
Comment by Azat Khuzhin [ 05/Dec/12 ]

And this is normal.

I have php-fpm. With max workers = 60.
So I must have max 60 connections, also some workers can be terminated and new one started, and mongo connection must be reinitialized.
And this means that some for some of connections mongos must use one replicaset member, and for others another member.
But in fact only one - PRIMARY - is used.

Comment by Randolph Tan [ 04/Dec/12 ]

Hi,

I just discussed this with our PHP driver engineer. The PHP driver does have a connection pool, but the ratio between host/port/auth to the connection used is always 1:1. This means that you will always end up using the same mongos connection if you perform queries to the mongos on the same PHP process.

Comment by Azat Khuzhin [ 04/Dec/12 ]

I use PHP driver for connection to mongos.
Which use pool of connections as far as I know.

Comment by Randolph Tan [ 04/Dec/12 ]

nearest is the all option. Assuming that the ping time of all the nodes are within --localThreshold and no tags, nearest would pick any members that are reachable. However, be aware that a connection to mongos is not stateless. It will use the same connection to the shards as much as possible. So if you are using only one connection to mongos to test this, you will always get the same node.

Comment by Azat Khuzhin [ 04/Dec/12 ]

> A primary can also be chosen as long as the ping time is not too high. You can also check out more of the details in our docs here:

It works as "nearest". Not as "ALL" (as described in issue head).
Therefore it is not suitable to me, I need to balancing load between replica set nodes.
But now, with "nearest" only PRIMARY handle "query"

Do you planning for "ALL" read preference?

Comment by Randolph Tan [ 19/Oct/12 ]

Resolving ticket -> stuff that still needs to be done are split into smaller tickets linked to this ticket.

Comment by Richard Kreuter (Inactive) [ 09/Sep/12 ]

Shell support for ReadPreferences doesn't yet match drivers'.

Comment by Randolph Tan [ 07/Aug/12 ]

Hi,

A primary can also be chosen as long as the ping time is not too high. You can also check out more of the details in our docs here:

http://docs.mongodb.org/manual/applications/replication/#nearest

We are also working on the PHP driver implementation which would hopefully make it when we release 2.2 version of the server.

Comment by Azat Khuzhin [ 31/Jul/12 ]

Also I have a question about "nearest"

Doesn't this new feature work, so that drivers will use only "primary" if it will be "nearest" ?
For my case, I would prefer to all SECONDARIES and PRIMARY handle READ requests.

Thanks.

Comment by Azat Khuzhin [ 31/Jul/12 ]

Randolph, thanks.

Unfortunately i don't use mongo shell
For the main purpose i use PHP driver

When this feature will be available in it?

Comment by Randolph Tan [ 30/Jul/12 ]

Hi Azat,

The original slaveOk corresponds to the secondaryPreferred read preference mode. So even before this was implemented, slaveOk actually meant use secondaries unless they cannot be reached. If you want the primary to also server the requests, you can use the nearest mode. This is a new feature for the upcoming 2.2 release and we are still working on the documentation for it. If you are using the mongo shell, you can do something like this:

db.user.find().readPref("nearest")

Comment by Azat Khuzhin [ 29/Jul/12 ]

Is because of this issue, clients (specifically mongos) will be route READ requests only to SECONDARY?
Or was it always so?

I use 5665ee8b85d600c0720cb1e12774f5a660afa456 commit.
And here my "mongostat --discover":

                                        insert  query update delete getmore command flushes mapped  vsize    res faults             locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn                    set repl       time 
                       localhost            26   1348     26      0       0       1                 20.2g  19.6g      0                                                      162k   485k    65                         RTR   20:04:44 
node1.mongo.backend:27018            26      0     27      0      62       2       0   250g   502g  5.74g      4            local:0.3%          0       0|0     0|0    12k    15k    79 backend-ad-data  PRI   20:04:44 
node4.mongo.backend:27018           *26     27    *26     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.6%          0       0|0     0|0     3k    25k    73 backend-ad-data  SEC   20:04:44 
node5.mongo.backend:27018            *0   1322     *0     *0       0     2|0       0   138g   277g  19.2g      0             stat:0.1%          0       0|0     0|0   151k   465k    20 backend-reports-stat  SEC   20:04:44 
node6.mongo.backend:27018             0      0      1      0       0       2       0   138g   279g  24.8g      0             stat:0.3%          0       0|0     1|0   239b     2k    79 backend-reports-stat  PRI   20:04:44 
 
                       localhost            21   1300     21      0       0       1                 20.2g  19.6g      0                                                      155k   419k    65                         RTR   20:04:45 
node1.mongo.backend:27018            21      0     22      0      46       2       0   250g   502g  5.74g      1            local:0.0%          0       0|0     0|0     9k    12k    79 backend-ad-data  PRI   20:04:45 
node4.mongo.backend:27018           *21     21    *21     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.4%          0       0|0     0|0     2k    20k    73 backend-ad-data  SEC   20:04:45 
node5.mongo.backend:27018            *0   1279     *0     *0       0     2|0       0   138g   277g  19.2g      0             stat:0.0%          0       0|0     1|0   146k   403k    20 backend-reports-stat  SEC   20:04:45 
node6.mongo.backend:27018             0      0      0      0       1       2       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   333b     2k    79 backend-reports-stat  PRI   20:04:45 
 
                       localhost            13   1322     13      0       0       1                 20.2g  19.6g      0                                                      155k   415k    65                         RTR   20:04:46 
node1.mongo.backend:27018            13      0     14      0      30       2       0   250g   502g  5.74g      0            local:0.3%          0       0|0     0|0     6k     8k    79 backend-ad-data  PRI   20:04:46 
node4.mongo.backend:27018           *13     13    *13     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.4%          0       0|0     0|0     1k    13k    73 backend-ad-data  SEC   20:04:46 
node5.mongo.backend:27018            *0   1309     *0     *0       0     2|0       0   138g   277g  19.2g      0             stat:0.0%          0       0|0     0|0   149k   406k    20 backend-reports-stat  SEC   20:04:46 
node6.mongo.backend:27018             0      0      0      0       0       2       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   239b     2k    79 backend-reports-stat  PRI   20:04:46 
 
                       localhost            23   1280     23      0       1       1                 20.2g  19.6g      0                                                      153k     4m    65                         RTR   20:04:47 
node1.mongo.backend:27018            23      0     24      0      52       3       0   250g   502g  5.74g      3             stat:5.1%          0       0|0     0|0    10k    13k    79 backend-ad-data  PRI   20:04:47 
node4.mongo.backend:27018           *23     23    *23     *0       1     2|0       0   250g   501g  34.5g      0             stat:0.3%          0       0|0     0|0     2k     4m    73 backend-ad-data  SEC   20:04:47 
node5.mongo.backend:27018            *0   1257     *0     *0       0     2|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     0|0   143k   419k    20 backend-reports-stat  SEC   20:04:47 
node6.mongo.backend:27018             0      0      0      0       0       2       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   239b     2k    79 backend-reports-stat  PRI   20:04:47 
 
                       localhost            17   1348     17      0       0       1                 20.2g  19.6g      0                                                      159k   420k    65                         RTR   20:04:48 
node1.mongo.backend:27018            17      0     18      0      39       3       0   250g   502g  5.74g      0            local:0.1%          0       0|0     0|0     7k    10k    79 backend-ad-data  PRI   20:04:48 
node4.mongo.backend:27018           *17     17    *17     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.4%          0       0|0     0|0     2k    16k    73 backend-ad-data  SEC   20:04:48 
node5.mongo.backend:27018            *0   1331     *0     *0       0     2|0       1   138g   277g  19.3g      0             stat:0.0%          0       0|0     0|0   151k   407k    20 backend-reports-stat  SEC   20:04:48 
node6.mongo.backend:27018             0      0      0      0       1       3       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   359b     2k    79 backend-reports-stat  PRI   20:04:48 
 
                                        insert  query update delete getmore command flushes mapped  vsize    res faults             locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn                    set repl       time 
                       localhost            27   1354     27      0       0       1                 20.2g  19.6g      0                                                      163k   473k    65                         RTR   20:04:49 
node1.mongo.backend:27018            27      0     28      0      59       2       0   250g   502g  5.74g      4             stat:3.7%          0       0|0     0|0    12k    15k    79 backend-ad-data  PRI   20:04:49 
node4.mongo.backend:27018           *28     27    *26     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.6%          0       0|0     0|0     3k    23k    73 backend-ad-data  SEC   20:04:49 
node5.mongo.backend:27018            *0   1326     *0     *0       0     2|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     0|0   151k   453k    20 backend-reports-stat  SEC   20:04:49 
node6.mongo.backend:27018             0      0      0      0       0       3       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   297b     2k    79 backend-reports-stat  PRI   20:04:49 
 
                       localhost            23   1337     23      0       0       1                 20.2g  19.6g      0                                                      160k   421k    65                         RTR   20:04:50 
node1.mongo.backend:27018            23      0     24      0      49       8       0   250g   502g  5.74g      1            local:0.2%          0       0|0     0|0    10k    16k    79 backend-ad-data  PRI   20:04:50 
node4.mongo.backend:27018           *23     23    *23     *0       0     4|0       0   250g   501g  34.5g      0             stat:0.5%          0       0|0     0|0     2k    24k    73 backend-ad-data  SEC   20:04:50 
node5.mongo.backend:27018            *0   1315     *0     *0       0     4|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     1|0   150k   404k    20 backend-reports-stat  SEC   20:04:50 
node6.mongo.backend:27018             0      0      0      0       1       2       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   286b     2k    79 backend-reports-stat  PRI   20:04:50 
 
                       localhost            29   1334     30      0       0       1                 20.2g  19.6g      0                                                      162k   432k    65                         RTR   20:04:51 
node1.mongo.backend:27018            29      0     31      0      68       8       0   250g   502g  5.74g      4             stat:3.2%          0       0|0     0|0    13k    19k    79 backend-ad-data  PRI   20:04:51 
node4.mongo.backend:27018           *29     30    *30     *0       0     4|0       0   250g   501g  34.5g      0             stat:0.7%          0       0|0     0|0     3k    29k    73 backend-ad-data  SEC   20:04:51 
node5.mongo.backend:27018            *0   1304     *0     *0       0     4|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     0|0   149k   410k    20 backend-reports-stat  SEC   20:04:51 
node6.mongo.backend:27018             0      0      0      0       0       8       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   611b     5k    79 backend-reports-stat  PRI   20:04:51 
 
                       localhost            31   1233     32      0       1       1                 20.2g  19.6g      0                                                      151k     4m    65                         RTR   20:04:52 
node1.mongo.backend:27018            31      0     33      0      67       2       0   250g   502g  5.74g      4             stat:4.6%          0       0|0     0|0    14k    17k    79 backend-ad-data  PRI   20:04:52 
node4.mongo.backend:27018           *31     32    *32     *0       1     2|0       0   250g   501g  34.5g      0             stat:0.6%          0       0|0     0|0     4k     4m    73 backend-ad-data  SEC   20:04:52 
node5.mongo.backend:27018            *0   1202     *0     *0       0     2|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     0|0   137k   370k    20 backend-reports-stat  SEC   20:04:52 
node6.mongo.backend:27018             0      0      0      0       1       8       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   658b     5k    79 backend-reports-stat  PRI   20:04:52 
 
                       localhost            24   1347     25      0       0       1                 20.2g  19.6g      0                                                      162k   446k    65                         RTR   20:04:53 
node1.mongo.backend:27018            24      0     26      0      57       2       0   250g   502g  5.74g      2             stat:3.0%          0       0|0     0|0    11k    14k    79 backend-ad-data  PRI   20:04:53 
node4.mongo.backend:27018           *24     25    *25     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.6%          0       0|0     0|0     3k    22k    73 backend-ad-data  SEC   20:04:53 
node5.mongo.backend:27018            *0   1320     *0     *0       0     2|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     1|0   150k   428k    20 backend-reports-stat  SEC   20:04:53 
node6.mongo.backend:27018             0      0      0      0       0       2       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   239b     2k    79 backend-reports-stat  PRI   20:04:53 
 
                                        insert  query update delete getmore command flushes mapped  vsize    res faults             locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn                    set repl       time 
                       localhost            17   1394     18      0       0       1                 20.2g  19.6g      0                                                      165k   453k    65                         RTR   20:04:54 
node1.mongo.backend:27018            17      0     19      0      45       3       0   250g   502g  5.74g      5             stat:3.5%          0       0|0     0|0     8k    11k    79 backend-ad-data  PRI   20:04:54 
node4.mongo.backend:27018           *17     18    *18     *0       0     2|0       0   250g   501g  34.5g      0             stat:0.4%          0       0|0     0|0     2k    17k    73 backend-ad-data  SEC   20:04:54 
node5.mongo.backend:27018            *0   1377     *0     *0       0     2|0       0   138g   277g  19.3g      0             stat:0.0%          0       0|0     0|0   157k   441k    20 backend-reports-stat  SEC   20:04:54 
node6.mongo.backend:27018             0      0      0      0       1       2       0   138g   279g  24.8g      0             stat:0.0%          0       0|0     1|0   286b     2k    79 backend-reports-stat  PRI   20:04:54 

node6 and node5 - have READONLY shard, that updated by mapreduce only.
And it will be great if node6 also handle READ requests.

Thanks.

Comment by auto [ 05/Jul/12 ]

Author:

{u'date': u'2012-06-14T09:58:05-07:00', u'email': u'randolph@10gen.com', u'name': u'Randolph Tan'}

Message: SERVER-4464 drivers and mongos should support 5 read preferences

Step#3 Integration to mongos and added helpers for mongo shell
Branch: master
https://github.com/mongodb/mongo/commit/83551169374ded2a86838c6b72c8415adb76b1a4

Comment by Randolph Tan [ 03/Jul/12 ]

Note: if a driver sends to mongos with $readPreference without the slaveOk bit, old versioned mongod would complain "not master and slaveOk=false".

Comment by auto [ 11/Jun/12 ]

Author:

{u'login': u'', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: SERVER-4464 drivers and mongos should support 5 read preferences

Step#2 Implement facility for selecting nodes given the read preference and tags.
Branch: master
https://github.com/mongodb/mongo/commit/d27fa2fd6b121916a804cd61cbde806fb6a22e8a

Comment by Randolph Tan [ 07/Jun/12 ]

Author:

{u'login': u'', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: Fixed buildbot failure on assertion failure when Node::toString() is called

Branch: master
https://github.com/mongodb/mongo/commit/84aeb0ef2e6ce205853129199ff2656a44ea4b18

Comment by auto [ 07/Jun/12 ]

Author:

{u'login': u'', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: SERVER-4464 drivers and mongos should support 5 read preferences

Step#1 Added checker for matching tags.
Branch: master
https://github.com/mongodb/mongo/commit/285360d3c6d8e3d3d7c684b1cca64d59f947d8fe

Comment by Scott Hernandez (Inactive) [ 25/Feb/12 ]

In addition all non-primary-only ReadPref modes should support a no-older-than option to cut off reads from stale/replication-delayed nodes.

Generated at Thu Feb 08 03:06:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.