[SERVER-7935] Create write lock levels & Introduce readPreferenceMode = avoidWriteLock Created: 13/Dec/12  Updated: 11/Jul/16  Resolved: 16/Dec/12

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 2.2.2
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Zaid Masud Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

Motivation

In replica-set environments with high bulk inserts, where stale reads are acceptable and read performance is critical, it would be very useful to attempt to find a replica set member that is not holding or pending a write lock. Read performance is expected to improve.

Please note that the below proposal is only an attempt at explaining one (of many) possible implementations of this feature.

Prerequisite

In order to facilitate the algorithm that is proposed below, a write lock level or similar construct is required to be introduced. In addition, each replica set member needs to be able to instantly respond on the current write lock level and with whether or not it is a write lock is pending. The write lock level may be determined by the duration of similar previous write operations, or through other heuristics.

Read Preference Algorithm

Scenario

A client driver receives a read request where the readPreferenceMode has been specified as avoidWriteLock where n is the minimum write lock level to avoid.

Algorithm (pseudo-code)

Server getReadServerAvoidingWriteLock(int writeLockLevel)
{
    foreach(Server m in ReplicaSetMembers)
    {
        if (!m.isHoldingOrPendingWriteLock(writeLockLevel))
        {
            return m;
        }
    }
    return getReadServerSecondaryPreferred();
}



 Comments   
Comment by Zaid Masud [ 16/Dec/12 ]

Just so I understand, suppose there is a very large bulk insert that takes a few seconds on the primary. Are you saying that the write-lock impact of this bulk insert on secondaries is going to be much less than it would be on the primary?

Comment by Eliot Horowitz (Inactive) [ 16/Dec/12 ]

Ok, resolving for now, let us know if things don't work they way you want.

Comment by Zaid Masud [ 15/Dec/12 ]

Thanks for the clarification... we will do some further profiling for our use case and will let you know if secondary preferred member selection is shown to not be appropriate.

I am thinking that secondaries could be deliberately set to different sync schedules. For predictable write patterns, I'm not sure if secondary preferred is completely optimal member selection for read performance.

Anyhow, will let you know if we should revisit this following our profiling.

Comment by Eliot Horowitz (Inactive) [ 15/Dec/12 ]

1) In the time to check if a primary is in a lock, it's likely that something else will start. So this method won't be very reliably in a real system. The time slicing is so fine grained, I'm not even sure its possible.

2) All secondaries will have the same write load. The load there is very time sliced, so any lock held for replication will be very short lived.

I think secondary preferred will give you the functionality you need more reliably.

Comment by Zaid Masud [ 15/Dec/12 ]

There are at least two practical differences from secondarypreferred:

  1. Secondary preferred will never pick the primary if a secondary is available, whereas this option might
  2. My understanding of secondarypreferred is that any of the closest secondaries can be selected at random, whether or not they are currently in a write lock. An avoidWriteLock read preference would select a replica set member only on the basis of whether it was holding or pending a write lock.
Comment by Eliot Horowitz (Inactive) [ 15/Dec/12 ]

I'm not sure how this is practically different than secondarypreferred?

Generated at Thu Feb 08 03:15:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.