[SERVER-19816] Oplog replication throughput is bounded by network latency Created: 07/Aug/15  Updated: 06/Dec/22  Resolved: 03/Jan/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 2.6.11, 3.0.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Andrew Ryder (Inactive) Assignee: Backlog - Replication Team
Resolution: Done Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-28511 OP_MSG Exhaust support for mongod ing... Closed
Related
is related to SERVER-21797 Simplify find batching logic Closed
Assigned Teams:
Replication
Operating System: ALL
Steps To Reproduce:

This probably depends on getting exhaust support for both find/getmore commands and makign it available to replication

Participants:
Case:

 Description   

Assuming a constant stream of operations totalling 50MB/s data (eg. insertions) to a replica-set, any Secondary that has more than ~40ms network latency (~80ms ping) to its next nearest member will be unable to keep up regardless of the network bandwidth between them.

If the ping time between any two members of a replica set is known, the upper limit for replication throughput in MB/s between those two members can be calculated with the following equation:

4 * 1000 / ping_time_in_ms

The limit arises because the tailable cursor retrieves at max 4MB per roundtrip but the request/reply is serial. Thus, the longer a roundtrip (ping) takes, the lower the throughput.



 Comments   
Comment by Judah Schvimer [ 03/Jan/20 ]

This is being fixed by (PM-1232) Use Exhaust Cursors for Oplog Fetching.

Comment by Eric Milkie [ 21/Dec/15 ]

We're planning on raising the limit to 16MB soon, which will greatly improve things – SERVER-21797. Raising the limit higher than that, or a rearchitecting of the current data fetching design, is a larger undertaking.

Generated at Thu Feb 08 03:52:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.