[DOCS-10544] Docs for SERVER-22766: Dynamic oplog sizing for WiredTiger nodes Created: 18/Jul/17  Updated: 13/Nov/23  Resolved: 24/Mar/23

Status: Closed
Project: Documentation
Component/s: Server
Affects Version/s: None
Fix Version/s: 3.5.10, 3.4.19, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Emily Hall Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-22766 Dynamic oplog sizing for WiredTiger n... Closed
Duplicate
is duplicated by DOCS-10811 Docs for SERVER-30151: Size specifica... Closed
Related
related to DOCS-12231 3.6 and 4.0 oplog resize procedures o... Closed
related to DOCS-12266 Docs for SERVER-38501: swap out new A... Closed
Participants:
Days since reply: 45 weeks, 5 days ago
Epic Link: DOCS: 3.6 Server

 Description   

Documentation Request Summary:

A new command replSetResizeOplog allows changing oplog size on replica set members using the WiredTiger storage engine.

Engineering Ticket Description:

Proposed title: Dynamic oplog sizing of replica set

Issue Status as of Jul 10, 2017

FEATURE DESCRIPTION
This new feature enables the dynamic resizing of a node's oplog, allowing users to grow and shrink the oplog to satisfy the operational needs of each node in a replica set.

VERSIONS
This feature is available in the 3.5.10 and newer development versions of MongoDB, and in the 3.6 and newer production releases.

RATIONALE
In a MongoDB replica set, the oplog is a special capped collection used to replicate data to other nodes. Users may specify the size of the oplog for each node when deploying a replica set, but altering that size required maintenance.

This new feature bypasses the need for maintance and allows users to dynamically change the size of the oplog in MongoDB nodes running with the WiredTiger storage engine only (nodes running the MMAPv1 storage engine still need to follow the maintenance procedure linked above).

OPERATION
To display the size of the oplog, one can use the stats() command. Here's a replica set primary node with a 9.1GB oplog:

replset:PRIMARY> use local
replset:PRIMARY> db.oplog.rs.stats()
{
        "ns" : "local.oplog.rs",
        "size" : 6781,
        "count" : 60,
        "avgObjSize" : 113,
        "storageSize" : 36864,
        "capped" : true,
        "max" : -1,
        "maxSize" : NumberLong("9790804377"),
        ...
}

The size of the oplog is displayed at deployment time in the logs as follows:

2017-07-17T17:47:11.870-0400 I REPL     [replication-0] creating replication oplog of size: 9337MB...

Users can change the size of the oplog with the replSetResizeOplog command, specifying a size in MB. For example:

replset:PRIMARY> use admin
replset:PRIMARY> db.runCommand({replSetResizeOplog:1, size: 16384})
{ "ok" : 1, "operationTime" : Timestamp(1500329291, 1) }

The command above changes the oplog size to 16384 MB (16GB/17179869184 bytes). This operation is recorded in the logs as follows:

2017-07-17T17:52:40.396-0400 I STORAGE  [conn11] replSetResizeOplog success, currentSize:17179869184

The output of the stats() command also reflects the change:

replset:PRIMARY> use local
replset:PRIMARY> db.oplog.rs.stats()
{
        "ns" : "local.oplog.rs",
        "size" : 6781,
        "count" : 60,
        "avgObjSize" : 113,
        "storageSize" : 36864,
        "capped" : true,
        "max" : -1, 
        "maxSize" : NumberLong("17179869184")
        ...
}

ADDITIONAL NOTES
Reducing the size of the oplog removes data from it. Reducing the size of the oplog in a given node may cause any members of a replica set syncing from that node to become stale and need a resync.

Original description

It would be very, very handy to have a method to alter the oplog size for an entire replicaset at the same time. That is, issue a command on the primary, it expands its oplog size, and the changes trickle down to the replicas.

Better yet would be a dynamic oplog where it doesn't need to be a specific size. Capped collections have advantages, I understand, but perhaps there are ways around limitations of uncapped ones for oplog purposes.



 Comments   
Comment by Sarah Olson [ 24/Mar/23 ]

Closing this out on the grounds that:

  • Details of this request are significantly outdated. 
  • Our documentation has evolved substantially since this request was made. 

Based on this, closing as WON'T DO. Please don't hesitate to give me a shout or to reopen if you disagree.

 

Comment by Eric Milkie [ 10/Dec/18 ]

Docs please note: Due to the way mongos works in 3.2, I had to change the backport for this command to use a pre-existing action type instead of creating a new one. This means to run the command on 3.4, one must have a role with replSetConfigure action type, not replSetResizeOplog as is the case on 3.6. The replSetConfigure action type is part of the clusterManager role, whereas on 3.6, the new replSetResizeOplog action type was placed in the hostManager role.
See SERVER-38501.

Comment by Shannon Bradshaw (Inactive) [ 07/Dec/18 ]

Thanks, kay.kim

Comment by Kay Kim (Inactive) [ 20/Sep/17 ]

https://github.com/mongodb/docs/commit/ce9d7a352a5640dfd93113aa93484e32dd9c72de

Comment by Louis Williams [ 08/Sep/17 ]

Updated to reflect that the command accepts the size in MB.

Generated at Thu Feb 08 08:00:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.