Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22766

Dynamic oplog sizing for WiredTiger nodes



    • Type: New Feature
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.19, 3.5.10
    • Component/s: Replication
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:


      Proposed title: Dynamic oplog sizing of replica set

      Issue Status as of Jul 10, 2017

      This new feature enables the dynamic resizing of a node's oplog, allowing users to grow and shrink the oplog to satisfy the operational needs of each node in a replica set.

      This feature is available in the 3.5.10 and newer development versions of MongoDB, and in the 3.6 and newer production releases.

      In a MongoDB replica set, the oplog is a special capped collection used to replicate data to other nodes. Users may specify the size of the oplog for each node when deploying a replica set, but altering that size required maintenance.

      This new feature bypasses the need for maintance and allows users to dynamically change the size of the oplog in MongoDB nodes running with the WiredTiger storage engine only (nodes running the MMAPv1 storage engine still need to follow the maintenance procedure linked above).

      To display the size of the oplog, one can use the stats() command. Here's a replica set primary node with a 9.1GB oplog:

      replset:PRIMARY> use local
      replset:PRIMARY> db.oplog.rs.stats()
              "ns" : "local.oplog.rs",
              "size" : 6781,
              "count" : 60,
              "avgObjSize" : 113,
              "storageSize" : 36864,
              "capped" : true,
              "max" : -1,
              "maxSize" : NumberLong("9790804377"),

      The size of the oplog is displayed at deployment time in the logs as follows:

      2017-07-17T17:47:11.870-0400 I REPL     [replication-0] creating replication oplog of size: 9337MB...

      Users can change the size of the oplog with the replSetResizeOplog command, specifying a size in MB. For example:

      replset:PRIMARY> use admin
      replset:PRIMARY> db.runCommand({replSetResizeOplog:1, size: 16384})
      { "ok" : 1, "operationTime" : Timestamp(1500329291, 1) }

      The command above changes the oplog size to 16384 MB (16GB/17179869184 bytes). This operation is recorded in the logs as follows:

      2017-07-17T17:52:40.396-0400 I STORAGE  [conn11] replSetResizeOplog success, currentSize:17179869184

      The output of the stats() command also reflects the change:

      replset:PRIMARY> use local
      replset:PRIMARY> db.oplog.rs.stats()
              "ns" : "local.oplog.rs",
              "size" : 6781,
              "count" : 60,
              "avgObjSize" : 113,
              "storageSize" : 36864,
              "capped" : true,
              "max" : -1, 
              "maxSize" : NumberLong("17179869184")

      Reducing the size of the oplog removes data from it. Reducing the size of the oplog in a given node may cause any members of a replica set syncing from that node to become stale and need a resync.

      Original description

      It would be very, very handy to have a method to alter the oplog size for an entire replicaset at the same time. That is, issue a command on the primary, it expands its oplog size, and the changes trickle down to the replicas.

      Better yet would be a dynamic oplog where it doesn't need to be a specific size. Capped collections have advantages, I understand, but perhaps there are ways around limitations of uncapped ones for oplog purposes.


          Issue Links



              geert.bosch Geert Bosch
              shakir.sadikali Shakir Sadikali
              1 Vote for this issue
              22 Start watching this issue