Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-23935

Disable oplog sampling in queryable backup mode

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.4.15, 3.6.0-rc1
    • Affects Version/s: 3.1.8
    • Component/s: Storage
    • Labels:
    • Fully Compatible
    • v3.4
    • Storage 2017-10-23

      When running with --queryableBackupMode, WiredTigerKVEngine::initRsOplogBackgroundThread() should not start a thread, and therefore not perform oplog sampling.

      In addition, we should consider adding a separate configuration option to disable this oplog sampling without needing to be running with --queryableBackupMode.

      Original Description

      In version 3.1.8 mongo added a new feature: SERVER-19551

      On startup WiredTiger now needs to read values from the oplog to figure out where to place the milestones:

      2016-04-26T15:42:22.066+0000 I STORAGE  [initandlisten] The size storer reports that the oplog contains 19279531 records totaling to 53948096450 bytes
      2016-04-26T15:42:22.410+0000 I STORAGE  [initandlisten] Sampling from the oplog between Apr 23 22:02:46:71 and Apr 26 14:10:01:d1 to determine where to place markers for truncation
      2016-04-26T15:42:22.410+0000 I STORAGE  [initandlisten] Taking 1004 samples and assuming that each section of oplog contains approximately 191863 records totaling to 536872169 bytes

      This process is fine if all of the data is stored locally on disk. The problem is when you are using external storage (for us, Amazon EBS). The sampling from cold EBS is taking somewhere between 20 and 30 minutes leading to incredibly slow startup times.

      Is there a good way to speed up this process / bring startup times back to 3.1.7 levels for EBS backed instances?

            neha.khatri Neha Khatri
            paetling@gmail.com Alex Etling
            0 Vote for this issue
            14 Start watching this issue