Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16724

Startup time under WiredTiger is extremely slow and gets worse as database grows

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.8.0-rc5
    • Affects Version/s: 2.8.0-rc3
    • Component/s: Performance, Storage
    • Labels:
    • Fully Compatible
    • ALL

      I did a test of my company's analytics app, using the WiredTiger storage engine, on Windows Server 2012, on a HyperV VM. During the test, I noticed that startup time for MongoDB was getting excessive - 2 hours in one case, 2.5 hours in another. It had been very quick the first few times I started it. I didn't take a lot of time to log the details of what was wrong, but the difference between the various times was only in the amount of data in the database - the more data, the slower the startup time. (Roughly 6MM records -> 2 hours, roughly 7.5MM -> 2.5 hours.)

      I don't have stats from those instances, but one of the databases in the instance is a single collection with one record per incoming message, stored as binary data, with a few fields of metadata and a few indexes. I assume it's that database that's causing the problem. (During the entire startup period, the system shows lots of read activity for a single huge file from the mongodb data directory.) The other databases have much lower overall volume/size.

      Sometime in the next day or so I'll need to restart it again, with about 20MM records in it. I'll collect more detailed info about it, then.

      I've attached log snippets from the two slow startups.

        1. additional_startup_time_logs.txt
          3 kB
        2. db_details_commands.txt
          0.4 kB
        3. db_details_output.txt
          30 kB
        4. mongodb_startup_wiredtiger_slow.txt
          3 kB
        5. more_shutdown_logs.txt
          5 kB

            mark.benvenuto@mongodb.com Mark Benvenuto
            narthur_pre Nathan Arthur
            0 Vote for this issue
            8 Start watching this issue