Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16922

Slow index build times on secondaries with Wired Tiger

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-rc6
    • Component/s: Replication, WiredTiger
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible

      Description

      Testing Migration to Wired Tiger from MMAP0 in 2.6 by adding a WT Replica.

      Servers:
      Data Volume - 1 TB
      Servers Amazon C4.2XLarge
      8 x CPU
      15GB RAM
      Disks 2 x 1TB SSD EBS with 4000PIOS in Raid 0 (8000 PIOPS)

      No load on source (primary) server
      522,771,027 Documents in collection

      Data Copying Phase:
      Began: 08:15 16th Jan
      Finished 18:23 16th Jan

      Creation of _id Index (_id is standard OID's)

      2015-01-16T18:23:17.006+0000 I - [rsSync] Index Build: 123300/522771027 0%

      Currently

      2015-01-17T19:09:58.004+0000 I - [rsSync] Index Build: 150625100/522771027 28%

      Not only over 24 hours for 28% but observably taking 4 minutes longer per 1%

      At 396 minutes for the final 1% Total predicted time is

      396/2 * 100 = 330 Hours to build one index.

      By comparison - I tested 2.6 on the same hardware and this step took 7hrs 36 minutes.

      Is this the case for all WT index creations - is time to build polynomial (and greater than n log n I think) with number of records - Is this just increasing indexes or all indexes. If I had 3 or 4 secondary indexes on this are they built sequentially or in parallel?

        Attachments

        1. jmongoiibench.java
          35 kB
        2. mongod.log
          1.38 MB
        3. POCDriver.jar
          921 kB
        4. run.simple.bash
          4 kB

          Activity

            People

            • Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: