Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16922

Slow index build times on secondaries with Wired Tiger

    • Fully Compatible

      Testing Migration to Wired Tiger from MMAP0 in 2.6 by adding a WT Replica.

      Servers:
      Data Volume - 1 TB
      Servers Amazon C4.2XLarge
      8 x CPU
      15GB RAM
      Disks 2 x 1TB SSD EBS with 4000PIOS in Raid 0 (8000 PIOPS)

      No load on source (primary) server
      522,771,027 Documents in collection

      Data Copying Phase:
      Began: 08:15 16th Jan
      Finished 18:23 16th Jan

      Creation of _id Index (_id is standard OID's)

      2015-01-16T18:23:17.006+0000 I - [rsSync] Index Build: 123300/522771027 0%

      Currently

      2015-01-17T19:09:58.004+0000 I - [rsSync] Index Build: 150625100/522771027 28%

      Not only over 24 hours for 28% but observably taking 4 minutes longer per 1%

      At 396 minutes for the final 1% Total predicted time is

      396/2 * 100 = 330 Hours to build one index.

      By comparison - I tested 2.6 on the same hardware and this step took 7hrs 36 minutes.

      Is this the case for all WT index creations - is time to build polynomial (and greater than n log n I think) with number of records - Is this just increasing indexes or all indexes. If I had 3 or 4 secondary indexes on this are they built sequentially or in parallel?

        1. mongod.log
          1.38 MB
        2. POCDriver.jar
          921 kB
        3. jmongoiibench.java
          35 kB
        4. run.simple.bash
          4 kB

            Assignee:
            atul.kachru Atul Kachru
            Reporter:
            john.page@mongodb.com John Page
            Votes:
            1 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: