Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-18314

Stall during fdatasync phase of checkpoints under WiredTiger and EXT4

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: 3.0.2
    • Fix Version/s: None
    • Component/s: WiredTiger
    • Labels:
    • Operating System:
      ALL
    • Case:

      Description

      • YCSB 30M documents, 10 fields, ~1kB/document, total ~30GB
      • 50/50 read/update workload
      • 40 GB cache, 128 GB memory, 32 CPUs
      • slow SSD disk (~80-100 MB/s)
      • no journal (to simplify the situation)
      • per mongostat, cache is at 100% utilization, 80% dirty pretty much throughout the test.

      During each checkpoint two calls to fdatasync are made. Because this scenario is i/o constrained the fdatasyncs take a substantial amount of time, and during both fdatasync calls throughput falls to exactly 0 for the duration of the fdatasync. This is seen in A-B, C-D, E-F, G-H, I-J, K-L below.

      In many, but not all, such cases WT bumps the "eviction server unable to reach goal" counter.

      Similar test with a larger cache (the default 64GB) does not show this issue.

      Note: this is the same test as reported in SERVER-18315; opening two separate tickets to track what may be separate issues.

        Attachments

        1. count=1000000.png
          count=1000000.png
          10 kB
        2. lz4.png
          lz4.png
          9 kB
        3. snappy.png
          snappy.png
          11 kB
        4. try-13.png
          try-13.png
          145 kB
        5. try-14.png
          try-14.png
          105 kB
        6. try-16.png
          try-16.png
          65 kB
        7. try-21.png
          try-21.png
          142 kB
        8. try-23.png
          try-23.png
          114 kB
        9. try-25.png
          try-25.png
          116 kB
        10. try-26.png
          try-26.png
          114 kB
        11. try-29.png
          try-29.png
          117 kB
        12. try-30.png
          try-30.png
          77 kB
        13. try-33.png
          try-33.png
          62 kB
        14. try-34.png
          try-34.png
          60 kB
        15. try-35.png
          try-35.png
          63 kB
        16. try-36.png
          try-36.png
          63 kB

          Issue Links

            Activity

              People

              Assignee:
              ramon.fernandez Ramon Fernandez Marina
              Reporter:
              bruce.lucas Bruce Lucas
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              38 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: