Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-68245

Degraded performance after 4.4 version upgrade

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.4.15
    • Component/s: None
    • ALL

      Hi,

      We metrics.2022-07-22T00-20-13Z-00000 replicaset, with 3 nodes. We were earlier on version 4.2
      Average Response time: 1.5 ms
      p55: 5ms

      The version upgrade to 4.4 was done around 21st July 10:00 pm. Immediately after the upgrade we did not notice any issues, but after a few hours, we started 

      seeing an increase in response time and a lot of slow queries on mongo side.
      Average Response time: 3.5 ms
      p55: 15ms

      Data size: 5TB(uncompressed)
      Index size: 35GB

      We also observed the following:

      • Increase in memory page faults
      • Dirty data and dirty pages in cache
      • Increase in blocks read
      • Decrease in blocks written
      • Increase in cache evictions
      • Increase in cache pages read
      • Decrease in cache pages written

      Please find attached the screenshots and the diagnostic data.

      Please help us in finding out the issue and suggest an appropriate fix for this.
      Thanks in advance!metrics.2022-07-22T05-45-13Z-00000

        1. blocks read.png
          blocks read.png
          89 kB
        2. blocks read vs vblocks written.png
          blocks read vs vblocks written.png
          97 kB
        3. blocks read vs vblocks written-1.png
          blocks read vs vblocks written-1.png
          97 kB
        4. cache pages read and written to disk.png
          cache pages read and written to disk.png
          99 kB
        5. cahce eviction.png
          cahce eviction.png
          197 kB
        6. dirty cache.png
          dirty cache.png
          203 kB
        7. metrics.2022-07-22T00-20-13Z-00000
          9.97 MB
        8. metrics.2022-07-22T05-45-13Z-00000
          4.17 MB

            Assignee:
            chris.kelly@mongodb.com Chris Kelly
            Reporter:
            ishrat.jahan@gojek.com Ishrat Jahan
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: