Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30391

Add QoS features to MongoDB

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: Usability
    • Labels:
      None

      Description

      I am currently writing an application that generates sustained, heavy writes for about one hour at >200Mbit/s. This application is regularly causing performance issues on the mongodb side.

      It currently runs relatively fine on our atlas production setup using write-concern: majority and a single insertation thread, but it is causing very high loads (>90% disk utilization) on our test setup. This is in would be desired (maxing out the server's resources is good because we get more performance), but we had a db crash in the past (https://jira.mongodb.org/browse/MMSSUPPORT-14543 caused probably by a performance difference between the primary and replication – https://jira.mongodb.org/browse/SERVER-24242) and we are still not sure this could not happen again.

      In addition, using write-concern: majority causes a significant latency wich seems to be slowing down our application by a lot (now taking 3h instead of 1h since I activated this).

      It would be very useful if there was a way to explicitly mark operations as high-impact/best-effort/background jobs. Mongodb should make sure that all operations that are not marked in such a way are given priority before the high-impact tasks. Mongodb should also make sure that high-impact operations can never overtax the server; these operations should either be rejected or blocked until enough resources are available to process them.

      (Blocking would be very useful because then on my side I could write a scheduler that reduces the number of insertation threads or throttles them when many jobs are rejected).

      This way it would be possible to operate high-load jobs at maximum efficiency without stealing resources from routine operations.

        Attachments

          Activity

            People

            • Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated: