Reduce impact of the database profiler

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The profiler has two main performance impacts:

      • Writes to the profiler are serialized using the "metadata" mutex. Under heavy load, this causes slow queries to queue, which increase latencies.
      • Writes to the profiler take write tickets. Under load, writes to the profiler queue on the metadata lock while holding tickets, which causes other write operations to queue, even those not profiling.

      We can mitigate the profiler's impact by doing one of the following:

      • Set a deadline on the OperationContext used perform the profile operation. This would cause the lock acquisitions to fail if any of locks are contended, which is simple, but this could cause some slow writes to fail after doing work, which would be wasteful.
      • More precise: specify a deadline to the lock acquisitions explicitly in the CollectionAcquisitionRequest, and also on the metadata lock here, which is what serializes the writes to the collection. This would ensure that we only fail when locks are contended, and not when writes are slow.

      With either option, we should add a serverStatus metric that indicates when we are dropping profiler entries.

            Assignee:
            Charlie Swanson
            Reporter:
            Louis Williams
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: