ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Done
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Cluster Scalability
    • Fully Compatible
    • ClusterScalability 25May-5June, ClusterScalability 8Jun-22June
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Report on results of the performance tests I did to see the impact of resharding validations during and right after cloning when collection count validations are run.

      Some high level things I noticed:

      • The current count implementation which uses _id index is very slow relative to resharding (even one that has to build 10 indexes) and in my tests took up to 1.4x the time resharding took to run.
      • The current count implementation is inherently not very predictable or reliable because its performance depends on the collection's data layout and its correlation to the entries in the _id index. In my tests the read amplification (bytes read by count / collection size) was up to 4.5x but it could easily be higher in the worst case.
      • Insert and update CRUD latencies are higher during cloning, likely from increased donor disk utilization caused by the inefficient count. Although the increased latency is likely exaggerated by the current count implementation that requires reading large amount of bytes.

      Recommendations for next steps are here in the doc. 

            Assignee:
            Wenqin Ye
            Reporter:
            Wenqin Ye
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: