Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-3744

Improve Online Archive query performance

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • 0
    • Not Needed

      I'm using a M10 MongoDB Atlas Cluster on AWS, server version 5.0.3 and MongoDB NodeJS driver version 4.1.3 (latest at the time of writing).

      I created an Online Archive on one of the collections using a date field that archives documents after 90 days.

      The query performance is seriously degraded for all queries when using the cluster and online archive connection string. I did some tests with the exact same code, just changing the connection string, with the following results.

      • If I use the cluster connection stringĀ  the performance is the same as before.
      • If I use the cluster and online archive connection string and query a document that is on the archive, the query takes 50x the time it takes with the cluster connection string
      • If I use the cluster and online archive connection string and query a document that is on the cluster, the query takes 10x the time it takes with the cluster connection string.

      A big difference between online archive + cluster and cluster query times is expected but only for records that are on the archive, and not for the ones still on the original collection. Besides, I was finding documents by the date field of the archiving rule so it should be very easy for the system to tell if it's either in the cluster or the archive and go there. So I don't really understand why there's such a difference when querying the cluster.

      The expectation with the Online Archive was that querying archived documents would take a lot longer (no issue there) but querying documents in the cluster would take nearly the same amount of time. This big performance degradation made us decide not to use the feature for other use cases.

            Assignee:
            Unassigned Unassigned
            Reporter:
            fer@arative.com Fernando Neubaum
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: