Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-48431

MongoDB concurrent queries on different collections are slow

    XMLWordPrintable

    Details

      Description

      There is linearly increasing latency observed with multiple concurrent aggregations on different collections.

      Scenario:

      Multiple mongodb collections are created with 100,000 documents in each collection and 10,000 columns in each document. There is a python script which executes aggregate queries in a multi-threaded fashion. Each thread invokes an aggregate on a separate collection. (The code of script is attached with this ticket).

      When the script is executed, the amount of time it takes to complete the aggregation is proportional to the number of threads. i.e. the latency linearly increases with number of queries concurrently executed. If single aggregation query takes 'x' amount of time, running 'n' queries concurrently takes almost 'n*x' amount of time.

      The expectation was that the multi-threaded queries would take roughly the same amount of time as the single-threaded one.  Is this the known limitation of mongodb? Is there any configuration parameter in mongodb to control concurrency?

        Attachments

        1. mongo_logs.tar
          165 kB
        2. perf_test3.py
          2 kB
        3. swaping.png
          swaping.png
          139 kB

          Activity

            People

            Assignee:
            dmitry.agranat Dmitry Agranat
            Reporter:
            surnisaa@gmail.com Ajinkya Surnis
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: