Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32115

Low mongos performance with adaptive Service Executor

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6.0-rc4
    • Component/s: Networking
    • Labels:
      None
    • Service Arch
    • ALL
    • Hide
      • Set up a two-shard cluster.
      • On my mongos I passed --setParameter ShardingTaskExecutorPoolMaxSize=100 --setParameter taskExecutorPoolSize=4 (to avoid hitting hard ulimit on open files)
      • Pre-spit the sharded collection with the make-sh.js I attached and wait for the balancing to complete
      • $ java -cp load:mongo-java-driver/mongo-java-driver/build/libs/mongo-java-driver-3.5.0.jar load host=172.31.43.41 op=ramp_query docs=1000000 threads=500 tps=10 qps=1000 dur=600 size=100 updates=10
      Show
      Set up a two-shard cluster. On my mongos I passed --setParameter ShardingTaskExecutorPoolMaxSize=100 --setParameter taskExecutorPoolSize=4 (to avoid hitting hard ulimit on open files) Pre-spit the sharded collection with the make-sh.js I attached and wait for the balancing to complete $ java -cp load:mongo-java-driver/mongo-java-driver/build/libs/mongo-java-driver-3.5.0.jar load host=172.31.43.41 op=ramp_query docs=1000000 threads=500 tps=10 qps=1000 dur=600 size=100 updates=10

      Reproduced on version r3.6.0-rc4-41-ge608b8b349.

      On a two-shard environment, I generated a 100% query workload via the load.java attached (which is a modified version of load.java from SERVER-30613).

      Configurations tested:
      A. The sharded cluster operating with synchronous (default) service executor.
      B. The sharded cluster operating with asynchronous (adaptive) service executor.
      C. The sharded cluster operating with asynchronous (adaptive) service executor, except for mongos which was operating in synchronous mode.

      Performance of A and C is comparable, whereas B shows 73% the query throughput of A.

      For the test I used four AWS EC2 m4.4xlarge. One machine for the traffic generator, one for mongos and 1-member config server, and one machine for each shard (1-member replica set).

      Attached logs and diagnostic data for the three configurations.

        1. load.java
          10 kB
        2. make-sh.js
          0.3 kB
        3. SERVER-32115.tar.gz
          1.47 MB

            Assignee:
            backlog-server-servicearch [DO NOT USE] Backlog - Service Architecture
            Reporter:
            josef.ahmad@mongodb.com Josef Ahmad
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: