Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-38148

Test performance when reading from mongos with large catalog cache

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Sharding
    • None
    • Sharding 2018-11-19, Sharding 2018-12-03

      The sharding catalog may see millions of chunks representing unsharded collections, we need to verify that this will not significantly affect performance of the sharding catalog.

      Goals

      1. Assess time to do the first read ever through a fresh mongos and up-to-date shards with a large sharding catalog
      2. Check that the steady state latency of a read from a collection that mongos has already seen is not affected by the size of the catalog
      3. Assess time to read from a collection that mongos has not yet seen on a database that said mongos has already seen

      Implementation
      Create perf workload in 10gen/workloads repo.
      1. Record baseline results. The catalog cache will have only the two collections we create and read from in the steps below.

      1. Create a sharded collection with one document through mongos1
      2. Read this same document from the same collection through mongos2 using benchRun() to time the operation
      3. Read this same document from the same collection through mongos2 a second time using benchRun() to time the operation
      4. Create a new collection with one document in the same database as the first collection through mongos1
      5. Read the document in this second collection through mongos2 using benchRun() to time the operation

      2. Record results when the catalog cache has additional collections. For each of nCollections = [1000, 10000, 100000, 1000000], do the following:

      1. Create a sharded collection with one document through mongos1
      2. Insert nCollections entries directly into config.collections and config.chunks through mongos1 for the same db as the above collection is in.
      3. Read in the one document in the first collection on every shard
      4. Read this same document from the same collection through mongos2 using benchRun() to time the operation
      5. Read this same document from the same collection through mongos2 a second time using benchRun() to time the operation
      6. Create a new collection with one document in the same database as the first collection through mongos1
      7. Read the document in this second collection through mongos2 using benchRun() to time the operation

            Assignee:
            janna.golden@mongodb.com Janna Golden
            Reporter:
            janna.golden@mongodb.com Janna Golden
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: