[SERVER-38148] Test performance when reading from mongos with large catalog cache Created: 15/Nov/18  Updated: 17/Oct/19  Resolved: 27/Nov/18

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Janna Golden Assignee: Janna Golden
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-37660 Add test suite with very large number... Closed
Sprint: Sharding 2018-11-19, Sharding 2018-12-03
Participants:

 Description   

The sharding catalog may see millions of chunks representing unsharded collections, we need to verify that this will not significantly affect performance of the sharding catalog.

Goals

  1. Assess time to do the first read ever through a fresh mongos and up-to-date shards with a large sharding catalog
  2. Check that the steady state latency of a read from a collection that mongos has already seen is not affected by the size of the catalog
  3. Assess time to read from a collection that mongos has not yet seen on a database that said mongos has already seen

Implementation
Create perf workload in 10gen/workloads repo.
1. Record baseline results. The catalog cache will have only the two collections we create and read from in the steps below.

  1. Create a sharded collection with one document through mongos1
  2. Read this same document from the same collection through mongos2 using benchRun() to time the operation
  3. Read this same document from the same collection through mongos2 a second time using benchRun() to time the operation
  4. Create a new collection with one document in the same database as the first collection through mongos1
  5. Read the document in this second collection through mongos2 using benchRun() to time the operation

2. Record results when the catalog cache has additional collections. For each of nCollections = [1000, 10000, 100000, 1000000], do the following:

  1. Create a sharded collection with one document through mongos1
  2. Insert nCollections entries directly into config.collections and config.chunks through mongos1 for the same db as the above collection is in.
  3. Read in the one document in the first collection on every shard
  4. Read this same document from the same collection through mongos2 using benchRun() to time the operation
  5. Read this same document from the same collection through mongos2 a second time using benchRun() to time the operation
  6. Create a new collection with one document in the same database as the first collection through mongos1
  7. Read the document in this second collection through mongos2 using benchRun() to time the operation


 Comments   
Comment by Githook User [ 04/Dec/18 ]

Author:

{'name': 'jannaerin', 'email': 'golden.janna@gmail.com', 'username': 'jannaerin'}

Message: SERVER-38148 test performance in one
Branch: PERF-1760
https://github.com/10gen/workloads/commit/5a666d995aab6695ce8ec4b3da0d55189371c7a1

Generated at Thu Feb 08 04:48:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.