[SERVER-70996] Add serverStatus reporting for query analysis Created: 01/Nov/22  Updated: 29/Oct/23  Resolved: 09/Mar/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0

Type: Task Priority: Major - P3
Reporter: Cheahuychou Mao Assignee: Israel Hsu
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
Related
related to SERVER-70997 Add currentOp reporting for query ana... Closed
Assigned Teams:
Sharding NYC
Backwards Compatibility: Fully Compatible
Sprint: Sharding NYC 2023-02-06, Sharding NYC 2023-02-20, Sharding NYC 2023-03-06
Participants:
Linked BF Score: 145

 Description   

From the design:

The serverStatus command response on mongos and mongod will include a “queryAnalyzers” section with the following diagnostics:

{
    activeCollections: <integer>, // On a shared cluster, this is a mongos-only field since shardsvr mongods don’t know about sample rates.  
    totalCollections: <integer>,
    totalSampledReadsCount: <integer>,
    totalSampledWritesCount: <integer>,
    totalSampledReadsBytes: <integer>,
    totalSampledWritesBytes: <integer>,
}



 Comments   
Comment by Githook User [ 09/Mar/23 ]

Author:

{'name': 'Israel Hsu', 'email': 'israel.hsu@mongodb.com', 'username': 'israelhsu'}

Message: SERVER-70996 Add serverStatus reporting for query analysis, reorg QueryAnalysis sample counting
Branch: master
https://github.com/mongodb/mongo/commit/5b99e24bf230cb1b22b1b8df3f485438ca3ef911

Comment by Israel Hsu [ 22/Feb/23 ]

As per discussion with Chou last week:

  1. Renamed the existing SampleCounters to CollectionSampleCounters and moved it to a class within the new QueryAnalysisSampleCounters class; CollectionSampleCounters also caches the per-collection sample rate for reporting by mongos.
  2. The global instance of QueryAnalysisSampleCounters is a decoration of ServiceContext and it is called on by both QueryAnalysisWriter and QueryAnalysisSampler. QueryAnalysisSampleCounters contains a map of CollectionSampleCounters indexed by collection namespace strings, and state for server-level counting of samples and sample sizes.
  3. $currentOp and ServerStatus reports are generated by calls to the global {{QueryAnalysisSampleCounters }}instance.
  4. Created BSON documents for the reports in analyze_shard_key_common.idl.

 

Comment by Israel Hsu [ 15/Feb/23 ]

cheahuychou.mao@mongodb.com This is my current thinking on this server-wide counting:

Rename the existing SampleCounters to CollectionSampleCounters.

I'd like to avoid having the calling code have to invoke an increment() function for both a collection-level counter and a server-level counter. I propose having a new object called SampleCounters that maintains both server-level and collection-level counters; it contains a map of the CollectionSampleCounters indexed by collection UUID. The new SampleCounters would contain all the logic to generate serverStatus section and the $currentOp reports. CollectionSampleCounters would be abstracted away from the POV of QueryAnalysisWriter and QueryAnalysisSampler.

Both QueryAnalysisSampler and QueryAnalysisWriter would have an instance of the new SampleCounters. When they need to count, they invoke SampleCounters::incrementRead(ns, collUuid, size) where size is boost::optional.

Thoughts?

Generated at Thu Feb 08 06:17:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.