[SERVER-48378] Expose settings of query knobs in agg explain output Created: 21/May/20  Updated: 29/Oct/23  Resolved: 04/Mar/21

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Improvement Priority: Major - P3
Reporter: David Storch Assignee: Timour Katchaounov
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-48380 Expose total data size in bytes proce... Closed
is related to SERVER-70327 Add internalQueryFrameworkControl to ... Closed
is related to SERVER-21784 Track 'nReturned' and 'executionTimeM... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query Optimization 2021-02-22, Query Optimization 2021-03-08
Participants:

 Description   

Various agg stages are subject to internal limits, defined as setParameters in query_knobs.idl. alex.bevilacqua would find it useful if these values were reported in agg explain. Since these are static settings rather than runtime stats, I think they would be reported at any explain verbosity level ("queryPlanner", "executionStats", or "allPlansExecution").

In particular, exposing at least following would be useful:

  • internalQueryFacetBufferSizeBytes
  • internalLookupStageIntermediateDocumentMaxSizeBytes
  • internalDocumentSourceGroupMaxMemoryBytes

Before scheduling this ticket, we should choose the exact set of knobs we want to expose in explain, and we should choose the names with which we want to expose them.



 Comments   
Comment by Githook User [ 04/Mar/21 ]

Author:

{'name': 'Timour Katchaounov', 'email': 'timour.katchaounov@mongodb.com'}

Message: SERVER-48378 Expose settings of query knobs in agg explain output

Added a top-level section in the explain that shows the most
interesting server parameters (limits).
Branch: master
https://github.com/mongodb/mongo/commit/1f8978300d04f74ac464a3e9221dbadb79e84875

Comment by Alex Bevilacqua [ 17/Feb/21 ]

timour.katchaounov serverParameters seems appropriate.

Comment by Timour Katchaounov [ 17/Feb/21 ]

After a quick look I suggest to add the following "knobs" based on prior experience:

  • internalQueryMaxBlockingSortMemoryUsageBytes
  • internalQueryProhibitBlockingMergeOnMongoS

Since these values are exposed to users, IMO there should be no "internal" prefix in the visible names. On the other hand they have to be easy to relate (ideally automatically) to the internal name. Therefore as a start I'll name them the same, but without the "internal" prefix. Someone with more experience can suggest better names before merging.

I suggest that these are reported as a separate sub-document that is added after "serverInfo". Is "serverParameters" a good name for this sub-document? For instance:

{
    ..... ,
        "serverInfo" : {
                "host" : "ip-10-122-15-79",
                "port" : 27017,
                "version" : "4.9.0-alpha4-353-g3ab59f7",
                "gitVersion" : "3ab59f7e9608e56cff7cd57e2b95a976c11ee292"
        },
        "serverParameters" : {
                "QueryFacetBufferSizeBytes" : 1234567,
                "LookupStageIntermediateDocumentMaxSizeBytes" : 9876543,
                "DocumentSourceGroupMaxMemoryBytes" : 424242
        },
        "ok" : 1
} 

 

alex.bevilacqua, david.storch may I have a quick opinion from you, so I can start working on this?

Generated at Thu Feb 08 05:16:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.