[COMPASS-6362] Investigate changes in SERVER-70267: Streaming group when id is on time Created: 09/Dec/22  Updated: 07/Jun/23  Resolved: 09/Dec/22

Status: Closed
Project: Compass
Component/s: None
Affects Version/s: None
Fix Version/s: No version

Type: Investigation Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-70267 Streaming group when id is on time Closed
Documentation Changes: Not Needed

 Description   
Original Downstream Change Summary

In some cases for timeseries (example below) optimizer may replace $group with $_internalStreamingGroup stage.

In addition to all $group fields, it will also contain $monotonicIdFields field that lists all _id fields that are considered monotonic by the optimizer. If _id is a single expression, it will just contain a single element “_id”.

Example of explain output with the new stage
{
"explainVersion" : "1",
"stages" : [
{
"$cursor" : {
"queryPlanner" : {
"namespace" : "timeseries.system.buckets.Collection0",
"indexFilterSet" : false,
"parsedQuery" : {

},
"queryHash" : "17830885",
"planCacheKey" : "17830885",
"maxIndexedOrSolutionsReached" : false,
"maxIndexedAndSolutionsReached" : false,
"maxScansToExplodeReached" : false,
"winningPlan" :

{ "stage" : "COLLSCAN", "direction" : "forward" }

,
"rejectedPlans" : [ ]
}
}
},
{
"$_internalUnpackBucket" :

{ "include" : [ "price", "time", "symbol" ], "timeField" : "time", "metaField" : "symbol", "bucketMaxSpanSeconds" : 3600, "assumeNoMixedSchemaData" : true, "includeMinTimeAsMetadata" : true "internalQueryFacetBufferSizeBytes" : 104857600, "internalQueryFacetMaxOutputDocSizeBytes" : 104857600, "internalLookupStageIntermediateDocumentMaxSizeBytes" : 104857600, "internalDocumentSourceGroupMaxMemoryBytes" : 104857600, "internalQueryMaxBlockingSortMemoryUsageBytes" : 104857600, "internalQueryProhibitBlockingMergeOnMongoS" : 0, "internalQueryMaxAddToSetBytes" : 104857600, "internalDocumentSourceSetWindowFieldsMaxMemoryBytes" : 104857600, "internalQueryFrameworkControl" : "tryBonsai" }

,
"command" : {
"aggregate" : "system.buckets.Collection0",
"pipeline" : [
{
"$_internalUnpackBucket" :

{ "timeField" : "time", "metaField" : "symbol", "bucketMaxSpanSeconds" : 3600, "assumeNoMixedSchemaData" : true, "includeMinTimeAsMetadata" : true [75/5353] }

},
{
"$_internalBoundedSort" : {
"sortKey" :

{ "time" : 1 }

,
"bound" :

{ "base" : "min", "offsetSeconds" : NumberLong(0) }

,
"limit" : NumberLong(0)
}
},
{
"$_internalStreamingGroup" : {
"_id" : {
"symbol" : "$symbol",
"time" : {
"$dateTrunc" : {
"date" : "$time",
"unit" :

{ "$const" : "minute" }

,
"binSize" :

{ "$const" : 61 }

}
}
},
"high" :

{ "$max" : "$price" }

,
"low" :

{ "$min" : "$price" }

,
"open" :

{ "$first" : "$price" }

,
"close" :

{ "$last" : "$price" }

,
"$monotonicIdFields" : [
"time"
]
}
}
],
"serverInfo" :

{ "host" : "ip-10-122-10-70", "port" : 27017, "version" : "6.3.0-alpha-261-geac3840", "gitVersion" : "eac384080001c4ba2dbbb72cb79550deb215a79a" }

,
"serverParameters" :

{ "internalQueryFacetBufferSizeBytes" : 104857600, "internalQueryFacetMaxOutputDocSizeBytes" : 104857600, "internalLookupStageIntermediateDocumentMaxSizeBytes" : 104857600, "internalDocumentSourceGroupMaxMemoryBytes" : 104857600, "internalQueryMaxBlockingSortMemoryUsageBytes" : 104857600, "internalQueryProhibitBlockingMergeOnMongoS" : 0, "internalQueryMaxAddToSetBytes" : 104857600, "internalDocumentSourceSetWindowFieldsMaxMemoryBytes" : 104857600, "internalQueryFrameworkControl" : "tryBonsai" }

,
"command" : {
"aggregate" : "system.buckets.Collection0",
"pipeline" : [
{
"$_internalUnpackBucket" :

{ "timeField" : "time", "metaField" : "symbol", "bucketMaxSpanSeconds" : 3600, "assumeNoMixedSchemaData" : true, "usesExtendedRange" : false }

},
{
"$sort" :

{ "time" : 1 }

},
{
"$group" : {
"_id" : {
"symbol" : "$symbol",
"time" : {
"$dateTrunc" :

{ "date" : "$time", "unit" : "minute", "binSize" : 61 }

}
},
"high" :

{ "$max" : "$price" }

,
"low" :

{ "$min" : "$price" }

,
"open" :

{ "$first" : "$price" }

,
"close" :

{ "$last" : "$price" }

}
}
],
"cursor" : {

},
"collation" :

{ "locale" : "simple" }

},
"ok" : 1
}

Description of Linked Ticket

null



 Comments   
Comment by PM Bot [ 07/Jun/23 ]

Fix Version updated for upstream SERVER-70267:
6.3.0-rc0, 6.0.7

Comment by Alena Khineika [ 09/Dec/22 ]

Does not impact the DevTools products.

Generated at Wed Feb 07 22:42:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.