[SERVER-64235] Last point TS Opt M3: Query rewrite for two {+-meta, +ts} cases Created: 04/Mar/22  Updated: 29/Oct/23  Resolved: 21/Mar/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0.0-rc0

Type: New Feature Priority: Major - P3
Reporter: Steve Tarzia Assignee: Alya Berciu
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
Related
related to SERVER-61657 Last point TS Opt M3: Query rewrite f... Closed
Backwards Compatibility: Fully Compatible
Participants:
Linked BF Score: 178

 Description   

Implement the appropriate query rewrite the 2 remaining cases with {+-meta, +ts}.  Assuming that we extend the indexes to both (min,max) and (max,min) the rewrites become:

For reference, here's the case we already did in Milestone :2 

index({meta.sensorId: 1, meta.sensorId: -1})
 
db.telemetry.aggregate([
    {$sort: {"metadata.sensorId": 1, "timestamp": -1}},
    {$group: {
      _id: "$metadata.sensorId",
      ts: {$first: "$timestamp"},
      temp: {$first: "$temp"}
    }}
 ])

translates to:

db.system.buckets.telemetry.explain("executionStats").aggregate([
   {$sort: {"meta.sensorId": 1, "control.max.timestamp": -1}},
   {$group: {
      _id: "$meta.sensorId",
      bucket: {$first: "$_id"},
   }},
   {$lookup: {
      from: "system.buckets.telemetry",
      foreignField: "_id",
      localField: "bucket",
      as: "bucket_data",
      pipeline:[
         {$_internalUnpackBucket: {
            timeField:"timestamp",
            metaField:"tags",
            bucketMaxSpanSeconds:NumberInt("60")
         }},
         {$sort: {"timestamp": -1}},
         {$limit:1}
      ]
   }},
   {$unwind: "$bucket_data"},
   {$replaceWith:{
      _id: "$_id",
      timestamp: "$bucket_data.timestamp",
      temp: "$bucket_data.temp"
   }}
]);

This is adapted as follows:

 

milestone user index $sort $group accumulator
2 (done) {meta:1, ts:-1} {meta:1, control.max.timestamp:-1} $first
3 {meta:-1, ts:-1} {meta:-1, control.max.timestamp:-1} $first
3 {meta:1, ts:1} {meta:1, control.max.timestamp:1} $last
3 {meta:-1, ts:1} {meta:-1, control.max.timestamp:1} $last

 

This tickets covers the last two rows above.

 



 Comments   
Comment by Githook User [ 21/Mar/22 ]

Author:

{'name': 'Alya Berciu', 'email': 'alyacarina@gmail.com', 'username': 'alyacb'}

Message: SERVER-64235 Implement timeseries lastpoint rewrite for ascending time
Branch: master
https://github.com/mongodb/mongo/commit/3bdf25eb7321aac652127c6021dd8a3178d71d38

Comment by Alya Berciu [ 16/Mar/22 ]

Reverting/reopening this to resolve the following BFs: BF-24629 BF-24638 BF-24630

Comment by Githook User [ 16/Mar/22 ]

Author:

{'name': 'Alya Berciu', 'email': 'alyacarina@gmail.com', 'username': 'alyacb'}

Message: Revert "SERVER-64235 Implement timeseries lastpoint rewrite for ascending time"

This reverts commit 811dc428a93105d0aa9091ca20e4dfb120e4e57d.
Branch: master
https://github.com/mongodb/mongo/commit/e102dbba6cd1f5b99991089b90e841007bef16d7

Comment by Githook User [ 15/Mar/22 ]

Author:

{'name': 'Alya Berciu', 'email': 'alyacarina@gmail.com', 'username': 'alyacb'}

Message: SERVER-64235 Implement timeseries lastpoint rewrite for ascending time
Branch: master
https://github.com/mongodb/mongo/commit/811dc428a93105d0aa9091ca20e4dfb120e4e57d

Generated at Thu Feb 08 05:59:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.