[SERVER-77888] Timeseries last-point query is incorrect for pre-1970 dates Created: 07/Jun/23  Updated: 31/Jan/24

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: David Percy Assignee: Nishith Atreya
Resolution: Unresolved Votes: 0
Labels: query-product-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-67825 Investigate where time-series functio... Closed
is related to SERVER-67816 Time-series bucket rounding for pre-1... Open
is related to SERVER-77889 Timeseries sharded on time can route ... Backlog
Assigned Teams:
Query Integration
Operating System: ALL
Participants:

 Description   

Timeseries "last point" queries are those that pick out the most recent event in each series. For example:

{$group: {
  _id: "$meta.sensor_id",
  last_time: {$top: {sortBy: {time: 1}, output: "$time"}},
  last_value: {$top: {sortBy: {time: 1}, output: "$value"}},
}}

We rewrite the query to pick out the latest bucket within each series, before unpacking. This assumes that the latest bucket contains the latest event, which is true when control.max.time is a tight upper bound.

However, for events before 1970 control.max.time is not a tight bound:

> db.createCollection('ts', {timeseries:{timeField:'t',metaField:'m'}})
{ "ok" : 1 }
> db.ts.insert({t: ISODate('1969-01-01T00:00:12.345')})
WriteResult({ "nInserted" : 1 })
> db.system.buckets.ts.find().pretty()
{
        "_id" : ObjectId("fe1eccbc6f22cc96f1dd557f"),
        "control" : {
                "min" : {
                        // Tight lower bound, not rounded.
                        "t" : ISODate("1969-01-01T00:00:12.345Z")
                },
                "max" : {
                        // Rounded upper bound, not tight.
                        "t" : ISODate("1969-01-01T00:01:00Z")
                }
        },
        "data" : ...
}

This means if a series contains only pre-1970 dates, the query can return a wrong event.

A simple / coarse way to fix this would be to disable this rewrite when the collection contains extended-range dates.


Generated at Thu Feb 08 06:36:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.