[SERVER-63235] Test control.max.time rounding's effect on DISTINCT_SCAN for TS Last Point Created: 02/Feb/22  Updated: 16/Feb/22  Resolved: 16/Feb/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Steve Tarzia Assignee: Steve Tarzia
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

james.wahlin left this comment on the technical design doc:

I know we round the value stored in bucket _id to second granularity. We should make sure we don't also round control.min.time and control.max.time. If we do then it is possible we could have 2 buckets starting at the same time with no guarantee that the first one we look at has the most recent or oldest point.

If there is indeed rounding being done on control.max.time then the DISTINCT_SCAN may return the one of several "equal-max-time" buckets which actually has a slightly older value than the true last.



 Comments   
Comment by Steve Tarzia [ 16/Feb/22 ]

A quick test on my benchmark database showed that we're storing control.max.timestamp with millisecond precision, not rounding to the second:

MongoDB Enterprise > db.system.buckets.telemetry.aggregate([{$sort:{"control.max.timestamp":1}}, {$limit: 1}, {$project: {"control":1}}]).pretty()
{
	"_id" : ObjectId("000000005db99da6e1198483"),
	"control" : {
		"version" : 2,
		"min" : {
			"_id" : ObjectId("620d8a0ea02f9f771425509d"),
			"timestamp" : ISODate("1970-01-01T00:00:00Z"),
			"temp" : 0
		},
		"max" : {
			"_id" : ObjectId("620d8a4aa02f9f771426b82e"),
			"timestamp" : ISODate("1970-01-01T00:00:09.208Z"),
			"temp" : 98.1
		},
		"count" : 1000
	}
}

So, the

{$sort: {"meta.sensorId": 1, "control.max.timestamp": -1}},
{$group: { _id: "$meta.sensorId",
          bucket: {$first: "$_id"}, }}

DISTINCT_SCAN optimization should return the a bucket with a last point within ISODate's millisecond accuracy.

Generated at Thu Feb 08 05:57:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.