[SERVER-69952] Filtering on dates outside of the 32 bit epoch range returns wrong results Created: 23/Sep/22  Updated: 29/Oct/23  Resolved: 01/Mar/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0, 5.0.16, 6.0.6, 6.3.2

Type: Bug Priority: Major - P3
Reporter: Joel Redman (Inactive) Assignee: David Percy
Resolution: Fixed Votes: 0
Labels: quick-tech-debt, time-series
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-72428 filter/$match not working on 1970-01-... Closed
Duplicate
is duplicated by SERVER-72428 filter/$match not working on 1970-01-... Closed
Problem/Incident
Related
related to SERVER-66469 Filtering timeseries with date-field ... Closed
Assigned Teams:
Query Optimization
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.3, v6.0, v5.0
Steps To Reproduce:

db.createCollection(timeColl, {timeseries: {timeField: time}});
 
// Note that all dates are within the epoch, i.e. between the years 1970 and 2038
const dates = [

{[timeFieldName]: new Date("1971-01-01")}

,

{[timeFieldName]: new Date("1975-01-01")}

,

{[timeFieldName]: new Date("1980-01-01")}

,

{[timeFieldName]: new Date("1995-01-01")}

,

{[timeFieldName]: new Date("2030-01-01")}

];
db.timeColl.insert(dates);
 
// 1920 falls outside the epoch, triggering the failure.
db.timeColl.find({time: {$gt:newDate("1920-01-01"), $lt:newDate("1980-01-01")}};
 
/* Should result in 
[

{time: new Date("1965-01-01")}

,

{time: new Date("1975-01-01")}

,
]);
but does not
*/

Sprint: QO 2023-02-06, QO 2023-02-20, QO 2023-03-06
Participants:
Linked BF Score: 166

 Description   

Literals outside the 32 bit epoch are currently compared to _id, which has already been trimmed to 32 bits,  so we cannot accurately match against them. This obvious fix is to determine whether the compared dates can be outside the epoch range and not relying on _id in that case.



 Comments   
Comment by Githook User [ 19/Apr/23 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-69952 Prevent timeseries from using _id when out of range
Branch: v6.3
https://github.com/mongodb/mongo/commit/d899c45380094adfc2e6f95f5eadf6831673a7a9

Comment by Githook User [ 22/Mar/23 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-69952 Prevent timeseries from using _id when out of range
Branch: v6.0
https://github.com/mongodb/mongo/commit/91d0883d7c4a5bee8b093e94e455b10ad0119479

Comment by Githook User [ 22/Mar/23 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-69952 Prevent timeseries from using _id when out of range
Branch: v5.0
https://github.com/mongodb/mongo/commit/43d4dc54f9e317cd8661eafe399e7902b88a16e0

Comment by Githook User [ 01/Mar/23 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-69952 Prevent timeseries from using _id when out of range
Branch: master
https://github.com/mongodb/mongo/commit/c80ca4373ac767037563691c9dd69f57258f2756

Comment by Githook User [ 02/Dec/22 ]

Author:

{'name': 'dziana.hurynovich', 'email': 'dziana.hurynovich@mongodb.com'}

Message: Revert "SERVER-69952 Prevent timeseries from using _id when out of range"

This reverts commit 362f7088aabd9e6f49e5f99525a0296723b3a9c3.
Branch: master
https://github.com/mongodb/mongo/commit/a76ea25ab405ae267b1e9b5a247465214374f7e0

Comment by Githook User [ 01/Dec/22 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-69952 Prevent timeseries from using _id when out of range
Branch: master
https://github.com/mongodb/mongo/commit/362f7088aabd9e6f49e5f99525a0296723b3a9c3

Generated at Thu Feb 08 06:14:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.