[SERVER-78080] Use fully random bytes for 8 low-order bytes of OID value for time series bucket _id Created: 14/Jun/23  Updated: 29/Oct/23  Resolved: 15/Jun/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 6.3.2, 6.0.7, 5.0.19, 7.0.0-rc5

Type: Improvement Priority: Major - P3
Reporter: Dan Larkin-York Assignee: Dan Larkin-York
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-78475 randomized_mixed_type_bug.js should c... Closed
Problem/Incident
Related
related to SERVER-79397 Fix and test logic to internally retr... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.0, v6.3, v6.0, v5.0
Sprint: Execution NAMR Team 2023-06-26
Participants:
Case:
Linked BF Score: 124

 Description   

The current implementation for time series bucket _id generation follows this process:

  1. Generate an OID.
  2. Replace the 4 high-order timestamp bytes of the OID with a rounded version of the user-specified timestamp from the first measurement that lands in the bucket.
  3. Add the difference between the original timestamp and the rounded version to the next five bytes of the OID (the cluster-unique bytes).

This process was devised originally to deal with coarse granularity in SERVER-61194, but is still susceptible to collisions when many buckets are opened using the exact same initial timestamp.

In order to further reduce the chances of collisions in these cases, we should simply generate the OID from scratch by filling in the 4 high-order with the rounded timestamp as we currently do, then fill in the remaining 8 low-order bytes with a randomly generated number.



 Comments   
Comment by Githook User [ 22/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Use fully random bytes for 8 low-order bytes of OID value for time series bucket _id

(cherry picked from commit 26a4bfd5bd61665498a6d17e69e861233e1d2d16)
Branch: v6.3
https://github.com/mongodb/mongo/commit/6480c400db55bab75b8ce0f05125895a885b038a

Comment by Githook User [ 20/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Make randomized_mixed_type_bug.js test more robust

(cherry picked from commit baff036b89dd909609139a6771ae1c8248593687)
Branch: v7.0
https://github.com/mongodb/mongo/commit/d65a83d105d441c7a03f3c01d58028ac74dab069

Comment by Githook User [ 20/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Make randomized_mixed_type_bug.js test more robust
Branch: master
https://github.com/mongodb/mongo/commit/baff036b89dd909609139a6771ae1c8248593687

Comment by Githook User [ 16/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Use fully random bytes for 8 low-order bytes of OID value for time series bucket _id

(cherry picked from commit 26a4bfd5bd61665498a6d17e69e861233e1d2d16)
Branch: v5.0
https://github.com/mongodb/mongo/commit/6b2b5dda7ee3ccde1a285c7168e573e44b0c4056

Comment by Githook User [ 16/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Use fully random bytes for 8 low-order bytes of OID value for time series bucket _id

(cherry picked from commit 26a4bfd5bd61665498a6d17e69e861233e1d2d16)
Branch: v7.0
https://github.com/mongodb/mongo/commit/ae6a2a8aa4dbf8682e601f25cdf0ff113d897750

Comment by Githook User [ 16/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Use fully random bytes for 8 low-order bytes of OID value for time series bucket _id

(cherry picked from commit 26a4bfd5bd61665498a6d17e69e861233e1d2d16)
Branch: v6.0
https://github.com/mongodb/mongo/commit/5b37e99faa6253957d7d28f25a381ea239bfe376

Comment by Githook User [ 15/Jun/23 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-78080 Use fully random bytes for 8 low-order bytes of OID value for time series bucket _id
Branch: master
https://github.com/mongodb/mongo/commit/26a4bfd5bd61665498a6d17e69e861233e1d2d16

Generated at Thu Feb 08 06:37:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.