[SERVER-60577] Add logic to check time-series buckets for mixed-schema data at index time Created: 08/Oct/21  Updated: 29/Oct/23  Resolved: 30/Oct/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.2.0

Type: Task Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
causes SERVER-69264 Fail time-series index builds due to ... Closed
Related
is related to SERVER-61070 Update time-series mixed-schema data ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2021-11-01, Execution Team 2021-11-15
Participants:

 Description   

Users that upgrade from earlier versions of the server may contain mixed-schema data in their time-series buckets. As long as the catalog flag that indicates whether a time-series bucket may contain mixed-schema data is set to true, we need to validate all the buckets when trying to index time-series measurements.

There are two possible outcomes:

  1. There are no buckets containing mixed-schema data. We set the catalog flag to false and subsequent index builds do not have to perform any validation on the buckets.
  2. There is at least one bucket containing mixed-schema data. The index build will fail in this case. We'll also provide a list of options for migrating the time-series buckets to a format that does not contain problematic mixed-schema data.


 Comments   
Comment by Githook User [ 30/Oct/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60577 Add logic to check time-series buckets for mixed-schema data at index time
Branch: master
https://github.com/mongodb/mongo/commit/8b405630fc1506abf620aaf4b08363d015327c9b

Comment by Githook User [ 30/Oct/21 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-60577 Add functionality to detect mixed-schema data in time-series bucket collections
Branch: master
https://github.com/mongodb/mongo/commit/2455e1c112c89a3bdde41d718cadf7f9cc9b5bf0

Comment by Dan Larkin-York [ 08/Oct/21 ]

As a brief note, it looks like the right place to plug in and add this logic isĀ here, prior to filtering. That said, we'll have to check that every insert path funnels through here, and there isn't some earlier filtering step in some cases.

Generated at Thu Feb 08 05:50:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.