[COMPASS-6702] Investigate changes in PM-2724: Improving Time-Series Scalability Created: 04/Apr/23  Updated: 19/Apr/23  Resolved: 19/Apr/23

Status: Closed
Project: Compass
Component/s: None
Affects Version/s: None
Fix Version/s: No version

Type: Investigation Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to COMPASS-6743 Add new time series options to create... Needs Triage
Documentation Changes: Not Needed

 Description   
Original Downstream Change Summary

There will be some required changes to documentation on a few subjects, specifically collection creation options and indexing options for time-series collections. We are planning to sync up with the docs team (Nick Villahermosa) when we complete the work for the new options and behaviors.

Description of Linked Ticket

Epic Summary

Summary

Add additional scalability features to better handle scalability challenges in Time Series collections and provide users with more flexibility options based on their use case such as: 

  • Bucket re-opening to ensure better dynamic bucketing and support more use cases such as high cardinality or high cardinality with asymmetric ingestion rates
  • Fixed time-range bucketing to provide users with more storage modeling flexibility and better index usage for use cases that rely on fixed period roll-ups and do not need handling for overlapping buckets/out-of-order data to avoid overlapping buckets
  • A command to support rewrites for merge/split/consolidation of buckets
  • Unique indexing to support duplicate data protection at the database layer

Motivation

Time Series data by nature is demanding and challenging. While almost all users have time series use cases, creating a solution that generally supports these demands, challenges, and use cases with the scalability needed both dynamically and at the users control is something that will allow us to support many more customers requirements and move closer to a full-replacement TSDB

Documentation

Scope Document
Technical Design Document
Docs Update


Generated at Wed Feb 07 22:44:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.