[SERVER-68000] Optimize index used to support query-based reopening Created: 12/Jul/22  Updated: 21/Mar/23  Resolved: 14/Dec/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Dan Larkin-York Assignee: Dan Larkin-York
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-66689 Build index on meta, time by default ... Closed
Related
Sprint: Execution Team 2022-11-28, Execution Team 2022-12-26
Participants:
Linked BF Score: 5

 Description   

The index we build by default to support query-based reopening can lead to about a 10% decrease in throughput for bulk insert workloads, and up to 30% increase in median latency for individual inserts. We can probably improve this by building the index directly on the system.buckets collection over {meta: 1, control.min.time: 1} rather than using the standard rewrite ({meta: 1, control.min.time: 1, control.max.time: 1}) to decrease the frequency of index key updates.

Additionally, we could potentially skip calculating index keys if we know the underlying fields haven't changed.



 Comments   
Comment by Dan Larkin-York [ 14/Dec/22 ]

Using the additional index as-is enables a number of query optimizations that may go away if we were to change the index definition. Additionally, calculating the index keys has at most a modest effect on the write throughput for time series collections, so we will not be investigating any special-cased optimizations at this time.

Generated at Thu Feb 08 06:09:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.