[SERVER-55337] Use cursors for index writes at the SortedDataInterface/IndexCatalog layer Created: 19/Mar/21  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Index Maintenance, Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: Backlog - Storage Execution Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-55338 Propagate bulk inserts from Collectio... Backlog
related to SERVER-55341 WiredTigerRecordStore should reserve ... Closed
Assigned Teams:
Storage Execution
Participants:

 Description   

Right now all of the SDI write ops are one-shot operations. This makes it difficult for storage engines such as wiredtiger that are able to reduce duplicated work when a cursor is used to insert many kv pairs into the same tree. While this doesn't matter much for simple indexes that may only have 1 or a handful of entries per document per index, it can be a real problem for wildcard where there can be hundreds of entrees per document per index.

This change would also stack nicely with propagating bulk insertion from the Collection layer to the IndexCatalog layer (SERVER-55338), since it will increase the probability of inserting keys that are close together, taking advantage of less btree lookup.


Generated at Thu Feb 08 05:36:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.