[SERVER-83410] Avoid quadratic behavior in File Copy Based Initial Sync with many collections Created: 17/Nov/23  Updated: 08/Jan/24  Resolved: 21/Nov/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Haley Connelly Assignee: Backlog - Catalog and Routing
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Duplicate
Related
Assigned Teams:
Catalog and Routing
Backport Requested:
v7.2, v7.0, v6.0
Participants:

 Description   

It's known that writes to the CollectionCatalog can be expensive, and resource consuming - see SERVER-58736SERVER-62618.

Since File Copy Based initial sync takes a different startup path than the one covered by SERVER-62618, we can make use of the batch writer functionality added to the CollectionCatalog by SERVER-58736 to avoid copying the full catalog each time we add a collection to it.



 Comments   
Comment by Josef Ahmad [ 21/Nov/23 ]

louis.williams@mongodb.com, henrik.edin@mongodb.com: I'm closing this ticket in favour of the "CollectionCatalog write performance improvements" backports to 6.0. Let me know if the backport gets parked and we'll reconsider this ticket.

Comment by Josef Ahmad [ 17/Nov/23 ]

As of 7.0, the PIT catalog lookup project has switched the CollectionCatalog to use immutable structures that are cheap to copy and should negate the need for the BatchCollectionCatalogWriter; see SERVER-81971. That said, FCBIS has been available since 6.0, which predates the immutable structures, so this ticket is primarily for 6.0 backporting purposes.

Generated at Thu Feb 08 06:52:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.