[DOCS-15977] [Server] Document sharding auto-merger Created: 22/Mar/23  Updated: 13/Nov/23  Resolved: 08/Jun/23

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 7.0.0-rc0, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Joseph Dougherty
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
Issue split
Related
related to DOCS-16150 Release Notes / Compat Changes for au... Closed
related to DOCS-16164 Concept page for Auto-Merger Closed
is related to DOCS-15976 [Server] Document `mergeAllChunksOnSh... Closed
Participants:
Days since reply: 46 weeks ago
Epic Link: DOCSP-26284

 Description   

When a migration commits, the current timestamp is always stored together with the migrated chunk. Starting from v7.0, sharding has a new component called auto-merger that is periodically taking care of scanning through such timestamps in order to spot "mergeable" chunks and then issue mergeAllChunksOnShard commands (both the definition of "mergeable" and of the new mergeAllChunksOnShard command are listed in DOCS-15976).

When the auto-merger is active, it does the following:

  1. Identify all the <nss, shard> pairs for which exist mergeable chunks.
  2. Schedule a mergeAllChunksOnShard command for each pair and wait for the operation to commit (with maxNumberOfChunksToMerge set to 1000).

New server parameters (to set on config server nodes)

Unless explicitly disabled, auto-merging will happen periodically every autoMergerIntervalSecs seconds, configurable server parameter defaulted to 1 hour. Auto-merge actions issued for the same collection will be spaced at least of autoMergerThrottlingMS seconds defaulted to 15 seconds (please, also document those new parameters).

Settings

Which parameter(s) are taken into account to decide whether auto-merging should happen for a collection? 
The following settings will be considered in order of precedence

If a balancing window is set, the auto-merger will only work during such window.
 


Generated at Thu Feb 08 08:14:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.