[DRIVERS-1449] Investigate changes in PM-1897: Faster Initial Sync Method Created: 20/Nov/20 Updated: 27/May/22 Resolved: 20/Nov/20 |
|
| Status: | Closed |
| Project: | Drivers |
| Component/s: | None |
| Fix Version/s: | None |
| Type: | Epic | Priority: | Major - P3 |
| Reporter: | Backlog - Core Eng Program Management Team | Assignee: | Unassigned |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Driver Changes: | Needed | ||||
| Server Compat: | 5.2 | ||||
| Upstream Changes Summary: | This project adds to the Enterprise Server a new method of adding a node to a replica set, or resyncing a node in a replica set. It is supported only on nodes which are running durable (not inMemory) WiredTiger as the storage engine, and the new node and an existing node (which it syncs from) must have the same WiredTiger configuration in terms of whether they are using the Encrypted Storage Engine, their storage.directoryPerDb setting, and their storage.directoryForIndexes setting. It will not be the default initial sync method on Enterprise Server. There will be a new parameter, initialSyncMethod, which can be set to "fileCopyBased" to use file copy base initial sync. We expect this new method to be considerably faster, but at least for now it is somewhat more limited. In addition to the storage engine limitations, only one node may be initial syncing (with this new method) from a given existing node at a time. That existing node cannot be undergoing a backup at a time, nor can a backup start on a node which is being synced from. Also possibly relevant for security, unlike logical initial sync, file copy based initial sync with encrypted storage engine cannot be used to re-key the data. |
||||
| Description |
|
Downstream Change Summary Preemptively checking off Drivers Attention for this project for the team to review and be aware of. Description of Linked TicketEpic Summary SummaryAdd an option to perform initial sync using file copies. MotivationInitial sync takes a long time. It does a logical copy, which will always be slower than solutions that do a physical copy. When doing initial sync, most users care the most about the speed of the initial sync instead of other benefits received from logical copy, such as compaction. Users don’t always have a backup to restore from to do initial sync. We don’t officially support tools/processes for initial sync using iterative rsync or backup cursors. By building this functionality directly into the server, we’ll make this the option of choice for users that don’t need to sync across storage engines or across versions. Cast of Characters
DocumentationScope Document |