[SERVER-62065] Upgrade path from 3.6 to 4.0 can leave chunk entries without history on the shards Created: 15/Dec/21 Updated: 29/Oct/23 Resolved: 05/Jan/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 4.0.27, 4.2.17, 4.4.10, 5.0.5, 5.1.1, 5.2.0-rc1 |
| Fix Version/s: | 5.3.0, 5.0.6, 4.2.19, 4.0.29, 5.2.1, 4.4.13 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kaloian Manassiev | Assignee: | Kaloian Manassiev |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||
| Backport Requested: |
v5.2, v5.1, v5.0, v4.4, v4.2, v4.0
|
||||||||||||||||||||||||||||
| Sprint: | Sharding EMEA 2021-12-27, Sharding EMEA 2022-01-10 | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||
| Linked BF Score: | 184 | ||||||||||||||||||||||||||||
| Description |
|
In 4.0 we introduced support for a multi-version routing table (PM-1013). This later (in 4.2) became the basis for distributed transactions and snapshot reads. As part of this project, we introduced a history field to the persisted chunk type, but we never actually used it in 4.0. Since we never used it, the backup/restore procedures ever since 4.0 have referenced that this field is safe do delete on restore. However, it is actually not safe to do so, because it breaks snapshot reads (routing and filtering at a point in time). Furthermore, due to the optimisation done under As a result of the above, we can have 4.0, 4.2, 4.4, 5.0, 5.1, 5.2 clusters which are missing the history fields for some chunks, which in turn breaks snapshot reads and distributed transactions, which will fail with an error saying Chunk has no history entries. These clusters will not have any issue functioning, until customers start using distributed transactions and snapshot reads. This ticket is to provide manual procedure for restoring the history fields and to implement a command, which will restore the history fields automatically. |
| Comments |
| Comment by Githook User [ 23/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: Make the CPP name of '_flushRoutingTableCacheUpdates' more user-friendly: Remaining: |
| Comment by Sviatlana Zuiko [ 21/Jan/22 ] |
|
Requesting a backport to 5.2 |
| Comment by Githook User [ 19/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: |
| Comment by Githook User [ 12/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: (cherry picked from commit 508f8dd9dd4aa27f15b327c84d5160146ffa8724) |
| Comment by Githook User [ 11/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: |
| Comment by Githook User [ 10/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: (cherry picked from commit 146c18b9954abc116046621dc4849cc7d97ef523) |
| Comment by Githook User [ 09/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: |
| Comment by Githook User [ 07/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: (cherry picked from commit 3b56acfe78e91b607eafc737ebf88d237db1460a) |
| Comment by Githook User [ 06/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: (cherry picked from commit 3b56acfe78e91b607eafc737ebf88d237db1460a) |
| Comment by Githook User [ 05/Jan/22 ] |
|
Author: {'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}Message: |