[SERVER-45094] Add passthrough tests for safe reconfig Created: 12/Dec/19 Updated: 29/Oct/23 Resolved: 17/Apr/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.0-rc6, 4.7.0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Siyuan Zhou | Assignee: | Pavithra Vetriselvan |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | safe-reconfig-testing | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Backport Requested: |
v4.4
|
||||||||
| Sprint: | Repl 2020-02-10, Repl 2020-02-24, Repl 2020-03-09, Repl 2020-03-23, Repl 2020-04-06, Repl 2020-04-20 | ||||||||
| Participants: | |||||||||
| Linked BF Score: | 25 | ||||||||
| Description |
|
The following passthrough test suites will be added:
We will start a 5-node replset with 3 nodes having vote: 1. The test suites periodically and randomly remove or add a node by changing the node’s vote so that the config has 1-5 voting nodes. replica_set_reconfig_kill_nodes_jscore_passthrough kills or restarts a node randomly to cover unclean shutdown and durability issues. The shutdown hook and reconfig hook will make sure a majority of nodes are alive at any time. These suites should be run with readConcern: majority, writeConcern: majority, causal consistency, readPreference: primary, and retryable writes so that reconfig does affect the operations. |
| Comments |
| Comment by Githook User [ 14/May/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message:
(cherry picked from commit f59f63db6c37c0d4657b57d559c95d830b0e34c2)
(cherry picked from commit 4d91fac171cbe3f2af53d9258965399e648a1947)
(cherry picked from commit a43cb23defc6182d08a7814e4731ef98f2d30b6a)
(cherry picked from commit 81e0ad27c280c02a49beb65ff4473d5dce62b089)
(cherry picked from commit 2debab7987b24bf902f9a128654ce928441c29a2)
(cherry picked from commit 91672e58f1169c7edd684b911f20f62b8a71f8d1)
(cherry picked from commit 81d53a715f49827a9f2538d4572f9b01f2b12887) |
| Comment by Pavithra Vetriselvan [ 17/Apr/20 ] |
|
siyuan.zhou For bookkeeping, we will need to backport this ticket along with |
| Comment by Githook User [ 17/Apr/20 ] |
|
Author: {'name': 'Siyuan Zhou', 'email': 'siyuan.zhou@mongodb.com', 'username': 'visualzhou'}Message: This reverts commit 0e7b476357a12802f1be1ca5c8a4b1a919000ef8. |
| Comment by Githook User [ 14/Apr/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: |
| Comment by Githook User [ 02/Apr/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: |
| Comment by Githook User [ 02/Apr/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: |
| Comment by Githook User [ 02/Apr/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: |
| Comment by Githook User [ 01/Apr/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: Revert " This reverts commit 49836a791fbab2c8f3726450cda1d3c708eff90a. |
| Comment by Githook User [ 01/Apr/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: |
| Comment by Githook User [ 30/Mar/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'email': 'pvselvan@umich.edu', 'username': 'pvselvan'}Message: |
| Comment by Githook User [ 17/Mar/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'username': 'pvselvan', 'email': 'pvselvan@umich.edu'}Message: |
| Comment by Githook User [ 16/Mar/20 ] |
|
Author: {'name': 'Pavi Vetriselvan', 'username': 'pvselvan', 'email': 'pvselvan@umich.edu'}Message: |
| Comment by Siyuan Zhou [ 30/Jan/20 ] |
|
Sounds good to me! Nice investigation. |
| Comment by Pavithra Vetriselvan [ 30/Jan/20 ] |
|
Talked to robert.guo and jason.chan about this. I think the hardest part will be adding the first suite, replica_set_reconfig_jscore_passthrough since that's where we'll want to figure out how the suite decides to reconfig a set of nodes. Stepdown and shutdown hooks already exist and can be adapted to the next two suites. From my understanding, we would like to start this suite with 5 nodes (all with votes: 1) and then use a background hook to incrementally change the votes of these nodes to 0 through reconfigs. We should only be trying to run reconfigs on sets of voting nodes. siyuan.zhou william.schultz, does that sound correct to you? |