[SERVER-85320] Investigate testing gap for binary upgrade Created: 17/Jan/24 Updated: 22/Jan/24 |
|
| Status: | Open |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Moustafa Maher | Assignee: | Backlog - Replication Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Replication
|
||||||||
| Participants: | |||||||||
| Description |
|
In most of our upgrade/downgrade tests, we perform FCV upgrades only on the latest binary. The tests start with the latest binary, then downgrade FCV, and later upgrade it again. However, this doesn't fully mimic the production scenario where we begin with a downgraded binary and FCV, then upgrade the binary, followed by an FCV upgrade. This ticket aims to investigate whether there is a gap in our testing approach and if it's necessary to adjust our testing to better align with production scenarios. So we do mostly test each individual state of this process, but not the whole process together, and not the specific step of swapping the binaries. We should investigate if it is actually worth it to test the whole process and/or the specific step of swapping the binaries. However, for targeted feature tests, each feature should still add tests to test the full upgrade (starting on downgraded binary -> swap to upgraded binary, do appropriate feature testing -> upgrade FCV, do appropriate feature testing). We currently have an automated ticket in each project that talks about FCV testing, but we could add language in this around full testing the binary upgrade and then FCV upgrade. We also have this example test that we could point to. In addition we could add more info in the README about this. |