[SERVER-75251] Update the FCV constant in the logkeeper snapshot workload for master Created: 24/Mar/23  Updated: 29/Oct/23  Resolved: 17/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Britt Snyman Assignee: Ahmad Shah
Resolution: Fixed Votes: 0
Labels: before-branch, branch-7.0
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2023-05-12-09-21-52-556.png     PNG File image-2023-05-12-09-24-53-433.png    
Issue Links:
Depends
Related
related to SERVER-77194 Update the FCV constant in the log ke... Closed
Assigned Teams:
Product Performance
Backwards Compatibility: Fully Compatible
Participants:
Linked BF Score: 120

 Description   
  • Use instructions in test_control.initialsync-logkeeper-snapshot-update.yml.
  • use evergreen patch -p sys-perf to submit patch
  • run initialsync-logkeeper-snapshot-update and initialsync-logkeeper-short tasks
  • need to copy the new data set to dsi-do-not remove s3 bucket
  • update the snapshotId, and the mongodb_dataset names.
  • backport to 7.0
  • ignore the doc at the end of the dsi file

Reach out in the #performance-tooling-users channel with any questions

 

As a result of this work, I have created new documentation to cover the process of updating the FCV constant: https://wiki.corp.mongodb.com/display/STAR/Update+InitialSync+LogKeeper+Snapshot



 Comments   
Comment by Ahmad Shah [ 17/May/23 ]

Standup:

Comment by Githook User [ 17/May/23 ]

Author:

{'name': 'Ahmad Shah', 'email': 'ahmad.shah@mongodb.com', 'username': 'MAhmadShah'}

Message: SERVER-75251 - Update snapshot for initial sync variant
Branch: master
https://github.com/mongodb/mongo/commit/8219766da20351edcda5fe21a18da254b382cd35

Comment by Ahmad Shah [ 16/May/23 ]

Standup:

  • Have a new snapshot with the FCV 7.0 successfully created, ran the initialsync-logkeeper task with the latest commit from the waterfall successfully
  • Results are in this patch: https://spruce.mongodb.com/version/6462b0e0a4cf4783d776bb27/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC
    • Just waiting for the initialsync-logkeeper-fcbis task to finish running before I send PR for approval
  • The snapshot_update workload for some reason did not work, it was uploading a snapshot with the FCV set to 6.3 still, even though the description stated it was 7.0
  • Will document the workflow I used to get this to work and see if I can fix the snapshot_update workload. This document was very helpful.
    • Had to spin up an instance using the last mongo repo commit where these tasks were working
    • SSH'd into the mongod.0 instance (had the seeded EBS device on it with FCV 6.3) and started the mongod whilst pointing to the workdir/dbs source folder
    • Connected to the mongod and updated the FCV to 7.0 via the mongoshell
    • Created a volume using the device storage and then a snapshot from that volume
  • Created PERF-4169 to investigate and fix the workload for uploading the snapshot
Comment by Ahmad Shah [ 15/May/23 ]

Standup:

  • Struggled with this one on Friday for a bit, but think I understand whats going on here
    • Testing is super slow since the snapshot takes ~2 hours to download during provisioning
  • Tried to load the snapshot onto a standalone instance so I could check the FCV version, realized that there was an issue with with auth being disabled for the initial sync tests that wasn't behaving well with the standalone since the mongodb_setup was failing
  • Retrying this with the initial-sync variant and seeing if I can ssh into the mongod.0 instance and connect to the mongo shell to check which FCV is loaded
  • Question: Once the snapshot is loaded onto the EC2 instance, is there a way I can check the FCV without having to use the mongoshell? The mongo logs from the failures didn't reveal anything
Comment by Kelsey Schubert [ 10/May/23 ]

reassigning to the performance team to help clarify the procedure since it's seems not trivial and the release team doesn't have the capacity or expertise to quickly move forward with it.

We should separately consider who should be the long term owner of this task and how it should fit into branching.

Generated at Thu Feb 08 06:29:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.