-
Type:
Task
-
Resolution: Fixed
-
Priority:
Unknown
-
None
-
Component/s: Evergreen Tools
-
None
-
Not Needed
Summary
Add a periodic task in drivers-evergreen-tools to remove old Azure resources.
Motivation
Azure resources are created for testing CSFLE and OIDC. Periodically, alerts get triggered due to leftover Azure resources created for testing.
Evergreen task groups for Azure clean-up resources in teardown.
However, this appears not not reliably work. For example, this Rust OIDC task logs:
[2025/02/24 18:29:43.330] Heartbeat received signal to abort task.
I expect the task was aborted before storing the VM name as an expansion:
/data/mci/5f3f691da9421fc38ad15c86db00c757/src/../<REDACTED:DRIVERS_ATLAS_LAMBDA_USER>-tools/.evergreen/auth_oidc/azure/delete-vm.sh: line 11: AZUREOIDC_VMNAME: unbound variable
Note: GCP resources already have a built-in expiration
Who is the affected end user?
DBX
How does this affect the end user?
Higher cost.
How likely is it that this problem or use case will occur?
Very likely. An alert was triggered for exceeding 15 active VMs.
If the problem does occur, what are the consequences and how severe are they?
Higher cost for Azure. With current alert, maybe $400 extra a month.
Is this issue urgent?
Yes? Azure resources were manually deleted, but I expect this can happen again.
Is this ticket required by a downstream team?
No.
Is this ticket only for tests?
Yes.
Acceptance Criteria
Delete old (1-2 hours?) Azure resources periodically.