Deleting an environment (AWS account)
Note: This process cannot be undone. All resources in an AWS account will be deleted.
You will need to complete the actions below in as short a time frame as possible. Certain workflows such as Terraform: New environment will re-create accounts/workspaces which you are trying to delete if it runs mid-way through your progress. Ideally warn the MP team not to merge any PRs whilst you carry out these actions.
This runbook describes the process required to delete an environment/s.
Running account deletion scripts locally
In modernisation-platform/scripts/account-deletion we have a set of bash scripts that are designed to automate the process of decommissioning an AWS account. The scripts perform the following tasks, in this order:
delete-tf-state.sh- Removes the AWS account from Modernisation Platform state managementupdate-files.sh- Removes the environment definition inenvironments/*.jsondelete-tf-resources- Deletes Terraform resources for the environment (in the MP and MP Environments repos)delete-tf-workspaces.sh- Deletes Terraform workspaces for the environment interraform/environments/bootstrap/*delete-files.sh- Removes Environment Files (only if removing all environments for an application)
They are all triggered in sequence by the delete-accounts.sh script.
Be aware that some resources such as s3 buckets cannot be destroyed until you manually empty all the objects and versions in them.
Ensure that you have fetched the most recent updates in your local MP and MPE directories by executing a
git pullcommand.Ensure that you have deleted all local
.terraformand.terraform.lock.hclfiles
To use the scripts, follow these steps:
- Navigate to the account-deletion folder and create a
config.txtfile within it - Open the example-config.txt file, copy its contents, and paste them into your newly created
config.txtfile. - Modify your
config.txtfile to include variables specific to your AWS account deletion task. - Open your terminal and ensure your current working directory is
modernisation-platform/scripts/account-deletion. - If required make the script executable by running the command:
chmod +x delete-accounts.sh - Execute the script using the command:
./delete-accounts.sh - While the script is running you will need to respond to prompts to confirm your actions at various stages
The script execution logs will be written to a file called
execution_log.txtwhich show the progress being made and help to debug any issues that may arise.If the script fails at any point you can try to re-run the
account-deletion.shscript or manually run a combination of the individual scripts that it calls using the order mentioned above.Where the script has made changes to files in the MP/MP Environments repos it will have created them on a new branch with a unique name. You will need to merge these in at a later stage of the process.
AWS Organizational Configuration
Note that a user with root account permissions will need to do this. Contact the MP teams TA/Lead Webops Engineer.
Move the account to the root organizational unit
This is to move the account out of an organizational unit where there is an SCP preventing any actions by the root user.
Log into the AWS Console and navigate to AWS Organizations, find the account and move it to the root OU.
Now the AWS account can be deleted.
Delete the actual AWS account
This process cannot be undone. Proceed carefully.
Navigate to the root OU and find the account, select the checkbox then go Action and choose Close.
Move the account to the closed accounts organizational unit
Ensure the account is selected and then move it to the Closed accounts OU.
Merge in your file changes
Raise a PR to merge in your file changes (created by the scripts above) in both the MP and MP Environments repositories which will trigger Terraform to perform the remaining clean up needed.
Check for any automated PRs that have been raised in the MP Environments repo to update the .github/CODEOWNERS file. Here is an example PR. These will need to be merged in.
Delete Github environments
Navigate to https://github.com/ministryofjustice/modernisation-platform-environments/settings/environments and delete the relevant GitHub environments.