Terraform Remote State Manager(tfremote)
tf is a python package for managing terraform remote state for: Google(Gcloud), AWS, and Azure. It sets a defined structure for all cloud providers by removing the overheard of configuring and managing the path in storage buckets.
It works with:
❗️ Note Best practice is to make sure buckets are versioned.
Install package
pip install tfremote --upgrade
Environment setup
-
Install Python 3.6+
-
Using virtualenv is strongly recommended:
python3 -m venv <venv name>
- Terraform 0.14.0 and above (download: https://www.terraform.io/downloads.html)
Default log level is WARNING, to change:
export TF_LOG_LEVEL to any of these: 'CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG'
❗️ Important - Two variables are required for usingtfpackage (used set creat path in remote storage):
- teamid
- prjid
Required variables can be defined using:
- As
inline variablese.g.:-var='teamid=demo-team' -var='prjid=demo-project'- Inside
.tfvarsfile e.g.:-var-file=<tfvars file location>Two optional variables:
workspaceandstate_keycan be defined using:
-w=<workspace_name>. If no workspace is provideddefaultworkspace is used.
s=<state_key name>. If no key is providedterraformis used.Path created in S3 backend:
/<teamid>/<prjid>/<workspace>/<state-key>.tfstateFor more information refer to Terraform documentation
Setup environment variables
Workspace list file location TF_WORKSPACE_FILE_LOCATION
export TF_WORKSPACE_FILE_LOCATION=<workspace yml file location>
Reference file: link
AWS
❗️ Important - s3 bucket for remote state should reside inus-west-2
Set these env variables:
export TF_AWS_BUCKET=<your_remote_state_bucket_name>
export TF_AWS_BUCKET_REGION=us-west-2
One of below environment variable is required:
export TF_AWS_PROFILE=<aws profile to use>
or
export AWS_ACCESS_KEY_ID=<aws access key>
export AWS_SECRET_ACCESS_KEY=<aws secret access key>
Azure
To create storage for remote state there is handy script.
Run scripts/remote_state.sh (fill in the required information)
Set below env variables:
export TF_AZURE_STORAGE_ACCOUNT=<remote state storage account name>
export TF_AZURE_CONTAINER=<remote state container>
export ARM_ACCESS_KEY=<storage account access key>
GCP(gcloud)
https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform
Set below env variables:
export TF_GCLOUD_BUCKET=<remote state storage bucket name>
export TF_GCLOUD_CREDENTIALS=json credentials file path>
Usage
For GCP(gcloud):
tf plan -c=gcloud -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace
The structure in Google Storage Bucket:
For AWS:
tf plan -c=aws -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace
The structure in AWS S3:
If you need to specify state_key in S3, specify -s=tryme-key
For Azure:
tf plan -c=azure -var=teamid=demo-team -var=prjid=demo-app -w=demo-workspace
The structure in Azure Storage:
For more available options:
tf --help
usage: tf [-h] [-var] [-var-file] [-c] [-w] [-wp] [-s] [-no-color] [-json] [-out] [-f] [-nf] [-v]
Terraform remote state wrapper package
--------------------------------------
Usage: Set below env variables to begin (more information: https://github.com/tomarv2/tfremote):
TF_WORKSPACE_FILE_LOCATION
aws: TF_AWS_BUCKET, TF_AWS_BUCKET_REGION=us-west-2, TF_AWS_PROFILE or AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
azure: TF_AZURE_STORAGE_ACCOUNT, TF_AZURE_CONTAINER, ARM_ACCESS_KEY
gcloud: TF_GCLOUD_BUCKET, TF_GCLOUD_CREDENTIALS
optional arguments:
-h, --help show this help message and exit
-var Set Terraform configuration variable. This flag can be set multiple times
-var-file Set Terraform configuration variables from a file. This flag can be set multiple times
-c Specify cloud provider (default: 'aws'). Supported values: gcloud, aws, or azure
-w Specify existing workspace name(default: 'default')
-wp Overwrite workspace directory path structure
-s File name in remote state (default: 'terraform.tfstate')
-no-color Disables terminal formatting sequences in the output
-json Enables the machine readable JSON UI output
-out Writes the generated plan to the given filename in an opaque file format
-f Enable FIPS endpoints (default: True)
-nf Disable FIPS endpoints
-v show program's version number and exit
21 Jul 25, 2022
1 Jan 24, 2022
6 Jun 4, 2022
311 Dec 21, 2022
8 Sep 20, 2022
6 Mar 14, 2022
28 Dec 29, 2022
37 Dec 28, 2022
2 Aug 18, 2022
1 Jan 7, 2022
8 Nov 22, 2022
71 Dec 29, 2022
1 May 20, 2022
1 Nov 3, 2021
144 Dec 16, 2022
45.3k Jan 2, 2023