Follow

Follow

How to use non-AWS S3 Compatible storage for Terraform Backend?

Suraj Narwade's photo
Suraj Narwade
·Feb 18, 2023·

2 min read

Play this article

As we know, Terraform supports S3 as a backend to store the state in AWS. In GCS and Azure, there are equivalent solutions for object storage available.

What if we want to store terraform state in our environment or on any other cloud provider?

The good news is that we can do that, as all we need is S3-compliant storage.

An example can be:

For this example, we will use minio:

  • Let's run minio in a docker container locally

  •   mkdir -p ${HOME}/minio/data
    
      docker run \
         -p 9000:9000 \
         -p 9090:9090 \
         --user $(id -u):$(id -g) \
         --name minio1 \
         -e "MINIO_ROOT_USER=ADMIN" \
         -e "MINIO_ROOT_PASSWORD=PASSWORD" \
         -v ${HOME}/minio/data:/data \
         quay.io/minio/minio server /data --console-address ":9090"
    
  • Once it is running, go to http://127.0.0.1:9090 and log in using the credentials

  • create access key http://127.0.0.1:9090/access-keys

  • Once you have the access keys, you can now define the backend as shown below:

terraform {
  backend "s3" {
    bucket                      = "state"
    key                         = "terraform.tfstate"
    region                      = "myregion"
    endpoint                    = "http://127.0.0.1:9090"
    skip_region_validation      = true
    skip_credentials_validation = true
    skip_metadata_api_check     = true
    force_path_style            = true
  }
}

What's happening here?

  • Setting skip_credentials_validation to true will disables validation of the credentials when communicating with the S3 service. This is needed as we are using non-AWS S3-compatible storage.

  • Setting skip_metadata_api_check to true will disable the metadata API check (specific to AWS) when communicating with the S3 service.

  • With force_path_style we are telling terraform to use a path style for the URL

  • Setting skip_region_validation to true will disable region validation as it is again specific to AWS.

You can also export credentials in the environment variables as shown instead of keeping them in the code

export AWS_S3_ENDPOINT=<minio-url>
export AWS_ACCESS_KEY_ID=<access-key>
export AWS_SECRET_ACCESS_KEY=<secret-key>

That's it. You are now ready to use non-AWS S3 compliant Storage for Terraform backend and do,

terraform init

I hope you learned something new from this blog post. Click here to learn about me and how you can support my work, Thank you.

Did you find this article valuable?

Support Suraj Narwade by becoming a sponsor. Any amount is appreciated!

See recent sponsors Learn more about Hashnode Sponsors
 
Share this