19th November, 2021 –

Here’s how to create S3 Bucket and DynamoDB Table for Terraform backend in a multi-account AWS environment.

I’m working with a customer who has deployed a multitude of AWS Accounts in their AWS Organisation. They have arranged the AWS accounts in multiple Organisational Units. They have also leveraged the AWS Control Tower to easily set up the governance, security and best practices across the AWS Organisation.

The customer has also adopted Terraform as the preferred Infrastructure as Code (IAC) tool, using a separate AWS S3 Bucket and a DynamoDb Table. Using the Terraform backend for each AWS Account is one of the community best practices.

This created the need to have a fully automated solution in place to deploy an AWS S3 Bucket and a DynamoDB Table in each of the existing and future AWS Accounts with a standard configuration, so that the teams across the organisation can use the same AWS S3 Bucket and the DynamoDB Table for the Terraform State management of their application stack environments.

In order to solve this problem, we decided to write a CloudFormation Template that will deploy a S3 Bucket and DynamoDB Table in every AWS Account using the CloudFormation StackSets – which lets you provision resources across multiple AWS Accounts and Regions.

AWS Control Tower deployment

When you deploy the AWS Control Tower service across the AWS Organisation, it creates AWSControlTowerExecution IAM Role in every child account enrolled and uses the AWSControlTowerStackSetRole IAM Role in the Management AWS Account to deploy stack sets in all the AWS accounts created by AWS Control Tower.CloudFormation Template – Click to expand!

9 steps: Deploying the CloudFormation Template

  1. Log in to the AWS Management accounts (Root Account) console and go to the AWS Organisation service page and make a copy of the Organisational Units id in which you wish to create the AWS S3 Bucket and AWS DynamoDB Table using the CloudFormation Stackset.
  2. Download the CloudFormation Template from this blog and save as terraform-state-backend-CloudFormation.yaml to your local computer.
  3. Next, Go to the AWS CloudFormation Service page and enable the CloudFormation Stackset trusted access with the AWS Organisations by following this link to AWS documentation.
  4. Click on StackSets after expanding the menu on the left, and then click the “create stackset” button and upload the terraform-state-backend-CloudFormation.yaml and then click Next
  5. On the next screen, Enter the stackset name – example: terraform-backend-stackset and click Next again.
  6. Choose the “Service-managed permissions” as default option and click Next
  7. On the “Set deployment options” form, choose the “Deploy to Organizational Units (OU)” Radio option and enter the Organisational Unit ids in which you wish to deploy, make sure the Account removal behavior Radio option is set to Retain and then Automatic deployment is Enabled, also select the AWS Region from the “Specify region” drop-down menu.
  8. On the Next screen, review the configuration and click the Submit button to begin the deployment.
  9. Once the deployment begins, you will see the stackset Operation in Running status and once completed an S3 Bucket and a DynamoDB table will be created in every account of your Organisational Units. This will be a good starting point for writing the Terraform stack configurations using the S3 and DynanmoDB as backend configuration, and if any more new accounts are created in those Organisational Units this stackset will be automatically deployed to them.

The final automation

Once the CloudFormation StackSet completes, each AWS account in the targeted Organisational Unit will have an S3 bucket and a DynamoDB called,

  • S3 Bucket:terraform-state-backend-<12 Digit AWS Account id>
  • Dynamodb: terraform-state-backend-lock-<12 Digit AWS Account id>

and they can be referenced in the Terraform backend configuration as follows:

// backend.tf
terraform {
  required_version = ">= 1.0.0"

  backend "s3" {
    region         = "ap-southeast-2"
    bucket         = "terraform-state-backend-111110011111"
    key            = "test-project-key/dev/terraform.tfstate"
    dynamodb_table = "terraform-state-backend-lock-111110011111"
    encrypt        = "true"
  }
}