Building and Managing a Multi-Region AWS Aurora Cluster with Failover Setup using Terraform
Todd Bernson / May 11, 2023
In today’s fast-paced digital world, the need for highly available and scalable database systems
has never been greater. AWS Aurora, a MySQL and PostgreSQL-compatible relational database built for the cloud, is a popular choice among businesses seeking to leverage the power of AWS’s scalable and distributed infrastructure. In this article, we’ll explore how to build and manage a multi-region AWS Aurora cluster with an active/active setup using Infrastructure as Code (IaC) tool, Terraform.
has never been greater. AWS Aurora, a MySQL and PostgreSQL-compatible relational database built for the cloud, is a popular choice among businesses seeking to leverage the power of AWS’s scalable and distributed infrastructure. In this article, we’ll explore how to build and manage a multi-region AWS Aurora cluster with an active/active setup using Infrastructure as Code (IaC) tool, Terraform.Understanding AWS Aurora
AWS Aurora is a part of Amazon’s Relational Database Service (RDS) offering, designed to provide high performance and availability, offering up to five times the throughput of standard MySQL and three times the throughput of standard PostgreSQL. Its multi-region, failover setup leverages the power of multiple geographical locations to deliver robustness and resilience, ensuring that your database stays up and running even in the face of region-wide outages.
Infrastructure as Code (IaC) with Terraform
Infrastructure as Code is a key practice in DevOps that promotes infrastructure management in a descriptive model, using the same versioning system as the DevOps team uses for source code. Among various tools available, Terraform stands out due to its cloud-agnostic approach. It allows you to manage various service providers and in-house custom solutions.
All the code with the README can be found here.
Setting Up Terraform
To start using Terraform for managing your AWS resources, you first need to set up your AWS CLI and Terraform CLI. Once these are appropriately installed and configured with your AWS credentials, you can kickstart your IaC journey.
Creating AWS Resources with Terraform
Now that your environment is set up, let’s use Terraform to create the necessary AWS resources. The following resources will be created:
VPC in us-east-2
VPC in eu-west-1
module “vpc_0” {
source = "terraform-aws-modules/vpc/aws" version = "~> 3.14.2" azs = local.availability_zones_0 cidr = local.vpc_cidr_0 create_database_subnet_group = true create_flow_log_cloudwatch_iam_role = true create_flow_log_cloudwatch_log_group = true database_subnets = local.database_subnets_0 enable_dhcp_options = true enable_dns_hostnames = true enable_dns_support = true enable_flow_log = true enable_ipv6 = true enable_nat_gateway = true flow_log_cloudwatch_log_group_retention_in_days = 7 flow_log_max_aggregation_interval = 60 name = local.environment one_nat_gateway_per_az = false private_subnet_suffix = "private" private_subnets = local.private_subnets_0 public_subnets = local.public_subnets_0 single_nat_gateway = true tags = var.tags }
Global AWS aurora cluster in us-east-2
AWS Aurora MySQL cluster in us-east-2
AWS Aurora MySQL cluster in eu-west-1
resource "aws_rds_global_cluster" "this" {
global_cluster_identifier = local.environment
engine = "aurora-mysql"
engine_version = "8.0.mysql_aurora.3.02.3"
storage_encrypted = true
}
module "aurora_primary" {
source = "terraform-aws-modules/rds-aurora/aws"
name = local.environment
database_name = aws_rds_global_cluster.this.database_name
engine = aws_rds_global_cluster.this.engine
engine_version = aws_rds_global_cluster.this.engine_version
global_cluster_identifier = aws_rds_global_cluster.this.id
instance_class = "db.r6g.large"
instances = { for i in range(1) : i => {} }
kms_key_id = data.aws_kms_key.rds_0.arn
vpc_id = module.vpc_0.vpc_id
db_subnet_group_name = module.vpc_0.database_subnet_group_name
security_group_rules = {
vpc_ingress = {
cidr_blocks = concat(
module.vpc_0.private_subnets_cidr_blocks,
module.vpc_1.private_subnets_cidr_blocks,
)
}
}
master_username = local.database_username
master_password = local.database_password
skip_final_snapshot = true
tags = var.tags
}
AWS Secret with username and password in us-east-2
resource "aws_secretsmanager_secret" "rds_credentials" {
name = "${local.environment}-aurora-credentials"
description = "${local.environment} aurora username and password"
recovery_window_in_days = "7"
depends_on = [module.aurora_primary]
}
resource "aws_secretsmanager_secret_version" "rds_credentials" {
secret_id = aws_secretsmanager_secret.rds_credentials.id
secret_string = jsonencode(
{
username = module.aurora_primary.cluster_master_username
password = module.aurora_primary.cluster_master_password
}
)
depends_on = [module.aurora_primary]
}
Private bastion in us-east-2
resource "aws_instance" "ubuntu" {
ami = data.aws_ami.ubuntu.id
disable_api_termination = false
ebs_optimized = true
iam_instance_profile = aws_iam_instance_profile.this.name
instance_type = "t3.small"
monitoring = true
subnet_id = module.vpc_0.private_subnets[0]
metadata_options {
http_endpoint = "enabled"
http_put_response_hop_limit = 3
http_tokens = "required"
}
user_data = <<-EOF
#!/bin/bash
sudo apt-get update -y
sudo apt-get upgrade -y
sudo apt-get dist-upgrade -y
sudo apt-get install mysql-client -y
EOF
tags = merge(var.tags, { "Name" = "${var.environment}_ubuntu" })
volume_tags = merge(var.tags, { "Name" = "${var.environment}_ubuntu_vol" })
root_block_device {
encrypted = true
volume_type = "gp3"
}
lifecycle {
ignore_changes = [user_data, ami]
}
}
To initiate the process, run:
cd terraform terraform init terraform plan -out=plan.out terraform apply
This will create 111 resources.
Managing Terraform State
Terraform uses a state file to map resources to your configuration. By default, Terraform stores this state file on your local filesystem. However, you’ll need a remote backend when working on a shared project or across different machines. Terraform supports multiple backends like AWS S3, Google Cloud Storage, etc. Remember to add a backend.tf file to your project with your desired backend for the state file.
Advanced Terraform Usage
Terraform scripts can be made more efficient and reusable by using variables, modules, and data sources. For instance, you can change some default values in your variables.tf file according to your needs.
Troubleshooting and Best Practices
When working with AWS and Terraform, you might encounter issues that require troubleshooting. Terraform provides detailed error messages that can help you identify and fix problems. Always ensure your AWS CLI and Terraform CLI are set up correctly and that you have the necessary permissions in your AWS account.