Splitting a Terraform / Spacelift stack in 2

A year or so ago, I imported a bunch of existing AWS resources into a Spacelift stack using Terraform. Parts of this stack included provisioning Github actions secrets from AWS into Github itself. Due to the way the Github provider and Github API work, I was starting to hit into rate limits due to my ever-increasing number of secrets.

Rather than do anything fancy with additional authentications with the Github API, or higher limits or refactorings within the stack, I opted to split the stack out into the more manageable and focused stacks, which I had already started with my latest deployment which had a stack all to itself.

Unfortunately, there is no “super easy” way to do this. I was dreaming of clicking a button and being able to drag and drop configuration and or state between the various stacks, that would be dreamy. But instead I had to code up some simple scripts to help me migrate the state locally.

High level process

First:

• Enable external state access on the source stack
• Access the state, and pull it into a local file

Fortunately the elements that I wanted to move were easily grepable for, as they all used modules. The command below would easily list all the ids that I wanted to move.

terraform state list | grep module.s3_admin_Code language: JavaScript (javascript)

I made a little script, that would take this new line delimited list of ids, and output (with --dry-run) or run a bunch of mv commands moving the state to another state file. By default from terraform.tfstate to new.tfstate.

#!/bin/bash

# Default values
DRY_RUN=false
FROM_FILE="terraform.tfstate"
TO_FILE="new.tfstate"

# Parse arguments
while [[ "$#" -gt 0 ]]; do
  case "$1" in
    --dry-run)
      DRY_RUN=true
      shift
      ;;
    --from)
      FROM_FILE="$2"
      shift 2
      ;;
    --to)
      TO_FILE="$2"
      shift 2
      ;;
    *)
      echo "Unknown argument: $1"
      exit 1
      ;;
  esac
done

# Read from stdin and execute commands
while IFS= read -r ID; do
  CMD="terraform state mv -state=$FROM_FILE -state-out=$TO_FILE $ID $ID"
  if $DRY_RUN; then
    echo "$CMD"
  else
    eval "$CMD"
  fi
done
Code language: PHP (php)

Doing this in combination with the previous greped command that would list the ids, would give me a bunch of mv command to run. (and removing the --dry-run would run them)

terraform state list | grep module.s3_site_admin_ | ./tfmv.sh --dry-runCode language: JavaScript (javascript)

This would leave me with a new.tfstate file that I could then import into my new stack.

• Import state file in the Spacelift UI

All that was left was for me to copy the module code across between the stacks, and the state should already be there, resulting in no change.

module "s3_site_admin_somesite_io" {
  source  = "./../../../tf/modules/s3-cloudfront-site"
  domain  = "admin.somesite.io"
  zone_id = data.aws_route53_zone.somesite.id
  acm_certificate_arn  = "arn:aws:acm:us-east-1:1234567:certificate/1234567-1234-1234-1234-1234567"
  github_repository    = "somesite"
  github_secret_prefix = "PRODUCTION_"
}
Code language: JavaScript (javascript)

There was still the issue of deleting the state from the old stack, but I wrote a little script for that too.

#!/bin/bash

# Default values
DRY_RUN=false
SINGLE_LINE=false

# Parse arguments
while [[ "$#" -gt 0 ]]; do
  case "$1" in
    --dry-run)
      DRY_RUN=true
      shift
      ;;
    --single-line)
      SINGLE_LINE=true
      shift
      ;;
    *)
      echo "Unknown argument: $1"
      exit 1
      ;;
  esac
done

# Read from stdin and generate commands
COMMANDS=()
while IFS= read -r ID; do
  COMMANDS+=("terraform state rm $ID")
done

if $SINGLE_LINE; then
  CMD_STRING=$(IFS=' && ' ; echo "${COMMANDS[*]}")
  if $DRY_RUN; then
    echo "$CMD_STRING"
  else
    eval "$CMD_STRING"
  fi
else
  for CMD in "${COMMANDS[@]}"; do
    if $DRY_RUN; then
      echo "$CMD"
    else
      eval "$CMD"
    fi
  done
fiCode language: PHP (php)

The resulting single line command can easily be pasted into the spacelift “Tasks” area to be run and monitored.

Tada, split stacks, with a fairly low amount of manual effort (Though a UI for modifying and updating state and config, or just some other commands would be nice)

And I’m also aware maybe it already exists, but I couldn’t find it while searching yesterday :D