Running AWS at the Edge — IAM Roles Anywhere
The Edge Computing Challenge Link to heading
Traditional AWS authentication relies on long-lived access keys or assumes you’re running within AWS infrastructure. Storing access keys on servers is insecure — and hardcoding them in application code is even worse. So what happens when you need secure AWS access from:
- On-premises servers
- Edge computing devices
- CI/CD pipelines in other clouds
- IoT devices at remote locations
- Hybrid cloud environments
Enter IAM Roles Anywhere — AWS’s solution for extending IAM roles beyond the AWS boundary using X.509 certificates.
By the end of this article, you’ll have a working setup where an edge device authenticates to AWS using a certificate from a free, self-hosted CA — no access keys anywhere. I rebuilt this exact flow recently on a Raspberry Pi + Mac Mini over Tailscale, and hit almost every gotcha this article now covers. Rather than write the sanitized happy-path version, I’m including the actual errors and fixes, because they’re the part that costs real time.
What is IAM Roles Anywhere? Link to heading
IAM Roles Anywhere lets workloads outside AWS assume IAM roles using X.509 certificates instead of long-lived access keys.
Key benefits:
- No more long-lived credentials stored on edge devices
- Certificate-based authentication with automatic rotation
- Fine-grained permissions using existing IAM policies
- Audit trail through CloudTrail integration
- Temporary credentials that expire automatically
Core Concepts Link to heading
Trust Anchor — This is the root of trust. You register a Certificate Authority (CA) with AWS. IAM Roles Anywhere will only accept credentials signed by a CA you’ve registered here. Think of it as telling AWS: “I vouch for any certificate signed by this CA.”
Profile — A Profile maps a certificate to an IAM role (or set of roles) with an optional permission boundary. When a workload presents a valid certificate, the Profile determines what role it can assume and what the maximum permissions are. You can have multiple Profiles — for example, one for read-only edge devices and one for devices that need to write to S3.
Credential Helper — The aws_signing_helper binary is a tool you run on the external workload. It handles the cryptographic handshake: presenting the certificate, signing the request, and exchanging it for temporary AWS credentials. It integrates directly with the AWS credential provider chain.
Setting Up step-ca Link to heading
Instead of paying for AWS Private CA, we’ll use step-ca — an open-source CA server that runs anywhere. It handles certificate issuance, renewal, and the full PKI lifecycle for free.
For this tutorial we’ll run step-ca on a dedicated machine reachable by your devices — I used a Raspberry Pi over Tailscale, which keeps the CA off the public internet entirely while still being reachable from anywhere my devices are. Install the step CLI and step-ca:
# macOS
brew install step
# Linux (Debian/Ubuntu, amd64)
wget https://dl.smallstep.com/cli/docs-cli-install/latest/step-cli_amd64.deb
sudo dpkg -i step-cli_amd64.deb
wget https://dl.smallstep.com/certificates/docs-ca-install/latest/step-ca_amd64.deb
sudo dpkg -i step-ca_amd64.deb
# Raspberry Pi (arm64) — swap the arch in the URL
wget https://dl.smallstep.com/cli/docs-cli-install/latest/step-cli_arm64.deb
sudo dpkg -i step-cli_arm64.deb
wget https://dl.smallstep.com/certificates/docs-ca-install/latest/step-ca_arm64.deb
sudo dpkg -i step-ca_arm64.deb
Gotcha: confirm where the package actually installs the binary before writing any systemd unit later. On my Pi it landed at
/usr/bin/step-ca, not/usr/local/bin/step-ca— a mismatch here causes a silent-looking203/EXECfailure in systemd that has nothing to do with your CA config. Check withwhich step-cabefore moving on.
Run the CA as a dedicated, non-root user Link to heading
It’s tempting to just run step ca init and step-ca as your normal user or as root, but neither is great practice for something network-facing that holds your root and intermediate private keys. Create a locked-down service account instead:
sudo useradd -r -m -s /usr/sbin/nologin step-ca
sudo mkdir -p /home/step-ca
sudo chown step-ca:step-ca /home/step-ca
Bind the CA to a port above 1024 (e.g. :9000) so this unprivileged user can start the listener without needing setcap or root — running on :443 would require one or the other.
Step 1 — Initialize the CA Link to heading
sudo -u step-ca -H step ca init \
--name "homelab-ca" \
--dns "<tailscale-ip>" \
--address ":9000" \
--provisioner "admin@homelab" \
--password-file /home/step-ca/.step/secrets/password.txt
Use your Pi’s actual Tailscale IP for --dns — step-ca embeds it as an IP SAN, and whatever address you put here is the only address your devices will be able to reach the CA at later. .local mDNS hostnames are tempting to use instead, but Linux hosts generally can’t resolve them across a Tailscale link or across subnets without extra avahi/nss-mdns config — save yourself the debugging and use the IP directly.
This generates a root certificate, an intermediate certificate, and a config file under ~/.step/. Save the fingerprint it prints — you’ll need it to bootstrap trust from other machines.
Lock down the secrets directory:
sudo chmod 700 /home/step-ca/.step/secrets
sudo chmod 600 /home/step-ca/.step/secrets/*
Step 2 — Run the CA as a systemd service Link to heading
# /etc/systemd/system/step-ca.service
[Unit]
Description=step-ca
After=network.target
[Service]
User=step-ca
Group=step-ca
ExecStart=/usr/bin/step-ca /home/step-ca/.step/config/ca.json --password-file /home/step-ca/.step/secrets/password.txt
Restart=on-failure
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable --now step-ca
sudo systemctl status step-ca
If this fails with 203/EXEC, it’s almost always the ExecStart binary path being wrong — re-check which step-ca and fix the unit file, not the CA config.
Step 3 — Issue a Certificate for Your Workload Link to heading
Issue a certificate for the edge device. The Common Name identifies the device and will appear as the session name in CloudTrail logs, so use something meaningful:
step ca certificate edge-device-01 device.crt device.key \
--ca-url https://<tailscale-ip>:9000 \
--root $(step path)/certs/root_ca.crt
This creates device.crt and device.key. Restrict permissions on the private key:
chmod 600 device.key
Gotcha you’ll almost certainly hit: by default,
step ca certificatewrites the leaf certificate bundled together with the intermediate CA cert into the same output file. That’s usually helpful — most TLS consumers want the full chain in one file. Butaws_signing_helperdoes not auto-split a bundled--certificatefile. Point it at a 2-cert bundle and AWS will reject it with a bare"Untrusted signing certificate"error, even though the chain is completely valid. Running with--debugreveals why: the request only carries anX-Amz-X509header (the leaf) and noX-Amz-X509-Chainheader at all.The fix is to split the file and pass the intermediate explicitly:
awk '/-----BEGIN CERTIFICATE-----/{n++} {print > ("part" n ".pem")}' device.crt # part1.pem = leaf, part2.pem = intermediate mv part1.pem device.crt mv part2.pem device-chain.crtThen reference both in your credential helper call — see the Test the Credential Helper section below. Verify the split worked with
grep -c "BEGIN CERTIFICATE" device.crt— it should return1.
Step 4 — Export the Root + Intermediate for the Trust Anchor Link to heading
This is a separate concern from Step 3’s leaf-cert splitting: AWS’s Trust Anchor needs the full CA chain (root + intermediate) to know what to trust, independent of what any individual leaf certificate looks like.
cat $(step path)/certs/root_ca.crt $(step path)/certs/intermediate_ca.crt > bundle.pem
The next section uses bundle.pem to register the Trust Anchor via Terraform.
Setting Up IAM Roles Anywhere with Terraform Link to heading
Rather than click through the console or chain together one-off CLI calls, manage the Trust Anchor, IAM Role, and Profile as Terraform. This keeps the whole trust relationship — including the CN condition that ties a role to a specific certificate identity — as reviewable, versioned code. Set it up in a directory like terraform/iam_roles_anywhere/, alongside a certs/ subdirectory for the CA bundle.
Fetch the CA certificates Link to heading
Terraform reads the CA bundle from a local file, so pull the root and intermediate certs off the Pi and concatenate them:
scripts/fetch-certs.sh:
#!/usr/bin/env bash
set -euo pipefail
REMOTE_HOST="<your-username>@<tailscale-ip>"
REMOTE_CERT_DIR="/home/<your-username>/.step/certs"
LOCAL_CERT_DIR="$(cd "$(dirname "$0")/.." && pwd)/certs"
mkdir -p "$LOCAL_CERT_DIR"
echo "==> Fetching certs from $REMOTE_HOST..."
scp "$REMOTE_HOST:$REMOTE_CERT_DIR/root_ca.crt" "$LOCAL_CERT_DIR/root.crt"
scp "$REMOTE_HOST:$REMOTE_CERT_DIR/intermediate_ca.crt" "$LOCAL_CERT_DIR/intermediate.crt"
echo "==> Building bundle..."
cat "$LOCAL_CERT_DIR/root.crt" "$LOCAL_CERT_DIR/intermediate.crt" > "$LOCAL_CERT_DIR/bundle.pem"
echo "Done. Bundle written to $LOCAL_CERT_DIR/bundle.pem"
chmod +x scripts/fetch-certs.sh
./scripts/fetch-certs.sh
This bundle is a separate concern from the leaf-cert-splitting gotcha above — the Trust Anchor needs the full chain regardless of how you structure your device’s individual leaf certificate file.
Terraform files Link to heading
terraform.tf — backend and provider:
terraform {
required_version = ">= 1.10" # use_lockfile requires 1.10+ (native S3 locking, no DynamoDB needed)
backend "s3" {
bucket = "<your-tf-state-bucket>"
key = "iam_roles_anywhere.tfstate"
region = "us-east-1"
encrypt = true
use_lockfile = true
}
required_providers {
aws = {
source = "hashicorp/aws"
version = "6.23.0"
}
}
}
provider "aws" {}
variables.tf:
variable "trust_anchor_name" {
type = string
description = "Name for the IAM Roles Anywhere trust anchor."
}
variable "profile_name" {
type = string
description = "Name for the IAM Roles Anywhere profile."
}
variable "role_name" {
type = string
description = "Name for the IAM role workloads will assume."
}
variable "ca_bundle_path" {
type = string
description = "Path to a PEM file containing the CA certificate(s) for the trust anchor. Concatenate root + intermediate into one file if needed."
}
variable "role_policy_arns" {
type = list(string)
description = "List of IAM managed policy ARNs to attach to the role."
default = []
}
variable "allowed_cn" {
type = string
description = "Certificate CN that is allowed to assume the role via Roles Anywhere."
}
variable "session_duration_seconds" {
type = number
description = "Max session duration (seconds) for credentials issued by the profile."
default = 3600
}
main.tf:
resource "aws_rolesanywhere_trust_anchor" "this" {
name = var.trust_anchor_name
enabled = true
source {
source_type = "CERTIFICATE_BUNDLE"
source_data {
x509_certificate_data = file(var.ca_bundle_path)
}
}
}
resource "aws_iam_role" "this" {
name = var.role_name
max_session_duration = var.session_duration_seconds
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Principal = { Service = "rolesanywhere.amazonaws.com" }
Action = [
"sts:AssumeRole",
"sts:TagSession",
"sts:SetSourceIdentity",
]
Condition = {
ArnEquals = {
"aws:SourceArn" = aws_rolesanywhere_trust_anchor.this.arn
}
StringEquals = {
"aws:PrincipalTag/x509Subject/CN" = var.allowed_cn
}
}
},
]
})
}
resource "aws_iam_role_policy_attachment" "this" {
for_each = toset(var.role_policy_arns)
role = aws_iam_role.this.name
policy_arn = each.value
}
resource "aws_rolesanywhere_profile" "this" {
name = var.profile_name
enabled = true
role_arns = [aws_iam_role.this.arn]
duration_seconds = var.session_duration_seconds
require_instance_properties = false
}
Two conditions do the real work in the trust policy: aws:SourceArn ties the role to this specific Trust Anchor (no other trust anchor’s certs can assume it), and aws:PrincipalTag/x509Subject/CN ties it to one specific certificate identity.
Important consequence of that CN condition: it’s a plain string match, with no reachability requirement behind it — which affects how you issue the certificate. See the callout below.
outputs.tf:
output "trust_anchor_arn" {
value = aws_rolesanywhere_trust_anchor.this.arn
}
output "profile_arn" {
value = aws_rolesanywhere_profile.this.arn
}
output "role_arn" {
value = aws_iam_role.this.arn
}
terraform.tfvars:
trust_anchor_name = "edge-device-ca"
profile_name = "edge-device-profile"
role_name = "EdgeDeviceRole"
ca_bundle_path = "./certs/bundle.pem"
role_policy_arns = [
# Attach only the policies the workload actually needs — avoid broad managed
# policies here. This role's permissions are the blast radius of a
# compromised device certificate.
# "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
]
allowed_cn = "edge-device-01"
session_duration_seconds = 3600
Keep role_policy_arns as narrow as possible — grant only what the workload actually needs.
Choosing a step-ca provisioner: ACME vs. JWK. step-ca supports multiple ways to authorize certificate issuance, and the choice matters more than it first appears once you add a CN condition like the one above.
- ACME issues a cert by having the CA make an actual network connection back to the requested name (an HTTP-01 challenge) to prove you control it. This works great when your CN/SAN is a real, reachable hostname or IP — and it enables fully unattended renewal with no password. But it fails outright if you request a CN that isn’t a routable address, because there’s nothing for the CA to connect to.
- JWK (the default password-based provisioner) instead authorizes issuance with a shared secret. It doesn’t care whether the CN is reachable — you can request literally any string as the CN, including an identifier like
edge-device-01that only exists as a label in your IAM policy.If your IAM trust policy conditions on a specific CN string that isn’t also a real hostname/IP your CA can dial back to, you need JWK for issuance. The good news: once issued,
step ca renewworks identically regardless of which provisioner issued the original cert — renewal authenticates via mTLS with the existing valid certificate, not the original provisioner secret. So a JWK-issued cert still renews unattended with no password prompt down the line.
Apply Link to heading
terraform init
terraform apply
Pull the ARNs you’ll need next straight from the outputs:
terraform output trust_anchor_arn
terraform output profile_arn
terraform output role_arn
If you’re updating an existing trust anchor to point at a new CA (say, you rebuilt your step-ca instance), re-run fetch-certs.sh against the new CA and terraform apply again — Terraform will update aws_rolesanywhere_trust_anchor.this in place. Double-check the applied source_data actually reflects the new root, since a stale trust anchor produces the exact same generic error as a bad certificate chain.
Test the Credential Helper Link to heading
With all three ARNs in hand, verify everything works. If you had to split your leaf cert from its bundled intermediate (see the callout in the certificate issuance section above), pass the intermediate via --intermediates:
aws_signing_helper credential-process \
--certificate /path/to/device.crt \
--private-key /path/to/device.key \
--intermediates /path/to/device-chain.crt \
--trust-anchor-arn $(terraform output -raw trust_anchor_arn) \
--profile-arn $(terraform output -raw profile_arn) \
--role-arn $(terraform output -raw role_arn)
You should see a JSON response with a temporary AccessKeyId, SecretAccessKey, and SessionToken. If you get “Untrusted signing certificate,” check both possible causes:
- Your Trust Anchor bundle is missing the intermediate CA cert (fix by re-running
fetch-certs.shandterraform apply). - Your leaf cert file is a bundle and you didn’t pass
--intermediatesseparately (fix at theaws_signing_helperinvocation level — the more common cause in practice). Run with--debugand check whether the request actually carries anX-Amz-X509-Chainheader; if it doesn’t, this is your problem.
Real-World Use Case — Edge Device Fleet Link to heading
Imagine you’re running a fleet of edge devices in the field — sensors, cameras, or industrial controllers — each needing to push data to S3 or publish to an SNS topic. The traditional approach is to bake an access key into each device — hard to rotate across a fleet, and a single compromised device exposes credentials shared by every other device.
With IAM Roles Anywhere, each device gets its own unique certificate issued by step-ca at provisioning time. The setup looks like this:
- Each device has a certificate with a unique Common Name (e.g.,
sensor-device-042). This becomes the session name in CloudTrail, so every API call is traceable to a specific device. - The certificate is valid for 24 hours and renewed automatically by the
stepCLI before expiry — no manual rotation. - If a device is decommissioned or physically destroyed, the private key is gone with it — no one can authenticate using that device’s credentials.
- Each device assumes the same
EdgeDeviceRole, but because sessions are scoped per certificate, you can use IAM condition keys likeaws:PrincipalTagto write policies that restrict each device to its own S3 prefix.
Note: If a device is lost rather than destroyed, revoke its certificate at the CA level immediately. With 24-hour certificates the exposure window is bounded, but revocation cuts it to zero. This is a major advantage over long-lived access keys, which remain valid indefinitely until manually deleted.
The end result: a fleet where every device has short-lived, automatically rotating credentials, full per-device auditability in CloudTrail, and a single revocation point if anything goes wrong.
Caveat: Each edge device needs network connectivity to your step-ca server — both at initial provisioning to receive its certificate, and every renewal cycle after that. If step-ca is unreachable, a device can’t renew and will lose AWS access once its certificate expires. For truly remote or intermittently connected devices, consider issuing longer-lived certificates, or ensuring step-ca is reachable via VPN/Tailscale rather than requiring public internet access.
Automating Renewal Link to heading
step ca renew handles rotation, but it’s worth knowing exactly how it behaves before you wire it into a scheduler:
- It’s a no-op unless the cert is within its renewal window (roughly the last third of its validity) — so running it frequently (hourly, even) is safe and cheap.
--forceoverrides the window check and always attempts renewal — useful for testing, but means every scheduled run actually round-trips to the CA.- Renewal uses mTLS with the current valid cert, not the original provisioner — so this works identically whether the cert was originally issued via ACME or JWK.
If your device’s leaf cert needs to stay split from its intermediate (per the earlier gotcha), remember that a plain step ca renew re-bundles them together again. Wrap the renewal in a small script that re-splits afterward if your credential helper invocation depends on separate files:
#!/bin/bash
set -e
step ca renew --force device.crt device.key
awk '/-----BEGIN CERTIFICATE-----/{n++} {print > ("/tmp/renew-part" n ".pem")}' device.crt
cp /tmp/renew-part1.pem device.crt
cp /tmp/renew-part2.pem device-chain.crt
rm -f /tmp/renew-part*.pem
Schedule this with cron, systemd timers, or (on macOS) a launchd LaunchAgent/LaunchDaemon with StartInterval.
Cleanup Link to heading
When you’re done, tear down the AWS resources to avoid any lingering costs or open trust relationships. Since everything’s managed in Terraform, this is a single command:
terraform destroy
This removes the Profile, IAM Role (and its policy attachments), and Trust Anchor together, in the correct dependency order.
If you’re done with step-ca entirely, stop the server and remove the config:
sudo systemctl stop step-ca
sudo systemctl disable step-ca
sudo rm -rf /home/step-ca/.step