Secure Secrets Management with HashiCorp Vault on EKS

Here's an uncomfortable truth about Kubernetes: Secrets are base64-encoded, not encrypted. Anyone with cluster access can decode them with a single command.

For production workloads handling sensitive data - database credentials, API keys, encryption keys - this isn't acceptable.

HashiCorp Vault provides enterprise-grade secrets management that Kubernetes Secrets can't match: real encryption, fine-grained access control, complete audit logging, and dynamic secrets that rotate automatically.

This guide covers deploying Vault on EKS and integrating it with your applications in a way that actually works in production.

_____

Why Vault on Kubernetes?

Centralized secrets - One source of truth across all environments and applications. No more secrets scattered across different systems.
Dynamic secrets - Generate short-lived database credentials on-demand. When a pod requests credentials, Vault creates a unique PostgreSQL user that expires in an hour. No more shared, long-lived passwords.
Audit logging - Every secret access is logged. When compliance asks "who accessed that API key?", you have the answer with timestamps and context.
Encryption as a service - Applications can encrypt data without managing keys themselves. Vault handles the cryptography.
Access control - Fine-grained policies controlling which applications access which secrets. A web frontend shouldn't have access to database admin credentials.

_____

Architecture Overview

The Vault Agent Injector automatically adds sidecar containers to pods that need secrets. Your application reads secrets from files - no code changes required.

_____

Part 1: Deploying Vault with Helm

Prerequisites

# Add HashiCorp Helm repo
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update

Vault Helm Values

Here's a production-ready configuration:

1# vault-values.yaml
2global:
3  enabled: true
4  tlsDisable: false
5
6server:
7  enabled: true
8  replicas: 3
9  
10  resources:
11    requests:
12      memory: "256Mi"
13      cpu: "250m"
14    limits:
15      memory: "512Mi"
16      cpu: "500m"
17
18  ha:
19    enabled: true
20    replicas: 3
21    raft:
22      enabled: true
23      setNodeId: true
24      config: |
25 ui = true
26 
27 listener "tcp" {
28 tls_disable = false
29 address = "[::]:8200"
30 cluster_address = "[::]:8201"
31 tls_cert_file = "/vault/userconfig/vault-tls/tls.crt"
32 tls_key_file = "/vault/userconfig/vault-tls/tls.key"
33 }
34 
35 storage "raft" {
36 path = "/vault/data"
37 retry_join {
38 leader_api_addr = "https://vault-0.vault-internal:8200"
39 }
40 retry_join {
41 leader_api_addr = "https://vault-1.vault-internal:8200"
42 }
43 retry_join {
44 leader_api_addr = "https://vault-2.vault-internal:8200"
45 }
46 }
47 
48 seal "awskms" {
49 region = "ap-south-1"
50 kms_key_id = "YOUR_KMS_KEY_ID"
51 }
52 
53 service_registration "kubernetes" {}
54
55  serviceAccount:
56    create: true
57    annotations:
58      eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT_ID:role/vault-kms-unseal"
59
60  dataStorage:
61    enabled: true
62    size: 10Gi
63    storageClass: gp3
64
65  volumes:
66    - name: vault-tls
67      secret:
68        secretName: vault-tls
69
70  volumeMounts:
71    - name: vault-tls
72      mountPath: /vault/userconfig/vault-tls
73      readOnly: true
74
75injector:
76  enabled: true
77  replicas: 2
78  resources:
79    requests:
80      memory: "64Mi"
81      cpu: "50m"
82    limits:
83      memory: "128Mi"
84      cpu: "100m"
85
86ui:
87  enabled: true
88  serviceType: ClusterIP

Deploy Vault

1# Create namespace
2kubectl create namespace vault
3
4# Create TLS secret (use cert-manager or your own certs)
5kubectl create secret tls vault-tls \
6  --cert=vault.crt \
7  --key=vault.key \
8  -n vault
9
10# Install Vault
11helm install vault hashicorp/vault \
12  -n vault \
13  -f vault-values.yaml

Initialize Vault

1# Initialize (only once, on first deployment)
2kubectl exec -n vault vault-0 -- vault operator init \
3  -key-shares=5 \
4  -key-threshold=3 \
5  -format=json > vault-init.json
6
7# With KMS auto-unseal, Vault unseals automatically
8# Verify status
9kubectl exec -n vault vault-0 -- vault status

______

Part 2: AWS KMS Auto-Unseal

Vault encrypts all data with a master key. By default, operators must manually unseal Vault after every restart - including pod restarts, node failures, or upgrades. In Kubernetes, this becomes impractical quickly.

KMS auto-unseal eliminates this operational burden. Vault uses AWS KMS to automatically unseal itself.

Terraform for KMS and IAM

1# kms.tf
2resource "aws_kms_key" "vault_unseal" {
3  description = "Vault auto-unseal key"
4  deletion_window_in_days = 7
5  enable_key_rotation = true
6
7  tags = {
8    Purpose = "vault-unseal"
9  }
10}
11
12resource "aws_kms_alias" "vault_unseal" {
13  name = "alias/vault-unseal"
14  target_key_id = aws_kms_key.vault_unseal.key_id
15}
16
17# IAM role for Vault pods (IRSA)
18module "vault_irsa" {
19  source = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
20  version = "~> 5.0"
21
22  role_name = "vault-kms-unseal"
23
24  oidc_providers = {
25    main = {
26      provider_arn               = module.eks.oidc_provider_arn
27      namespace_service_accounts = ["vault:vault"]
28    }
29  }
30}
31
32resource "aws_iam_role_policy" "vault_kms" {
33  name = "vault-kms-access"
34  role = module.vault_irsa.iam_role_name
35
36  policy = jsonencode({
37    Version = "2012-10-17"
38    Statement = [
39      {
40        Effect = "Allow"
41        Action = [
42          "kms:Encrypt",
43          "kms:Decrypt",
44          "kms:DescribeKey"
45        ]
46        Resource = aws_kms_key.vault_unseal.arn
47      }
48    ]
49  })
50}

______

Part 3: Kubernetes Authentication

Enable pods to authenticate with Vault using their service account tokens. This is the cleanest integration pattern.

1# Enable Kubernetes auth method
2kubectl exec -n vault vault-0 -- vault auth enable kubernetes
3
4# Configure Kubernetes auth
5kubectl exec -n vault vault-0 -- vault write auth/kubernetes/config \
6  kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"

Create Policy and Role

1# Create a policy for application secrets
2kubectl exec -n vault vault-0 -- vault policy write app-policy - <<EOF
3path "secret/data/app/*" {
4 capabilities = ["read"]
5}
6
7path "database/creds/app-role" {
8 capabilities = ["read"]
9}
10EOF
11
12# Create Kubernetes auth role
13kubectl exec -n vault vault-0 -- vault write auth/kubernetes/role/app-role \
14  bound_service_account_names=app-sa \
15  bound_service_account_namespaces=application \
16  policies=app-policy \
17  ttl=1h

_______

Part 4: Injecting Secrets into Pods

The Vault Agent Injector automatically injects secrets into pods via annotations. No code changes to your application.

Application Deployment

1# app-deployment.yaml
2apiVersion: v1
3kind: ServiceAccount
4metadata:
5  name: app-sa
6  namespace: application
7---
8apiVersion: apps/v1
9kind: Deployment
10metadata:
11  name: myapp
12  namespace: application
13spec:
14  replicas: 2
15  selector:
16    matchLabels:
17      app: myapp
18  template:
19    metadata:
20      labels:
21        app: myapp
22      annotations:
23        # Enable Vault Agent injection
24        vault.hashicorp.com/agent-inject: "true"
25        vault.hashicorp.com/role: "app-role"
26        
27        # Inject database credentials
28        vault.hashicorp.com/agent-inject-secret-db-creds: "database/creds/app-role"
29        vault.hashicorp.com/agent-inject-template-db-creds: |
30 {{- with secret "database/creds/app-role" -}}
31 export DB_USERNAME="{{ .Data.username }}"
32 export DB_PASSWORD="{{ .Data.password }}"
33 {{- end }}
34 
35        # Inject application secrets
36        vault.hashicorp.com/agent-inject-secret-config: "secret/data/app/config"
37        vault.hashicorp.com/agent-inject-template-config: |
38 {{- with secret "secret/data/app/config" -}}
39 {
40 "api_key": "{{ .Data.data.api_key }}",
41 "encryption_key": "{{ .Data.data.encryption_key }}"
42 }
43 {{- end }}
44    spec:
45      serviceAccountName: app-sa
46      containers:
47        - name: app
48          image: myapp:latest
49          command: ["/bin/sh", "-c"]
50          args:
51            - source /vault/secrets/db-creds && ./start.sh
52          volumeMounts:
53            - name: secrets
54              mountPath: /vault/secrets
55              readOnly: true

Here's what happens behind the scenes:

Pod starts with Vault Agent as an init container
Agent authenticates to Vault using the Kubernetes service account
Agent fetches secrets and writes them to /vault/secrets/
Your application reads secrets as files
Agent sidecar continues running, refreshing secrets before they expire

______

Part 5: Dynamic Database Secrets

Static passwords are a security risk. Vault can generate unique, short-lived database credentials for each pod.

1# Enable database secrets
2kubectl exec -n vault vault-0 -- vault secrets enable database
3
4# Configure PostgreSQL connection
5kubectl exec -n vault vault-0 -- vault write database/config/postgres \
6  plugin_name=postgresql-database-plugin \
7  allowed_roles="app-role" \
8  connection_url="postgresql://{{username}}:{{password}}@postgres.database:5432/appdb" \
9  username="vault_admin" \
10  password="admin_password"
11
12# Create role for dynamic credentials
13kubectl exec -n vault vault-0 -- vault write database/roles/app-role \
14  db_name=postgres \
15  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
16  default_ttl="1h" \
17  max_ttl="24h"

Now every time a pod requests database/creds/app-role, Vault creates a unique PostgreSQL user with 1-hour validity. When the pod terminates or the TTL expires, Vault revokes the credentials.

______

Best Practices

After running Vault on EKS in production, here's what I've learned:

Backup regularly - Raft snapshots should be automated and stored securely in S3. Test your restore procedure before you need it.
Monitor seal status - Alert immediately if Vault becomes sealed. This breaks all applications depending on secrets.
Rotate root tokens - After initial setup, revoke the root token and use regular authentication. Root tokens are for emergencies only.
Use namespaces - Vault Enterprise namespaces isolate teams and environments. Even with open source, use different paths for different applications.
Audit everything - Enable audit logging to S3 or CloudWatch. When something goes wrong - and it will - you need to know what happened.

_______

Wrapping Up

Vault on EKS transforms secrets management from a liability into a security strength. Dynamic secrets eliminate password sprawl. Kubernetes auth removes the need for static tokens. KMS auto-unseal ensures high availability without manual intervention.

The initial setup requires effort, but you gain a secrets infrastructure that scales with your organization and satisfies even strict compliance requirements. Your applications get secrets automatically. Your security team gets audit trails. Everyone sleeps better.

That's the goal of secrets management - security that works with your developers, not against them.