Skip to main content

Internal encryption

platform v0.9.11verified 2026-05-14

How to flip Postgres and Redis from plaintext to TLS across every service without an outage. Each step is reversible and the system stays usable in every intermediate state.

The shape of the rollout​

  • Postgres has two legs: app → PgBouncer (Leg 1) and PgBouncer → Postgres (Leg 2). PgBouncer brokers between them so clients can upgrade independently.
  • Redis ships in plaintext mode and supports a parallel TLS listener; both stay up during transition. Each consumer flips at its own pace.

Cert material​

All three TLS variables live in Secrets Manager and are decoded to the filesystem by init.sh. The same server cert serves Postgres, PgBouncer, and Redis — SANs must cover the Database instance's private DNS.

NameSourceScopeDefaultDescription
INTERNAL_CA_CRT_B64Secrets Managerall—Base64 PEM of the internal CA.
INTERNAL_TLS_CERT_B64Secrets Managerall—Base64 PEM of the server cert.
INTERNAL_TLS_KEY_B64Secrets Managerall—Base64 PEM of the server private key. Decoded with mode 0600, owner 70:70.

Postgres rollout​

Order, one step per environment / deploy cycle:

  1. Provision certs — push the three base64 PEMs into Secrets Manager.
  2. POSTGRES_TLS_ENABLED=on — appends -c ssl=on to the Postgres command. Plaintext listener still works.
  3. PGBOUNCER_SERVER_TLS_SSLMODE=verify-ca — PgBouncer → Postgres now uses TLS.
  4. PGBOUNCER_CLIENT_TLS_SSLMODE=allow — clients can negotiate TLS to PgBouncer but aren't required.
  5. Flip each app's DATABASE_SSL_MODE=verify-full — one service at a time: api → web → voice → ops → others.
  6. PGBOUNCER_CLIENT_TLS_SSLMODE=require — refuse plaintext on Leg 1.
  7. (Optional) POSTGRES_FORCE_SSL=on — rewrites pg_hba.conf to hostssl-only. Belt-and-braces.

Each app's DATABASE_SSL_CA_FILE should point at /etc/ssl/internal/ca.crt; init.sh writes the decoded CA there from INTERNAL_CA_CRT_B64.

Redis rollout​

Redis runs both listeners during transition: plaintext on :6379 and TLS on REDIS_TLS_PORT (default :6380).

  1. REDIS_TLS_ENABLED=true — restarts Redis with both listeners; the OTel collector's REDIS_ENDPOINT is auto-switched to :6380 by init.sh.
  2. Flip each app — set REDIS_TLS_ENABLED=true and REDIS_TLS_CA_FILE=/etc/ssl/internal/ca.crt per service. Cycle order: api, web, voice, ops, telpro.
  3. REDIS_FORCE_TLS=true — once every client moved, set --port 0 to drop the plaintext listener.
TelPro Kamailio

The voiceai-telpro Kamailio image needs hiredis_ssl baked in for ndb_redis to talk TLS. Plain redis-cli health probes and bash tooling don't need the SSL-enabled image. Ship the SSL build of voiceai-telpro before flipping REDIS_TLS_ENABLED=true on TelPro.

Verification​

# Postgres TLS server-side
docker compose exec voiceai-postgres psql -U voiceai \
  -c "SELECT ssl, version FROM pg_stat_ssl WHERE pid=pg_backend_pid();"

# Postgres TLS client-side from an app instance
psql "postgresql://voiceai@<db-host>:5432/voiceai?sslmode=verify-full&sslrootcert=/etc/ssl/internal/ca.crt" -c "SELECT 1;"

# Redis TLS listener
docker exec voiceai-redis redis-cli -p 6380 --tls \
  --cacert /tmp/redis-tls/ca.crt --insecure ping

# Redis TLS client-side from an app instance
redis-cli -h <db-host> -p 6380 --tls \
  --cacert /etc/ssl/internal/ca.crt -a "${REDIS_PASSWORD}" ping

Rollback​

Every step is reversible by setting the variable back and rerunning ./update.sh --restart-only on the relevant host. Because plaintext listeners stay up during the rollout, a partial rollback leaves the system functional.

See also​