API service operations
The API service runs the TelAPI server — a Fastify-based HTTP and WebSocket API that handles call tokens, browser-to-ARI channel communication, and external API endpoints. Multiple instances run behind a managed Load Balancer with TLS termination and sticky sessions.
- Overview
- Runbook
- Configuration
- Troubleshooting
Containers
| Container | Image | Port | Purpose |
|---|---|---|---|
voiceai-telapi | ${ECR_REGISTRY}/voiceai-telapi:${ECR_TAG} (Node 24-alpine) | 3001 | API + WebSocket gateway (host network) |
voiceai-otel-collector | otel/opentelemetry-collector-contrib:0.150.1 | — | Telemetry collector |
Load balancer
| Field | Value |
|---|---|
| Frontend | HTTPS :443 with managed TLS for DOMAIN_API |
| Backend | :3001 |
| Health check | GET /health |
| Sticky sessions | Enabled — required for WebSocket |
| Target selection | role=api,environment=${ENVIRONMENT} |
Networking
Private network only — no public IP. All traffic arrives through the LB. Outbound to ECR, AWS Secrets, and provider APIs goes through the Squid proxy.
Scaling
Set the initial API instance count in your provisioning tool. The Ops service Scaler can also manage count dynamically based on active connections.
Instances are named ${environment}-api-01, -02, etc. New instances register with the LB automatically via labels.
Deploy and update
cd /opt/services/api
./init.sh # bring up
./update.sh --ecr-tag v0.9.11 # roll a new image
./update.sh --restart-only
init.sh fetches config from S3, loads env vars from SSM / Secrets Manager into memory via fetch-env.sh, logs into ECR, and starts Compose. Standard flags: --skip-images, --skip-fetch, --restart-only, --dry-run.
TTS media cache
TelAPI reads/writes the Media service cache over HTTPS using TTS_MEDIA_CACHE_BASE_URL, TTS_MEDIA_CACHE_UPLOAD_TOKEN, and TTS_MEDIA_CACHE_CA_BUNDLE. The bundle is decoded in memory (base64 or raw PEM) at process startup.
If TLS to the media host fails, check the TelAPI startup line TTS media cache TLS bootstrap (sanitized) — the tlsDiag fields (mediaCacheTlsMode, decodedCaBundleByteLength, decodedCaLooksLikePem) pinpoint the issue without logging secrets. The helper common/check-tts-media-cache-tls.sh --docker voiceai-telapi reproduces the same TLS stack with OpenSSL and curl.
| Name | Source | Scope | Default | Description |
|---|---|---|---|---|
TELAPI_PORT | SSM | all | 3001 | HTTP listen port. |
LOG_LEVEL | SSM | all | info | Application log level. |
DATABASE_URL | Secrets Manager | all | — | PostgreSQL connection string. |
DATABASE_SSL_MODE | SSM | all | disable | disable | require | verify-ca | verify-full. |
REDIS_HOST | SSM | all | — | Redis server address. |
REDIS_PASSWORD | Secrets Manager | all | — | Redis authentication. |
REDIS_TLS_ENABLED | SSM | all | false | Enable TLS to Redis. |
JWT_SECRET | Secrets Manager | all | — | JWT signing key — must match TelWeb NEXTAUTH_SECRET pairings. |
DOMAIN_TELPRO | SSM | all | — | TelPro domain — used to build WebRTC config. |
TTS_MEDIA_CACHE_BASE_URL | SSM | all | — | HTTPS base URL for the Media service. |
TTS_MEDIA_CACHE_UPLOAD_TOKEN | Secrets Manager | all | — | Bearer token for media cache writes / deletes. |
TTS_MEDIA_CACHE_CA_BUNDLE | SSM | all | — | Raw or base64 PEM CA bundle for private media TLS. |
OTEL_ENABLED | SSM | all | true | Enable OpenTelemetry export. |
The authoritative list lives in this service's vars.yaml. Run common/show-env.sh --service api on the host for the full set.
See Configuration model for how SSM / Secrets Manager / TelWeb / DB sources fit together.
| Symptom | Likely cause | Check |
|---|---|---|
| LB targets unhealthy | TelAPI startup crash | docker compose ps; check /health on an instance. |
| 502 from LB | All instances down | At least one healthy instance is required. |
| WebSocket disconnects | Sticky sessions misconfigured | LB sticky sessions must be enabled for WebSocket. |
| 401 auth failures | JWT_SECRET drift between API and TelWeb | Both must read the same Secrets Manager value. |
DEPTH_ZERO_SELF_SIGNED_CERT on media fetch | TTS_MEDIA_CACHE_CA_BUNDLE empty / wrong | Inspect tlsDiag fields in TelAPI startup logs; run common/check-tts-media-cache-tls.sh --docker voiceai-telapi. |
| Postgres connection drops | DB or PgBouncer cycling | See Database operations. |
See also
- Web operations — TelWeb consumer.
- Voice operations — channel-WS counterpart.
- Media operations — TTS cache target.