Voice service operations

The Voice service runs the actual voice-AI processing: TelSys (Asterisk 22.9.0) provides PBX and media handling, TelPhi (Node 24) orchestrates AI conversations via realtime and modular speech providers, and AudioProc does optional VAD / noise reduction. SIP calls arrive from TelPro and are processed here. Multiple Voice instances run in parallel for horizontal scaling.

Overview

Instance

Instance vCPU and RAM are in Provision instances. The reference size (3 vCPU / 4 GB RAM) targets ~100 concurrent calls per Voice instance — provision additional instances (manually or via the Ops Scaler) for higher load.

Field	Value
Network	Private only — no public IP; traffic arrives via managed load balancer
Scalable	Yes — set initial count in your provisioning tool; the Ops Scaler can adjust count dynamically

Containers

Container	Base	Port(s)	Purpose
voiceai-telsys	Asterisk 22.9.0 (built from source on debian:bookworm-slim)	5080 SIP · 8088 ARI · 10000–11999 RTP	Asterisk PBX / media
voiceai-telphi	Node 24-alpine	12001 media WS	AI conversation engine
voiceai-audioproc	python:3.11-slim (CPU) or nvidia/cuda:12.4.1-runtime-ubuntu22.04 (GPU)	8790 WS	Audio preprocessing (optional)
log-to-span	alpine:3.19 + Go binary	—	Converts TelSys structured logs to OTLP spans
voiceai-otel-collector	otel/opentelemetry-collector-contrib:0.150.1	—	Telemetry collector

AudioProc is the optional audio-preprocessing sidecar in the Voice stack. When the audioPreprocessing feature flag is off, the generated Compose file drops voiceai-audioproc and TelPhi runs without that preprocessing hop. When it is on, TelPhi can send inbound audio frames through AudioProc before they reach the configured speech or realtime AI provider, mainly to improve noisy carrier audio with VAD / noise reduction.

Voice stack baseline

Asterisk upgraded to 22.9.0 with missing slin RTP codec entries fixed. chan_websocket enables WebSocket-oriented voice infrastructure. TelPhi media-session handling improved; WebRTC start-call no longer requires two clicks.

Call flow

1. TelPro routes an INVITE to TelSys (VAIPROXIESPIPED:SIPPORTTELPRO).
2. TelSys prepares the call and bridges to TelPhi via Asterisk ExternalMedia over the TelPhi media WebSocket (TELPHI_MEDIA_PORT, default 12001).
3. TelPhi prepares the configured AI provider and media path. In v0.9.13, inbound readiness policy can delay answer until the provider and media path are ready.
4. AudioProc (when AUDIOPROC_ENABLED=true) preprocesses inbound audio frames.
5. Managed actions such as transfer, hangup, and failover flow through TelPhi and TelSys, then route out through TelPro when they need SIP egress.

Managed telephony actions

v0.9.13 unifies transfer, hangup, failover, and inbound-not-ready behavior across Flow Builder nodes, managed LLM callbacks, and sandbox/runtime commands. Operationally this means:

• TelPhi validates and executes the call action.
• TelSys performs the Asterisk-side bridge, REFER, BYE, reinvite, or disconnect behavior.
• TelPro handles SIP egress, outbound routing, and failover toward the carrier or PBX.
• Transfer restrictions are checked before a managed transfer is attempted.

When debugging a managed action, correlate TelPhi application logs, TelSys PJSIP logs, and TelPro SIP ladder spans for the same call ID.

RFC 3262 (100rel / PRACK)

SIP_100REL_MODE on TelSys (voiceai-telsys) sets the PJSIP endpoint 100rel option for the TelPro-facing leg. It controls whether Asterisk sends reliable provisional responses (180/183 with Require: 100rel) and accepts PRACK from the peer.

Value	Meaning
off	No PRACK / 100rel. Default — use when the carrier does not require reliable provisionals.
peer_supported	Send reliable 180/183 only when the peer advertises Supported: 100rel. Start here when a carrier rejects early media or expects PRACK.
yes	Always use reliable provisionals on this endpoint, regardless of peer capability.
required	Require 100rel on the endpoint (strictest). Use only when the carrier mandates PRACK on every call.

TelSys also accepts common synonyms (no / disabled / false for off; on / true / enabled for yes; and similar). An unknown value falls back to peer_supported with a warning in TelSys logs.

Kamailio on TelPro relays in-dialog PRACK and labels reliable 1xx / PRACK in the Debug SIP ladder. Configure SIP_100REL_MODE on the Voice service env bundle (vars.yaml / SSM).

Runbook

Scaling

Set the initial Voice instance count in your provisioning tool. The Ops Scaler can adjust this dynamically based on active call count. Instances are named ${environment}-voice-01, -02, etc.

When the Scaler creates a new Voice instance, it appears in the Kamailio dispatcher set in Redis (voiceai:dispatcher:*) and starts receiving calls automatically.

Deploy and update

cd /opt/services/voice
./init.sh
./update.sh --ecr-tag v0.9.13
./update.sh --restart-only

init.sh copies the log-to-span binary into the right path, fetches config from S3, loads env vars from SSM / SM, logs into ECR, and starts Compose. Standard flags apply.

RTP port allocation

Component	Range
TelSys (carrier-facing)	10000–11999
TelPhi	media WebSocket only (no separate RTP range)
Janus (on TelPro)	10000–12000
RTPEngine (on TelPro)	20000–40000

RTP range

Adjust the carrier-facing range via RTPSTART / RTPEND in SSM if your upstream carrier needs a different window.

Recording

When RECORDING_ENABLED=true, TelPhi writes recordings to the S3 bucket RECORDING_BUCKET using AWS credentials available to the container. Recording is post-processed by Tasker on the Ops service.

Configuration

TelSys (Asterisk)

Name	Source	Scope	Default	Description
VAIPROXIESPIPED	SSM	all	—	Kamailio SIP proxy (TelPro private IP).
SIP_PORT_INTERNAL	SSM	all	—	Internal SIP port (from TelPro).
SIPPORTTELPRO	SSM	all	—	TelPro SIP port TelSys dials out to.
RTPSTART	SSM	all	10000	Carrier-facing RTP start.
RTPEND	SSM	all	11999	Carrier-facing RTP end.

TelPhi (Voice AI)

Highlights — full list in vars.yaml / show-env.sh --service voice

Name	Source	Scope	Default	Description
DATABASE_URL	Secrets Manager	all	—	PostgreSQL connection.
DATABASE_POOL_SIZE	SSM	all	50	Connection pool size.
DATABASE_SSL_MODE	SSM	all	disable	Postgres client TLS: disable | require | verify-ca | verify-full.
DATABASE_SSL_CA_BUNDLE_B64	Secrets Manager	all	—	Base64 PEM CA for TelPhi Postgres TLS (same semantics as API).
DATABASE_SSL_CA_FILE	env	all	/etc/ssl/database/ca.crt	Decoded DB CA path in the TelPhi container.
REDIS_HOST	SSM	all	—	Redis server.
REDIS_PASSWORD	Secrets Manager	all	—	Redis auth.
REDIS_TLS_ENABLED	SSM	all	false	Enable TLS to Redis.
REDIS_TLS_CA_BUNDLE_B64	Secrets Manager	all	—	Base64 PEM CA for Redis TLS; optional for Amazon-signed Redis.
REDIS_TLS_CA_FILE	env	all	/etc/ssl/redis/ca.crt	Decoded Redis CA path in the TelPhi container.
TELPHI_MEDIA_PORT	SSM	all	12001	TCP port for ExternalMedia WS.
TELPHI_MEDIA_PATH	SSM	all	/media	URL path Asterisk dials.
TELPHI_MEDIA_SCHEME	SSM	all	ws	ws or wss.
RECORDING_ENABLED	SSM	all	false	Enable S3 call recording.
RECORDING_BUCKET	SSM	all	recordings	S3 recordings bucket.
AUDIOPROC_ENABLED	SSM	all	false	Toggle AudioProc preprocessor.
SIP_100REL_MODE	env	all	off	TelSys PJSIP 100rel mode: off (default) | peer_supported | yes | required. Controls PRACK / reliable provisional responses toward TelPro. See Voice operations — RFC 3262 (100rel / PRACK).
SIP_CAPTURE_ENABLE_PJSIP_LOGGER	env	all	1	Enables detailed PJSIP message logging for SIP ladder and log-to-span diagnostics. Review log volume before enabling in high-traffic environments.
ENABLE_PII_LOGGING	env	all	false	When not true, TelPhi redacts transcript text and caller numbers before DB persist; voiceai-otel-collector scrubs exported logs/traces. Production: false except short support sessions; staging/dev may use true for debugging.
MSISDN_UNMASKED_DIGITS	env	all	4	Trailing digits kept when masking caller numbers (TelPhi + collector). Set on voiceai-telphi and voiceai-otel-collector.
TTS_MEDIA_CACHE_BASE_URL	SSM	all	—	HTTPS base URL for Media service.
TTS_MEDIA_CACHE_UPLOAD_TOKEN	Secrets Manager	all	—	Bearer for media writes.
TTS_MEDIA_CACHE_CA_BUNDLE	SSM	all	—	CA bundle for private media TLS.
HTTP_PROXY	SSM	all	—	Squid proxy for outbound HTTP (OpenAI, etc.).
HTTPS_PROXY	SSM	all	—	Squid proxy for outbound HTTPS.
VONAGE_SMS_API_KEY	Secrets Manager	all	—	Vonage SMS API key (when SMS feature enabled).

Troubleshooting

Call processing

Symptom	Likely cause	Check
Calls ring but no AI response	TelPhi not connected to Asterisk ARI	TelPhi logs for ARI connection errors.
TelSys won't start (90s timeout)	Asterisk config error	docker compose logs voiceai-telsys.
TelPhi can't reach OpenAI	Proxy misconfigured or Squid down	Verify HTTP_PROXY / HTTPS_PROXY; check Squid.
Call drops after ~30s	Realtime API timeout / WS drop	TelPhi logs for WebSocket close; proxy stability.
No SIP from TelPro	VAIPROXIESPIPED wrong	Verify matches TelPro private IP.
Caller hears no AI audio	TelSys can't reach TelPhi media WS	Tail TelPhi for media_session connect; check chan_websocket errors in TelSys.

Transfer to human

Symptom	Cause	Check
Transfer not initiating	transfer() tool not on agent, AI never calls it	Agent tools in TelWeb; TelPhi logs for tool invocations.
Outbound leg rings, no answer	Target unreachable / no egress trunk	TelPhi logs for SIP response codes; egress trunk linked to the base number.
Mini AI session silent	ExternalMedia bridge / RTP wiring failed	Look for Mini session audio wired.
Transfer blocked	Transfer restriction or egress policy denied it	Check flow/team transfer restrictions and TelPhi action logs.
Carrier rejects early media	100rel / PRACK mode mismatch	Set SIP_100REL_MODE to peer_supported (see Overview → RFC 3262); check TelSys PJSIP logs and the Debug SIP ladder.

See also

• TelPro operations — SIP / WebRTC entry point.
• Database operations — Redis keys used for routing.
• Monitoring in SigNoz — trace IDs, call-ID correlation, logs, and dashboards.