Skip to main content
Version: 0.9.18

Operate the voice services (TelSys, TelPhi, AudioProc)

Use this page to keep a Voice host healthy. A Voice host answers the calls that TelPro forwards, runs the conversation flow for each call and reports its capacity back to the edge.

Availability: From 0.9.18, the TelSys image raises its ARI limits by default on every Voice host: ARIHTTPSESSIONLIMIT is 1000 and ARIWEBSOCKETWRITETIMEOUT is 1000 ms. No flag is needed, and the standard Voice bundle does not override them. Before you change either value, read ARI session limits.

Roles and the ARI connection​

All Voice containers use host networking, restart automatically unless you stop them, and run from /opt/services/voice.

ContainerComponentRole
voiceai-telsysTelSys (Asterisk)Accepts SIP from TelPro, handles RTP and exposes ARI and the manager interface
voiceai-telphiTelPhiControls calls through ARI, runs flows, connects AI and speech providers
voiceai-audioprocAudioProcOptional caller-audio processing
voiceai-otel-collectorTelemetry collectorExports Voice telemetry to SigNoz
log-to-spanLog converterTurns structured log events into trace spans

TelSys accepts calls only from the TelPro addresses in VAIPROXIESPIPED, on SIPPORTTELPRO. It runs with a raised open-file limit, because each media leg holds several file handles.

TelPhi starts after TelSys reports healthy. It then connects to ARI on the same host, by default at http://127.0.0.1:8088, and registers its Stasis application. For each call leg, TelSys opens a media WebSocket to TelPhi on TELPHI_MEDIA_PORT (default 12001).

Kamailio hands the call to TelSys, TelSys reports it to TelPhi over ARI, and the call audio then flows between TelSys and TelPhi over the media WebSocket.

If the ARI WebSocket drops and TelPhi cannot reconnect, TelPhi shuts down with exit code 1. Docker then restarts the container, and TelPhi reconnects from a clean state.

ARI session limits​

The Voice variables reference defines both limits and their defaults. These are the symptoms when a limit is too low:

  • ARIHTTPSESSIONLIMIT too low: TelSys logs HTTP session count exceeded, and rejected ARI requests leave answered calls without a bot. The upstream Asterisk default is 100.
  • ARIWEBSOCKETWRITETIMEOUT too low: A short TelPhi pause closes the ARI WebSocket. The Stasis application goes away, and new calls are declined with 603. The upstream Asterisk default is 100 ms.

The standard Voice Compose file does not pass these variables, so the image defaults apply. To change them, add them to the TelSys service in a reviewed Voice bundle change, publish the bundle and refresh the Voice hosts. Do not lower them below the defaults on a host that takes bursts of calls.

Heartbeat and call admission​

Every 10 seconds, TelPhi publishes a heartbeat for its Voice host to Redis and emits a telphi.heartbeat trace span. The span carries the server ID, status, active calls, used RTP ports and the interval.

Kamailio reads these heartbeats when it chooses a Voice host for a new call:

  • It skips a host whose heartbeat is 60 seconds old or older, or whose status is not active.
  • When the whole server group is at its limit, it answers 486 Busy Here.
  • When the selected host has no free capacity, it answers 503.

When TelPhi stops normally, it sends a final heartbeat with status offline, so Kamailio stops sending new calls to that host straight away. It also stops the SIP trunk health monitor and the usage monitor.

No-traces alert​

The TelphiNoTraces alert fires with severity critical when SigNoz receives no traces from the telphi service for 10 minutes. SigNoz evaluates it every minute.

TelPhi emits its heartbeat span even when no call is active. A firing alert therefore means one of these:

  • The TelPhi process is down or restarting repeatedly.
  • The local telemetry collector on the Voice host is not running or cannot reach SigNoz.
  • The collector pipeline in SigNoz is broken.

Check docker compose ps on each Voice host first. For alert setup and the collector pipeline, see Telemetry and monitoring.

Media connection timeouts​

The Voice host monitors incoming caller audio on each call leg. It plays a warning and then disconnects after the configured intervals without audio. Configure the warning in Configure loss-of-audio warnings and diagnose a dropped call with Diagnose a caller media timeout.

AudioProc​

AudioProc processes caller audio before TelPhi uses it. TelPhi sends audio to AudioProc only when AUDIOPROC_ENABLED is true; the container runs either way. Its health check calls http://localhost:8790/health every 15 seconds. The Environment variable reference lists its address, port and device settings.

Restart and drain a Voice host​

A direct container or host restart ends the calls that run on that host. For planned changes, drain the host first:

  • To apply configuration or image changes, queue a TelPhi refresh in Service Refresh. The drain-and-restart strategy replaces Voice hosts in a rolling sequence.
  • To understand the capacity needed during a refresh and how Voice scale-down drains a host, see Maintain capacity during a refresh.

If you must restart by hand, run ./init.sh in /opt/services/voice on a host without active calls. Initialization waits until TelSys reports healthy.

Read Voice logs​

cd /opt/services/voice
docker compose logs -f voiceai-telphi

Replace voiceai-telphi with voiceai-telsys, voiceai-audioproc or otel-collector for the other components.

  • Docker keeps SIP_CAPTURE_TELSYS_LOG_MAX_FILES files of SIP_CAPTURE_TELSYS_LOG_MAX_SIZE each for TelSys (default 5 files of 100 MB).
  • LOG_INTEGRATION_PAYLOADS adds integration payloads to TelPhi transaction records. Redaction still applies. See Protect diagnostic logs before you turn it on.
  • To find one call's records, see Logs and Log records.

Verify the Voice host​

  1. Run docker compose ps in /opt/services/voice and confirm that voiceai-telsys and voiceai-audioproc are healthy and voiceai-telphi is running.
  2. In SigNoz, confirm that telphi.heartbeat spans arrive from this host every 10 seconds.
  3. Place a test call and confirm that the flow answers and both sides hear audio.