SIP trunk log reference
Use this page when you see a SIP trunk log line in SigNoz or container stdout and need to know what triggered it and which fields to inspect.
Log entry titles are colored by severity: ERROR red, WARN yellow, INFO green, DEBUG blue.
For the SIP signaling ladder and response codes, see SIP signaling reference. For general log levels and SigNoz queries, see Monitoring in SigNoz.
Edge probes and Voice health evaluation
Since v0.9.17-patch1, inbound trunk monitoring uses SIP OPTIONS emitted by the SIP edge. Voice services read the latest shared result and publish health status and alarms. This replaced the AMI endpoint polling described in the v0.9.17 docs. It checks reachability; it does not change routing or prove that a complete call can be established.
Probe configuration
| Setting | Default | Meaning |
|---|---|---|
HEALTH_PROBE_ENABLED | true | Enable edge OPTIONS probing. |
HEALTH_PROBE_INTERVAL | 60 | Seconds between probe cycles. |
HEALTH_PROBE_TARGET_PORT | 5060 | Destination SIP port. |
Each enabled trunk uses its first allowed source-IP entry as the probe target.
A bare IPv4 address is used directly. The current CIDR conversion uses the
address prefix with its final octet replaced by 1; verify this is an actual
carrier endpoint before relying on it. A source allowlist is not necessarily a
carrier's preferred health-check destination. Trunks without a first entry are
skipped with a warning.
The probe uses the SIP transaction timeout, currently 45 seconds. The legacy
HEALTH_PROBE_TIMEOUT example does not control it. HEALTH_PROBE_TARGET_HOST and
HEALTH_PROBE_TRUNK_IDS no longer select targets. AMI credentials are not used by
this monitor; keep them if another integration still requires AMI.
Lifecycle and failures
| Log message or condition | Meaning and action |
|---|---|
Starting SIP trunk health monitor | Voice evaluation is starting; fields include serverId, intervalMs and redisTtlMs. |
SIP trunk health monitor started | Recurring evaluation is scheduled. The initial read does not block startup. |
Stopping SIP trunk health monitor | Recurring evaluation is being stopped. |
Initial poll failed / Poll cycle failed | Check database and shared-result availability; the next poll retries. |
Failed to load SIP trunks from database | Enabled trunk configuration could not be loaded. Check database connectivity. |
| Missing, malformed or stale edge result | The trunk is evaluated as unavailable; check the edge probe, target and shared-result path. |
| No allowed source-IP entry | Configure a usable target in the trunk; no OPTIONS is sent for that entry. |
Status transitions and alarms
Voice evaluation defaults to a 30-second interval, a 60-second result
freshness window and two consecutive failures before an unavailable
transition. Reachable probes exceeding 500 ms are degraded. Inspect the
recorded reason, rttMs, trunkId, serverId and previous/new status when
investigating an alarm. Notification delivery and persistence failures are
reported separately from the reachability result.
A successful OPTIONS response proves SIP reachability only. Validate real call setup, RTP, transfer and teardown separately. A disabled probe, an unsuitable source-address entry or a carrier that refuses OPTIONS can produce unavailable status without proving that its call path is broken.
TelAPI
Source: apps/telapi/src/routes/trunk-health.ts (serves the /health/sip-trunks REST endpoint used by TelWeb and operators).
Failed to scan Redis for trunk health keysERROR
When it fires
The Redis SCAN for voiceai:services:*:sip_trunks keys fails.
What it means
TelAPI cannot reach Redis or the key pattern scan failed. The endpoint returns 503 Redis unavailable and no trunk health data is served. Check Redis connectivity and the getRedisClient() path.
Payload fields
| Field | Value |
|---|---|
err | Error message string. |
Ignoring malformed sip_trunks heartbeat payloadWARN
When it fires
A value stored under a :sip_trunks Redis key is not valid JSON.
What it means
One Voice server's heartbeat payload is corrupt. The key is skipped so the rest of the fleet's data can still be returned. If many keys are malformed, investigate the writing service (TelPhi heartbeat / ServerHeartbeat).
Payload fields
| Field | Value |
|---|---|
key | Full Redis key that contained the bad payload. |
Failed to enrich trunk health from DBERROR
When it fires
The optional database enrichment step (looking up friendly trunk names from SipTrunk) fails.
What it means
The endpoint still returns Redis-derived trunk health, but trunkName fields may be missing. Operators can still identify trunks by trunkId. This is best-effort; the endpoint does not fail the whole request.
Payload fields
| Field | Value |
|---|---|
err | Error message string. |
Tooling
Source: tooling/scripts/import-twilio-numbers.ts. These lines are not structured service logs and do not reach SigNoz; they appear in the terminal when running the import script.
TWILIO_TRUNK_SID not set — skipping SIP configurationERROR
When it fires
The script is run without the optional TWILIO_TRUNK_SID environment variable.
What it means
Phone numbers will be imported into the platform, but Twilio SIP trunk routing will not be configured on them. Set TWILIO_TRUNK_SID if you need the numbers attached to a trunk.
Configuring SIP trunk routing (trunk: {trunkSid})...INFO
When it fires
The script starts attaching imported numbers to the configured Twilio trunk.
What it means
The SIP routing step is beginning. Each number is checked against the trunk's current members and only missing ones are added.
{phoneNumber} — already on trunk, skippingINFO
When it fires
A target number is already associated with the configured Twilio trunk.
What it means
No change was made for this number. This is expected on re-runs of the script.
{phoneNumber} — SIP trunk configuredINFO
When it fires
A target number was successfully added to the configured Twilio trunk.
What it means
Inbound calls to this Twilio number will now route to the platform via the configured trunk SID.
Where to look
| Surface | Best for |
|---|---|
| SigNoz Logs | Filter attributes.alarm.name = "sip_trunk_health" or attributes.logType = "sip_trunk_health" for status transitions and connection failures. |
| TelWeb → platform admin → SIP trunks | Trunk configuration and historical health snapshots. |
TelAPI /trunk-health | Real-time fleet-wide trunk status aggregated from Redis heartbeats. |
| Voice container stdout | docker compose logs voiceai-telphi when SigNoz is unavailable. |
Useful SigNoz queries
attributes.alarm.name = "sip_trunk_health" AND attributes.alarm.state = "unavailable"
attributes.logType = "sip_trunk_health" AND attributes.alarm.resource.id = "twilio_us"
service.name = "telapi" AND attributes.msg = "Failed to scan Redis for trunk health keys"
See also
- SIP signaling reference — SIP methods, response codes, and capture points.
- Monitoring in SigNoz — structured logging schema, per-service log levels, PII redaction, and dashboards.
- Application and call-flow logging — call lifecycle logs from TelSys, TelPro, and RTPEngine.
- Voice operations — TelSys, ARI, and Voice host operations.