Log and telemetry obfuscation

Use this page to understand what is obfuscated in logs and telemetry, where each redaction happens, and how to control it. Delphi scrubs personally identifiable information (PII) in three independent layers, so a value missed by one layer is usually caught by another:

1. In-process logger — redactForLogs() in @delphi/logger rewrites every structured log entry before it leaves the service (secrets → [REDACTED], phone numbers and transcripts masked).
2. DTMF-aware helper — logDTMF() replaces DTMF digit / digits field values with * when PII logging is off.
3. OTel collector — transform/pii OTTL processors re-scrub logs and traces inside the collector, just before export to SigNoz.

All three layers read the same two environment variables from each service host's vars.yaml: ENABLE_PII_LOGGING (master switch) and MSISDN_UNMASKED_DIGITS (trailing digits to keep on phone numbers). See the environment variable index for the canonical definitions and Monitoring in SigNoz for the collector-specific operational view.

The master switch: ENABLE_PII_LOGGING

Variable	Default	Meaning
ENABLE_PII_LOGGING	false	When not true, redaction is active. In production, set true only for short support/incident-debug sessions, then revert. Staging/dev may leave it true with test traffic.
MSISDN_UNMASKED_DIGITS	4	Trailing phone-number digits kept when masking MSISDNs (for example +447911123456 → ***3456). Must match on voiceai-telphi, telapi, and every voiceai-otel-collector host.

The switch is a PII debug toggle, never an authorization to emit secrets. Authentication material is redacted unconditionally by the in-process logger, regardless of this flag.

Layer 1 — In-process logger (redactForLogs)

Every structured log emitted by a Delphi service passes through redactForLogs() in packages/logger/src/log-redaction.ts before it reaches Pino / stdout. When ENABLE_PII_LOGGING is not true, it deep-walks the entry and rewrites string leaves.

Secrets — always [REDACTED]

Fields whose normalized name matches a credential list (password, secret, apikey, token, authorization, cookie, connectionstring, databaseurl, …) are replaced with the literal string [REDACTED]. This runs unconditionally — even when PII logging is enabled — because the switch is never an authorization to emit secrets. Secrets embedded in URLs as query parameters (?token=…, ?api_key=…, ?password=…) are also scrubbed from any string leaf.

Phone numbers — masked, trailing digits kept

Fields named callerNumber, calledNumber, msisdn, phoneNumber, transferTarget, target (and the paths from.identifier.name, to.identifier.name) are masked with maskDigits(), keeping the last MSISDN_UNMASKED_DIGITS digits (default 4). Phone-like substrings (+44 7911 123456) and sip: / tel: URIs anywhere in a string are masked the same way. Strings shorter than 5 digits are left alone, so short identifiers are not accidentally destroyed.

Transcripts — [REDACTED]

Fields named text, transcript, response, transcription, textPreview, responsePreview, synthesisPreview, … (and a set of known nested paths such as transcription.text, content.textmessage.textPlain) are replaced with [REDACTED].

Safety guarantees

• Circular references become [Circular] rather than throwing.
• Depth cap (MAX_DEPTH = 16): subtrees deeper than the cap are replaced with [REDACTED] (fail-closed) so deeply nested credentials cannot bypass the walker.
• The original entry is returned unchanged when no field needed redaction, so redaction is zero-cost for clean logs.

Layer 2 — DTMF digit masking (logDTMF)

DTMF keypad input (the digits a caller presses) is PII. Call sites that log DTMF events use the logDTMF() helper instead of logSystem() / logTelephony(). logDTMF is a sibling of the standard log helpers with one extra behaviour: when ENABLE_PII_LOGGING is not true, it replaces the value of the digit or digits field in metadata before emission.

Field	Input example	Obfuscated output
metadata.digit	"5"	"*"
metadata.digits	"1234#"	"****"

• digit (single) → a single "*".
• digits (multi) → "*" repeated to the length of the original value, so the masked width still reflects how many digits were collected.
• If metadata has neither field, logDTMF emits an error log (No digit/digits field found in metadata) to flag a call-site mistake — digit data should always land in one of these two fields, never interpolated into message.
• The emitted logType defaults to system; call sites route DTMF events to the bucket they belong to (telephony, sandbox, …) by setting logType on the payload.

Because logDTMF writes * rather than [REDACTED], the in-process redactForLogs layer intentionally does not redact DTMF-named fields — otherwise it would overwrite logDTMF's * with [REDACTED]. DTMF digit obfuscation is the sole responsibility of logDTMF.

SIP Signal= DTMF

In SIP message bodies, in-band DTMF is carried as Signal=5 lines. The OTel collector (Layer 3) masks these — see below.

Layer 3 — OTel collector OTTL transforms

Each service ships a transform/pii (and on Voice, transform/pii_traces) processor in its collector config (.docker/otel-collector/otel-collector-config.*.yaml, per-service configs in .infrastructure/services/*/otel-collector-config.yaml). When ENABLE_PII_LOGGING is not true, the collector re-scrubs logs and traces just before export to SigNoz. This is a defense-in-depth pass on top of Layers 1 and 2.

The transforms are OTTL replace_pattern / set statements. The notable ones:

DTMF in SIP bodies

- replace_pattern(attributes["rawMessage"],
  "(?im)(^|\r?\n)(Signal\s*=\s*)[0-9A-D#*]+", "$1$2***")
  where IsString(attributes["rawMessage"]) and attributes["event"] == "sip_message"
  and "${env:ENABLE_PII_LOGGING}" != "true"

In-band DTMF Signal= lines in captured SIP messages become Signal=***. The same rule applies to attributes["sip.raw"].

MSISDN / phone numbers

Phone-number attributes (callerNumber, calledNumber, phoneNumber, transferTarget, call.caller_number, call.called_number) are masked with the trailing-MSISDN_UNMASKED_DIGITS rule (^.*([0-9]{N})$ → ***$1). E.164-looking substrings inside log bodies are masked the same way.

Free-text PII in bodies

Pattern	Replacement
Email local part (alice@example.com)	****@example.com
Card-shaped number 1234-5678-9012-3456	****-****-****-****
Conversation transcript (logType=conversation)	text → ***, body → conversation (redacted)
textPreview, responsePreview	***

What is not masked

SIP ladder logs (event=sip_message) keep their raw SIP text except for the Signal= DTMF rule above. Scrubbing the full SIP body would break call-routing analysis in the ladder view — only the DTMF signal is redacted there.

Layer ordering and precedence

call site ──▶ logDTMF (digit/digits → *) ──▶ redactForLogs (secrets/phone/transcript → [REDACTED])
   ──▶ stdout/OTLP ──▶ collector transform/pii (re-scrub) ──▶ SigNoz

• Layer 2 (logDTMF) runs first, inside the helper, so * is already in place before Layer 1.
• Layer 1 (redactForLogs) then scrubs secrets/phones/transcripts but skips DTMF-named fields, preserving the *.
• Layer 3 (collector) is a final belt-and-suspenders pass for anything that slipped through (and is the only layer that touches SIP bodies).

Operational notes

• Production — keep ENABLE_PII_LOGGING=false. Enable it only for short support/incident-debug sessions, then revert and restart the affected Voice/API hosts and collectors.
• Staging / dev — ENABLE_PII_LOGGING=true is acceptable when you need full transcripts in SigNoz and TelWeb for integration testing. Use synthetic or consented test data.
• Set MSISDN_UNMASKED_DIGITS consistently on Voice, API, and every collector host — mismatched values make SigNoz and the DB timeline disagree.
• PII redaction is not transcript retention. Whether full transcript text is kept long-term is controlled by the app Recording toggle. When recording is on, full text is written to the S3 sidecar regardless of ENABLE_PII_LOGGING. See Conversations → Recording and transcript retention.
• To verify scrubbing: place a test call with PII logging off, then check SigNoz logs/traces and the TelWeb conversation Timeline — transcript text should be ***, caller numbers partially masked (***3456), and SIP Signal= lines should read Signal=***.

Where this is configured

Layer	Code / config
In-process redaction	packages/logger/src/log-redaction.ts (redactForLogs, redactString)
DTMF helper	packages/logger/src/structured-logger.ts (logDTMF)
Collector transforms	.docker/otel-collector/otel-collector-config.*.yaml, .infrastructure/services/*/otel-collector-config.yaml (transform/pii, transform/pii_traces)
Env vars	Environment variable index — ENABLE_PII_LOGGING, MSISDN_UNMASKED_DIGITS
Collector operations	Monitoring in SigNoz — PII redaction