Log and telemetry obfuscation

Use this page to understand what is obfuscated in logs and telemetry, where each redaction happens, and how to control it. Delphi scrubs personally identifiable information (PII) in three independent layers, so a value missed by one layer is usually caught by another:

1. In-process logger — redactForLogs() in @delphi/logger rewrites every structured log entry before it leaves the service (secrets → [REDACTED], phone numbers and transcripts masked).
2. DTMF-aware helper — logDTMF() replaces DTMF digit / digits field values with * when PII logging is off.
3. OTel collector — transform/pii OTTL processors re-scrub logs and traces inside the collector, just before export to SigNoz.

All three layers read the same two environment variables from each service host's vars.yaml: ENABLE_PII_LOGGING (master switch) and MSISDN_UNMASKED_DIGITS (trailing digits to keep on phone numbers). See the environment variable index for the canonical definitions and Monitoring in SigNoz for the collector-specific operational view.

The master switch: ENABLE_PII_LOGGING

Variable	Default	Meaning
ENABLE_PII_LOGGING	false	When not true, redaction is active. In production, set true only for short support/incident-debug sessions, then revert. Staging/dev may leave it true with test traffic.
MSISDN_UNMASKED_DIGITS	4	Trailing phone-number digits kept when masking MSISDNs (for example +447911123456 → ***3456). Must match on voiceai-telphi, telapi, and every voiceai-otel-collector host.

The switch is a PII debug toggle, never an authorization to emit secrets. Authentication material is redacted unconditionally by the in-process logger, regardless of this flag.

Layer 1 — In-process logger (redactForLogs)

Every structured log emitted by a Delphi service passes through redactForLogs() in packages/logger/src/log-redaction.ts before it reaches Pino / stdout. When ENABLE_PII_LOGGING is not true, it deep-walks the entry and rewrites string leaves.

Secrets — always [REDACTED]

Fields whose normalized name matches a credential list (password, secret, apikey, token, authorization, cookie, connectionstring, databaseurl, …) are replaced with the literal string [REDACTED]. This runs unconditionally — even when PII logging is enabled — because the switch is never an authorization to emit secrets. Secrets embedded in URLs as query parameters (?token=…, ?api_key=…, ?password=…) are also scrubbed from any string leaf.

Phone numbers — masked, trailing digits kept

Fields named callerNumber, calledNumber, msisdn, phoneNumber, transferTarget, target (and the paths from.identifier.name, to.identifier.name) are masked with maskDigits(), keeping the last MSISDN_UNMASKED_DIGITS digits (default 4). Phone-like substrings (+44 7911 123456) and sip: / tel: URIs anywhere in a string are masked the same way. Strings shorter than 5 digits are left alone, so short identifiers are not accidentally destroyed.

Transcripts — [REDACTED]

Fields named text, transcript, response, transcription, textPreview, responsePreview, synthesisPreview, … (and a set of known nested paths such as transcription.text) are replaced with [REDACTED].

Safety guarantees

• Circular references become [Circular] rather than throwing.
• Depth cap (MAX_DEPTH = 16): subtrees deeper than the cap are replaced with [REDACTED] (fail-closed) so deeply nested credentials cannot bypass the walker.
• The original entry can be returned when no value changes, preserving its identity; the redactor still traverses and inspects the data, so this is not zero-cost.

Layer 2 — DTMF digit masking (logDTMF)

DTMF keypad input (the digits a caller presses) is PII. Call sites that log DTMF events use the logDTMF() helper instead of logSystem() / logTelephony(). logDTMF is a sibling of the standard log helpers with one extra behaviour: when ENABLE_PII_LOGGING is not true, it replaces the value of the digit or digits field in metadata before emission.

Field	Input example	Obfuscated output
metadata.digit	"5"	"*"
metadata.digits	"1234#"	"*****"

• digit (single) → a single "*".
• digits (multi) → "*" repeated to the length of the original value, so the masked width still reflects how many digits were collected.
• When helper PII logging is disabled and neither metadata.digit nor metadata.digits is a string, logDTMF emits an error log (No digit/digits field found in metadata) to flag a call-site mistake — digit data should always land in one of these two fields, never interpolated into message.
• The emitted logType defaults to system; call sites route DTMF events to the bucket they belong to (telephony, sandbox, …) by setting logType on the payload.

In this version, the generic redactForLogs classifier has no dedicated DTMF-field or DTMF-message rule. Use logDTMF with a string digit or digits field: it masks digit first, or digits when digit is not a string. Supply the appropriate single field rather than relying on both being masked by the helper. The collector's separate SIP Signal= rules do not protect arbitrary application metadata.

SIP Signal= DTMF

SIP INFO digit payloads can carry Signal=5 lines; this is distinct from audio in-band tones and RTP telephone events. The OTel collector (Layer 3) masks these — see below.

Layer 3 — OTel collector OTTL transforms

Each service host ships a transform/pii processor in its collector config (and Voice additionally ships transform/pii_traces for spans). When ENABLE_PII_LOGGING is not true, the collector re-scrubs logs and traces just before export to SigNoz. This is a defense-in-depth pass on top of Layers 1 and 2. To change these rules yourself — the replacement text, or how many trailing phone digits stay visible — see OTel collector configuration.

The transforms are OTTL replace_pattern / set statements. The notable ones:

DTMF in SIP bodies

The example is Collector configuration syntax: keep doubled backslashes for the OTTL regex string and escape capture references as $$1, $$2. Validate against the selected collector image before activation.

- replace_pattern(attributes["rawMessage"],
  "(?im)(^|\\r?\\n)(Signal\\s*=\\s*)[0-9A-D#*]+", "$$1$$2***")
  where IsString(attributes["rawMessage"]) and attributes["event"] == "sip_message"
  and "${env:ENABLE_PII_LOGGING}" != "true"

Matched SIP Signal= digit lines in captured messages become Signal=***. The same rule applies to attributes["sip.raw"].

MSISDN / phone numbers

Phone-number attributes (callerNumber, calledNumber, phoneNumber, transferTarget, call.caller_number, call.called_number) are masked with the trailing-MSISDN_UNMASKED_DIGITS rule (^.*([0-9]{N})$ → ***$1). E.164-looking substrings inside log bodies are masked the same way.

Free-text PII in bodies

Pattern	Replacement
Email local part (alice@example.com)	****@example.com
Card-shaped number 1234-5678-9012-3456	****-****-****-****
Conversation transcript (logType=conversation)	text → ***, body → conversation (redacted)
textPreview, responsePreview	***

What is not masked

SIP ladder logs retain routing detail rather than anonymizing the complete frame. The log-to-span capture parser always replaces configured sensitive headers in its emitted assembled frame, and masks phone-like URI users and display names when PII logging is disabled. The collector adds conditional Signal= DTMF rules on specific copies. Coverage differs by role and version; see OTel collector configuration. These rules do not guarantee that original container logs, every derived attribute, or all remaining SIP content are scrubbed.

Layer ordering and precedence

call site ──▶ logDTMF (digit/digits → *) ──▶ redactForLogs (secrets/phone/transcript → [REDACTED])
   ──▶ stdout/OTLP ──▶ collector transform/pii (re-scrub) ──▶ SigNoz

• Layer 2 (logDTMF) runs first, inside the helper, so * is already in place before Layer 1.
• Layer 1 (redactForLogs) then scrubs secrets/phones/transcripts but skips DTMF-named fields, preserving the *.
• Layer 3 (collector) applies its configured telemetry rules before export. The capture parser separately redacts selected parts of assembled SIP frames before this layer.

Operational notes

• Production — keep ENABLE_PII_LOGGING=false. Enable it only for short support/incident-debug sessions, then revert and restart the affected Voice/API hosts and collectors.
• Staging / dev — ENABLE_PII_LOGGING=true is acceptable when you need full transcripts in SigNoz and TelWeb for integration testing. Use synthetic or consented test data.
• Set MSISDN_UNMASKED_DIGITS consistently for the intended consumers, then verify each output separately: application, capture-parser and collector rules have different matching and validation behavior.
• PII redaction is not transcript retention. Whether full transcript text is kept long-term is controlled by the app Recording toggle. When recording is on, full text is written to the S3 sidecar regardless of ENABLE_PII_LOGGING. See Conversations → Recording and transcript retention.
• To verify scrubbing: place a test call with PII logging off, then check SigNoz logs/traces and the TelWeb conversation Timeline — transcript text should be ***, caller numbers partially masked (***3456), and SIP Signal= lines should read Signal=***.

Where this is configured

Layer	Code / config
In-process redaction	packages/logger/src/log-redaction.ts (redactForLogs, redactString)
DTMF helper	packages/logger/src/structured-logger.ts (logDTMF)
Collector transforms	otel-collector-config.yaml on each service host (transform/pii, transform/pii_traces) — see OTel collector configuration
Env vars	Environment variable index — ENABLE_PII_LOGGING, MSISDN_UNMASKED_DIGITS
Collector operations	Monitoring in SigNoz — PII redaction