Quality, dashboards and analytics
How teams judge and measure their apps, from automatic conversation scoring and reviewing single conversations to dashboards, reporting periods, satisfaction metrics and platform-wide usage figures. Browse the feature map or configuration finder.
Availability: Next. Entries that depend on an optional feature, a role or a deployment choice name it in their own Availability line.
Features in this family
- Conversation QA scoring
- QA scoring defaults for the platform and teams
- QA scoring per app
- Review a conversation's QA result
- Scoring job and scoring failures
- Dashboard reporting period
- Dashboard widgets
- Call and usage metrics
- Caller satisfaction on the dashboard
- Conversation history and review
- Platform usage by team and app
Conversation QA scoring
After a call, an AI model rates the conversation against a scoring prompt, for example on goal completion and caller satisfaction, when the plan, team and app all allow it.
Availability: Requires the Conversation QA scoring feature. The QA screens are shown only in development builds of the web app.
For: Team administrators, Application builders, Deployment operators
| Level | Where to change it | Owner section |
|---|---|---|
| Portal and feature profiles | Deployment profile › Conversation QA scoring(qaScoring, FEATURE_QA_SCORING) | Release profiles in the Portal |
Read more:
- Reference: Web UI feature conditions
- Set it up: Configure QA defaults when available
- Set it up: Set application QA scoring
Related features: QA scoring defaults for the platform and teams, QA scoring per app, Review a conversation's QA result, Scoring job and scoring failures.
QA scoring defaults for the platform and teams
Super Admins set the default scoring provider, model, credential reference and prompt, and a team can switch scoring off or replace those defaults with its own.
Availability: Requires the Conversation QA scoring feature. The QA screens are shown only in development builds of the web app.
For: Team administrators, Deployment operators
| Level | Where to change it | Owner section |
|---|---|---|
| Platform settings | Admin Settings › Platform Settings › QA Scoring Defaults(qaScoring, llmDefaults.scoring, scoringAiKey) | Configure AI defaults and check their consumers |
| Team settings | Settings › Team › QA Scoring Defaults(qaScoringEnabled, qaScoringPrompt, qaScoringProvider, qaScoringModel, qaScoringKeyTemplate) | Configure QA defaults when available |
Read more:
- Set it up: Configure QA defaults when available
- Reference: Configure AI defaults and check their consumers
- Reference: Feature-dependent cards and save boundaries
Related features: Conversation QA scoring, QA scoring per app, Platform settings.
QA scoring per app
Each app inherits the team and platform scoring choice, or turns scoring off or sets its own prompt and model.
Availability: Requires the Conversation QA scoring feature. The QA screens are shown only in development builds of the web app.
For: Application builders
| Level | Where to change it | Owner section |
|---|---|---|
| Application settings | App › Flow settings › QA Scoring(qaScoringEnabled, qaScoringPrompt, qaScoringProvider, qaScoringModel) | Set application QA scoring |
Read more:
- Set it up: Set application QA scoring
Related features: QA scoring defaults for the platform and teams, Conversation QA scoring.
Review a conversation's QA result
The conversation detail shows the score, caller satisfaction and any hard failures the scoring found, next to the transcript they are based on.
Availability: Requires the Conversation QA scoring feature. The QA screens are shown only in development builds of the web app.
For: Application builders, Team administrators
| Level | Where to change it | Owner section |
|---|---|---|
| Application settings | Conversations › conversation detail › QA(qa) | Review a QA result |
Read more:
- Set it up: Review a QA result
- Troubleshoot: 5. Resolve a missing result and finish
Related features: Conversation QA scoring, Conversation history and review, Caller satisfaction on the dashboard.
Scoring job and scoring failures
A scheduled background job scores conversations that are still unscored, and its job history tells operators why a conversation stayed without a result.
Availability: Requires the Conversation QA scoring feature.
For: Deployment operators
| Level | Where to change it | Owner section |
|---|---|---|
| Platform settings | Admin Settings › Cron Schedules › conversation-scoring(conversation-scoring, SCORING_CONVERSATIONS) | Configure the scoring job |
| Services and containers | Ops environment(SCORING_OPENAI_BASE_URL, SCORING_OPENAI_MODEL) | Configure the scoring job |
Read more:
- Set it up: Configure the scoring job
- Troubleshoot: QA scoring failures
Related features: Conversation QA scoring, Review a conversation's QA result.
Dashboard reporting period
The dashboard figures cover a chosen period, picked from presets such as today or a custom date range, and the selection can be checked and reset.
For: Team administrators, Application builders
| Level | Where to change it | Owner section |
|---|---|---|
| Team settings | Dashboard › period selector(timeframe) | Choose a preset |
Read more:
- How it works: Understand the dashboard's scope
- Set it up: Choose a preset
- Set it up: Choose specific dates
- Set it up: Check or reset the selection
Related features: Dashboard widgets, Call and usage metrics.
Dashboard widgets
Users add, arrange and configure widgets in edit mode so the dashboard shows the charts and tables relevant to their apps.
For: Team administrators, Application builders
| Level | Where to change it | Owner section |
|---|---|---|
| Team settings | Dashboard › Edit Dashboard(dashboard, widget) | Add and arrange widgets |
| Application settings | App settings › excluded from dashboard metrics(excludeFromDashboard) | Understand the Overview screen |
Read more:
- Set it up: Add and arrange widgets
- Troubleshoot: Investigate what the dashboard shows
Related features: Dashboard reporting period, Call and usage metrics.
Call and usage metrics
Reports count calls, minutes, unique callers, durations per flow, AI tokens, synthesized speech characters and errors over time for the team's apps.
For: Team administrators, Application builders
| Level | Where to change it | Owner section |
|---|---|---|
| Team settings | Dashboard › widget metric(totalCallsByTimeframe, totalCallMinutesByTimeframe, uniqueCallersByApp, tokensByApp, errorsPerAppTimeSeries, totalTtsCharactersByTimeframe) | Read the available metrics |
Read more:
- Reference: Read the available metrics
Related features: Dashboard widgets, Text-to-speech usage over the API, Platform usage by team and app.
Caller satisfaction on the dashboard
Scored conversations feed a satisfaction figure per app, which helps spot apps whose callers leave unhappy.
Availability: Requires the Conversation QA scoring feature.
For: Team administrators, Application builders
| Level | Where to change it | Owner section |
|---|---|---|
| Team settings | Dashboard › Caller Satisfaction widget(satisfactionByApp) | Investigate caller satisfaction |
Read more:
- Troubleshoot: Investigate caller satisfaction
Related features: Conversation QA scoring, Review a conversation's QA result, Call and usage metrics.
Conversation history and review
The Conversations list finds past calls and chats by app, caller or time, and the detail view walks through overview, interaction, errors and logs of one conversation.
For: Application builders, Team administrators
| Level | Where to change it | Owner section |
|---|---|---|
| Team settings | Conversations(conversation) | 1. Find the right conversation |
Read more:
- Set it up: 1. Find the right conversation
- Set it up: 2. Start with Overview, then follow the
- Set it up: 3. Inspect the interaction and errors
- Set it up: Search the logs of a call
Related features: Recording processing and playback, Call transcripts.
Platform usage by team and app
Super Admins compare usage and activity across all teams and apps from one administration page.
Availability: Needs the super-admin or super-viewer role.
For: Deployment operators
| Level | Where to change it | Owner section |
|---|---|---|
| Platform settings | Administration › Usage & Metrics(adminUsage) | Team and platform administration |
Read more:
- Reference: Team and platform administration
Related features: Call and usage metrics.