Voice Evaluators

Voice simulation evaluators assess how your agent sounds, listens, and converses. After a simulated voice interaction completes, evaluators score the agent’s audio output (TTS), transcription quality (STT), and conversational delivery — so you can catch robotic voices, mispronunciations, transcription errors, and unbalanced dialogue before they reach real users.

Why Voice Simulation Evaluators Matter

Voice agents fail in ways text agents do not. Voice simulation evaluators target these failure modes:

Challenge	How Voice Simulation Evaluators Help
Unnatural audio	Verify the agent’s voice sounds human, expressive, and pleasant to listen to
Mispronunciation	Catch incorrect pronunciation of names, jargon, and technical terms
Language drift	Ensure the agent speaks only in its intended language(s)
Transcription errors	Measure how accurately speech-to-text converts user speech
Response lag	Confirm transcripts appear quickly enough for natural conversation
Unbalanced dialogue	Detect when the agent talks too much, too little, or too fast

Evaluators Dashboard

Navigate to Library → Evaluators from the left navigation panel. Switch to the Library tab to browse all preconfigured evaluators. Netra organizes voice simulation evaluators into three categories: TTS, STT, and Conversational.

Library Evaluators

Netra provides 11 preconfigured library evaluators across three voice-specific categories: TTS, STT, and Conversational.

TTS Evaluators

TTS evaluators score the audio your agent produces.

Evaluator	What It Measures
Humanness	How natural and human-like the agent’s voice sounds — tone, rhythm, word clarity, and breathing pauses
Language Accuracy	Whether the agent sticks to the expected language(s) without unintended switches
Pronunciation Accuracy	Whether the agent says words correctly, especially names and technical terms
Speaking Rate / Pace	Whether the agent speaks at a comfortable speed — not too fast, not too slow

STT Evaluators

STT evaluators measure how accurately your pipeline converts user speech to text.

Evaluator	What It Measures
Time to First Transcript	How quickly the first transcript appears after the user starts speaking
Transcription Correctness	Overall transcript accuracy on a 0-1 scale, where 1 means every word was transcribed correctly
Word Error Rate (WER)	How many words the speech-to-text system gets wrong — swapped, missing, or added

Conversational Evaluators

Conversational evaluators assess how the agent behaves across the interaction.

Evaluator	What It Measures
Backchannelling Naturalness	Whether the agent uses natural acknowledgements like “uh-huh” or “I see” at appropriate moments
Expressiveness	Whether the agent’s vocal tone matches the emotional context of the conversation
Noise Robustness	How well the conversation remains understandable and effective with background noise
Speaking Percentage	Whether the agent’s share of talking time stays within a balanced range
Speakability	Whether the agent’s text reads naturally when spoken aloud — catches awkward phrasing, long sentences, and written-only constructs

Evaluator Configuration

Evaluator	Category	Type	Eval Scope	Output	Default Pass Criteria
Humanness	TTS	LALM	Multi-turn	Numerical (0-1)	>= 0.7
Language Accuracy	TTS	LALM	Multi-turn	Pass/Fail	is true
Pronunciation Accuracy	TTS	LALM	Multi-turn	Numerical (0-1)	>= 0.7
Speaking Rate / Pace	TTS	Rule	Multi-turn	Numerical (0-1)	>= 0.7
Time to First Transcript	STT	Rule	Multi-turn	Pass/Fail	is true
Transcription Correctness	STT	LALM	Multi-turn	Numerical (0-1)	>= 0.7
Word Error Rate (WER)	STT	LALM	Multi-turn	Numerical	>= 0.8
Backchannelling Naturalness	Conversational	LALM	Multi-turn	Numerical (0-1)	>= 0.7
Expressiveness	Conversational	LALM	Multi-turn	Numerical (0-1)	>= 0.7
Speaking Percentage	Conversational	Rule	Multi-turn	Pass/Fail	is true
Speakability	Conversational	LLM	Multi-turn	Numerical (0-1)	>= 0.7

LALM (Large Audio Language Model) evaluators listen to the recorded audio and score it with a judge model. Rule evaluators apply deterministic rules against measured values such as duration or latency.

You can adjust the pass criteria threshold for any evaluator based on your requirements. A higher threshold enforces stricter quality standards.

Configurable Parameters

Several voice evaluators accept parameters that you configure during evaluation creation:

Parameter	Evaluator	Format	Default
Expected languages	Language Accuracy	JSON array, e.g. `["English", "Spanish"]`	`["english"]`
Minimum words per second	Speaking Rate / Pace	Number	2.5
Maximum words per second	Speaking Rate / Pace	Number	4
Expected TTFT (seconds)	Time to First Transcript	Number	1
Minimum speaking percentage	Speaking Percentage	Number (%)	30
Maximum speaking percentage	Speaking Percentage	Number (%)	60

Best Practices

Choosing Evaluators by Scenario Type

Scenario Type	Recommended Evaluators
Phone support agents	Humanness, Pronunciation Accuracy, Speaking Percentage
Multilingual voice agents	Language Accuracy, Pronunciation Accuracy
Real-time assistants	Time to First Transcript, Speaking Rate / Pace
Long-form conversations	Expressiveness, Backchannelling Naturalness, Speaking Percentage
Dictation and transcription	Transcription Correctness, Word Error Rate (WER), Time to First Transcript

Getting Started with Evaluators

Start with Humanness and Transcription Correctness — these cover the most critical aspects of any voice simulation
Add scenario-specific evaluators — Language Accuracy for multilingual agents, Time to First Transcript for real-time use cases
Adjust pass criteria if the default thresholds are too lenient or strict for your needs
Monitor results across the first few test runs to ensure evaluators align with your expectations

Combining Voice and Text Evaluators

Voice evaluators assess audio quality, but they don’t evaluate what the agent said. For complete coverage, pair voice evaluators with text evaluators on the same evaluation:

Voice Evaluator (How It Was Said)	Text Evaluator (What Was Said)
Humanness, Pronunciation Accuracy	Answer Correctness, Factual Accuracy
Speaking Rate / Pace	Conversation Completeness
Language Accuracy	Topic Adherence (Guardrails)
Word Error Rate (WER)	Goal Fulfillment (Agentic)

When configuring a voice simulation evaluation, select both voice evaluators and text evaluators. Voice evaluators listen to the audio while text evaluators analyze the transcript — giving you end-to-end evaluation of the interaction.

Evaluators Overview - Understand the full evaluator framework
Text Evaluators - LLM-as-Judge, code, and rule-based text evaluators
Image Evaluators - Multimodal and rule-based image evaluators
Simulation Overview - Understand the full simulation framework
Voice Evaluations - Create voice simulation evaluations
Test Runs - View transcripts and synced call audio

Get Started

Observability

Insights

Evaluations

Simulations

Evaluators

Red Teaming

Prompt Management

Monitoring & Dashboard

Account settings

Migrations

Why Voice Simulation Evaluators Matter

Evaluators Dashboard

Library Evaluators

TTS Evaluators

STT Evaluators

Conversational Evaluators

Evaluator Configuration

Configurable Parameters

Best Practices

Choosing Evaluators by Scenario Type

Getting Started with Evaluators

Combining Voice and Text Evaluators

​Why Voice Simulation Evaluators Matter

​Evaluators Dashboard

​Library Evaluators

​TTS Evaluators

​STT Evaluators

​Conversational Evaluators

​Evaluator Configuration

​Configurable Parameters

​Best Practices

​Choosing Evaluators by Scenario Type

​Getting Started with Evaluators

​Combining Voice and Text Evaluators

​Related

Why Voice Simulation Evaluators Matter

Evaluators Dashboard

Library Evaluators

TTS Evaluators

STT Evaluators

Conversational Evaluators

Evaluator Configuration

Configurable Parameters

Best Practices

Choosing Evaluators by Scenario Type

Getting Started with Evaluators

Combining Voice and Text Evaluators

Related