> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getnetra.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Voice Evaluators
> Assess voice AI simulations in Netra using STT, TTS, and conversational evaluators. Measure audio naturalness, transcription accuracy, latency, and speaking behavior.
Voice simulation evaluators assess how your agent sounds, listens, and converses. After a simulated voice interaction completes, evaluators score the agent's audio output (TTS), transcription quality (STT), and conversational delivery — so you can catch robotic voices, mispronunciations, transcription errors, and unbalanced dialogue before they reach real users.
## Why Voice Simulation Evaluators Matter
Voice agents fail in ways text agents do not. Voice simulation evaluators target these failure modes:
| Challenge | How Voice Simulation Evaluators Help |
| -------------------- | ---------------------------------------------------------------------------- |
| Unnatural audio | Verify the agent's voice sounds human, expressive, and pleasant to listen to |
| Mispronunciation | Catch incorrect pronunciation of names, jargon, and technical terms |
| Language drift | Ensure the agent speaks only in its intended language(s) |
| Transcription errors | Measure how accurately speech-to-text converts user speech |
| Response lag | Confirm transcripts appear quickly enough for natural conversation |
| Unbalanced dialogue | Detect when the agent talks too much, too little, or too fast |
## Evaluators Dashboard
Navigate to **Library → Evaluators** from the left navigation panel. Switch to the **Library** tab to browse all preconfigured evaluators.
Netra organizes voice simulation evaluators into three categories: **TTS**, **STT**, and **Conversational**.
## Library Evaluators
Netra provides 11 preconfigured library evaluators across three voice-specific categories: **TTS**, **STT**, and **Conversational**.
### TTS Evaluators
TTS evaluators score the audio your agent produces.
| Evaluator | What It Measures |
| -------------------------- | ------------------------------------------------------------------------------------------------------ |
| **Humanness** | How natural and human-like the agent's voice sounds — tone, rhythm, word clarity, and breathing pauses |
| **Language Accuracy** | Whether the agent sticks to the expected language(s) without unintended switches |
| **Pronunciation Accuracy** | Whether the agent says words correctly, especially names and technical terms |
| **Speaking Rate / Pace** | Whether the agent speaks at a comfortable speed — not too fast, not too slow |
### STT Evaluators
STT evaluators measure how accurately your pipeline converts user speech to text.
| Evaluator | What It Measures |
| ----------------------------- | ---------------------------------------------------------------------------------------------- |
| **Time to First Transcript** | How quickly the first transcript appears after the user starts speaking |
| **Transcription Correctness** | Overall transcript accuracy on a 0-1 scale, where 1 means every word was transcribed correctly |
| **Word Error Rate (WER)** | How many words the speech-to-text system gets wrong — swapped, missing, or added |
### Conversational Evaluators
Conversational evaluators assess how the agent behaves across the interaction.
| Evaluator | What It Measures |
| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| **Backchannelling Naturalness** | Whether the agent uses natural acknowledgements like "uh-huh" or "I see" at appropriate moments |
| **Expressiveness** | Whether the agent's vocal tone matches the emotional context of the conversation |
| **Noise Robustness** | How well the conversation remains understandable and effective with background noise |
| **Speaking Percentage** | Whether the agent's share of talking time stays within a balanced range |
| **Speakability** | Whether the agent's text reads naturally when spoken aloud — catches awkward phrasing, long sentences, and written-only constructs |
## Evaluator Configuration
| Evaluator | Category | Type | Eval Scope | Output | Default Pass Criteria |
| --------------------------- | -------------- | ---- | ---------- | --------------- | --------------------- |
| Humanness | TTS | LALM | Multi-turn | Numerical (0-1) | >= 0.7 |
| Language Accuracy | TTS | LALM | Multi-turn | Pass/Fail | is true |
| Pronunciation Accuracy | TTS | LALM | Multi-turn | Numerical (0-1) | >= 0.7 |
| Speaking Rate / Pace | TTS | Rule | Multi-turn | Numerical (0-1) | >= 0.7 |
| Time to First Transcript | STT | Rule | Multi-turn | Pass/Fail | is true |
| Transcription Correctness | STT | LALM | Multi-turn | Numerical (0-1) | >= 0.7 |
| Word Error Rate (WER) | STT | LALM | Multi-turn | Numerical | >= 0.8 |
| Backchannelling Naturalness | Conversational | LALM | Multi-turn | Numerical (0-1) | >= 0.7 |
| Expressiveness | Conversational | LALM | Multi-turn | Numerical (0-1) | >= 0.7 |
| Speaking Percentage | Conversational | Rule | Multi-turn | Pass/Fail | is true |
| Speakability | Conversational | LLM | Multi-turn | Numerical (0-1) | >= 0.7 |
**LALM** (Large Audio Language Model) evaluators listen to the recorded audio and score it with a judge model. **Rule** evaluators apply deterministic rules against measured values such as duration or latency.
You can adjust the pass criteria threshold for any evaluator based on your requirements. A higher threshold enforces stricter quality standards.
## Configurable Parameters
Several voice evaluators accept parameters that you configure during evaluation creation:
| Parameter | Evaluator | Format | Default |
| --------------------------- | ------------------------ | ----------------------------------------- | ------------- |
| Expected languages | Language Accuracy | JSON array, e.g. `["English", "Spanish"]` | `["english"]` |
| Minimum words per second | Speaking Rate / Pace | Number | 2.5 |
| Maximum words per second | Speaking Rate / Pace | Number | 4 |
| Expected TTFT (seconds) | Time to First Transcript | Number | 1 |
| Minimum speaking percentage | Speaking Percentage | Number (%) | 30 |
| Maximum speaking percentage | Speaking Percentage | Number (%) | 60 |
## Best Practices
### Choosing Evaluators by Scenario Type
| Scenario Type | Recommended Evaluators |
| --------------------------- | -------------------------------------------------------------------------- |
| Phone support agents | Humanness, Pronunciation Accuracy, Speaking Percentage |
| Multilingual voice agents | Language Accuracy, Pronunciation Accuracy |
| Real-time assistants | Time to First Transcript, Speaking Rate / Pace |
| Long-form conversations | Expressiveness, Backchannelling Naturalness, Speaking Percentage |
| Dictation and transcription | Transcription Correctness, Word Error Rate (WER), Time to First Transcript |
### Getting Started with Evaluators
1. **Start with Humanness and Transcription Correctness** — these cover the most critical aspects of any voice simulation
2. **Add scenario-specific evaluators** — Language Accuracy for multilingual agents, Time to First Transcript for real-time use cases
3. **Adjust pass criteria** if the default thresholds are too lenient or strict for your needs
4. **Monitor results** across the first few test runs to ensure evaluators align with your expectations
## Combining Voice and Text Evaluators
Voice evaluators assess audio quality, but they don't evaluate what the agent said. For complete coverage, pair voice evaluators with [text evaluators](/Evaluators/text-evaluators) on the same evaluation:
| Voice Evaluator (How It Was Said) | Text Evaluator (What Was Said) |
| --------------------------------- | ------------------------------------ |
| Humanness, Pronunciation Accuracy | Answer Correctness, Factual Accuracy |
| Speaking Rate / Pace | Conversation Completeness |
| Language Accuracy | Topic Adherence (Guardrails) |
| Word Error Rate (WER) | Goal Fulfillment (Agentic) |
When configuring a voice simulation evaluation, select both voice evaluators and text evaluators. Voice evaluators listen to the audio while text evaluators analyze the transcript — giving you end-to-end evaluation of the interaction.
## Related
* [Evaluators Overview](/Evaluators/overview) - Understand the full evaluator framework
* [Text Evaluators](/Evaluators/text-evaluators) - LLM-as-Judge, code, and rule-based text evaluators
* [Image Evaluators](/Evaluators/image-evaluators) - Multimodal and rule-based image evaluators
* [Simulation Overview](/Simulations/Simulation-overview) - Understand the full simulation framework
* [Voice Evaluations](/Simulations/voice-simulations/Simulations) - Create voice simulation evaluations
* [Test Runs](/Simulations/TestRuns) - View transcripts and synced call audio