> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getnetra.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Evaluators
> Assess simulated AI conversations in Netra using library and custom evaluators. Measure turn-by-turn quality, goal completion, and agent performance.
Simulation evaluators assess entire multi-turn conversations at the session level. After a simulated conversation completes, evaluators determine whether your agent achieved its goal, communicated facts correctly, and maintained quality throughout the interaction.
## Why Simulation Evaluators Matter
Multi-turn conversations require different evaluation approaches than single-turn responses:
| Challenge | How Simulation Evaluators Help |
| --------------------- | ----------------------------------------------------------------------------------- |
| Goal achievement | Verify whether the conversation reached its objective |
| Fact accuracy | Track whether critical information was communicated correctly across multiple turns |
| Conversation quality | Ensure the conversation maintains logical flow, completeness, and consistency |
| Information gathering | Assess whether the agent effectively collected required information from the user |
## Evaluators Dashboard
Navigate to **Evaluation → Evaluators** from the left navigation panel. Switch to the **Library** tab and filter by **Multi turn** to see the simulation evaluators.
Netra organizes simulation evaluators into two categories: **Quality** and **Agentic**.
## Library Evaluators
Netra provides 8 preconfigured library evaluators across two categories. All evaluators run at the session level, assessing the entire conversation after it completes.
### Quality Evaluators
Quality evaluators assess how well your agent maintains conversation standards.
| Evaluator | What It Measures |
| ----------------------------- | ------------------------------------------------------------------------------------- |
| **Guideline Adherence** | Whether the assistant followed its given instructions throughout the conversation |
| **Conversation Completeness** | Whether all human intents were addressed during the conversation |
| **Profile Utilization** | Whether the assistant correctly used provided human profile information when relevant |
| **Conversational Flow** | Whether the conversation flowed logically and the assistant maintained consistency |
| **Conversation Memory** | Whether the assistant remembered and correctly used information shared earlier |
| **Factual Accuracy** | Whether the assistant's claims were consistent with provided reference facts |
### Agentic Evaluators
Agentic evaluators assess goal-directed and information-gathering behavior.
| Evaluator | What It Measures |
| --------------------------- | ---------------------------------------------------------------------- |
| **Goal Fulfillment** | Goal achievement and progress toward the stated conversation objective |
| **Information Elicitation** | How effectively the agent gathered required information from the user |
## Evaluator Configuration
All 8 library evaluators share the same configuration:
| Setting | Value |
| ------------------------- | ------------------------------------------ |
| **Type** | LLM as Judge |
| **Eval Scope** | Session (entire conversation) |
| **Output** | Numerical (0-1, normalized from 1-5 scale) |
| **Default Pass Criteria** | >= 0.6 |
You can adjust the pass criteria threshold for any evaluator based on your requirements. A higher threshold enforces stricter quality standards.
## Using Evaluators in Simulations
When configuring a [multi-turn dataset](/Simulation/Datasets), you select and configure evaluators in Step 4 of the dataset creation flow. You can choose any combination of Quality and Agentic evaluators based on what you want to measure.
## Best Practices
### Choosing Evaluators by Scenario Type
| Scenario Type | Recommended Evaluators |
| -------------------- | ------------------------------------------------------------------- |
| Customer Support | Conversation Completeness, Factual Accuracy, Guideline Adherence |
| Technical Assistance | Conversation Completeness, Conversational Flow, Goal Fulfillment |
| Sales Conversations | Profile Utilization, Factual Accuracy, Information Elicitation |
| Troubleshooting | Conversation Completeness, Conversation Memory, Conversational Flow |
### Getting Started with Evaluators
1. **Start with Goal Fulfillment and Factual Accuracy** — these cover the most critical aspects of any simulation
2. **Add Quality evaluators** based on your use case — Conversation Completeness and Guideline Adherence are strong defaults
3. **Adjust pass criteria** if the default threshold of 0.6 is too lenient or strict for your needs
4. **Monitor results** across the first few test runs to ensure evaluators align with your expectations
## Related
* [Simulation Overview](/Simulation/Simulation-overview) - Understand the full simulation framework
* [Datasets](/Simulation/Datasets) - Create scenarios that use evaluators
* [Test Runs](/Simulation/TestRuns) - View evaluation results from simulations
* [Evaluation Evaluators](/Evaluation/Evaluators) - Standard single-turn evaluators documentation