Skip to main content
AI systems don’t fail loudly. They drift, regress, and quietly degrade over time. Netra’s Evaluation framework makes that invisible failure visible, giving you a structured, repeatable way to measure how your AI behaves—not just once, but continuously across releases, prompts, models, and environments.

Quick Start: Evaluation

New to evaluations? Get your first evaluation running in minutes.

​
Why Evaluation Matters

Without systematic evaluation, you’re flying blind. Netra helps you answer critical questions with confidence:

​
Core Building Blocks

The Evaluation suite is built on three interconnected pillars:

​
Evaluators

Evaluators are the scoring logic that assesses your AI’s performance. Netra organizes evaluators by modality:
  • Text Evaluators — LLM-as-Judge, code-based, and rule-based evaluators for text outputs
  • Voice Evaluators — TTS, STT, and conversational evaluators for audio quality
  • Image Evaluators — Multimodal and rule-based evaluators for image generation and editing
Netra provides a Library of 49 preconfigured evaluators across 12 categories. Customize any evaluator and save it to My Evaluators for reuse across evaluations.

​
Evaluations

Evaluations are collections of test cases that define what you want to evaluate.

​
Test Runs

Test Runs execute your evaluations through the evaluation pipeline, providing point-in-time snapshots of system health.

​
Supported Agent Types

Netra evaluates multiple types of AI agents:

Text Agents

Evaluate chatbots, copilots, and LLM-based agents that produce text outputs.

Voice Agents

Evaluate voice agents for call quality, transcription accuracy, and conversational delivery.

Image Agents

Evaluate image generation, editing, and analysis systems for visual quality and accuracy.

​
Use Cases

​
Regression Testing

Catch quality degradation before it reaches production:
  1. Create an evaluation from your golden test cases
  2. Run evaluations after each model or prompt change
  3. Compare results across test runs to identify regressions

​
Continuous Quality Monitoring

Track quality metrics over time:
  1. Build an evaluation that reflects your production traffic patterns
  2. Schedule regular evaluation runs
  3. Set up alerts when pass rates drop below thresholds

​
Model Comparison

Evaluate different models or prompts objectively:
  1. Create a standardized evaluation
  2. Run the same inputs through different model configurations
  3. Compare scores across test runs to make data-driven decisions

​
Image Quality Testing

Evaluate AI-generated or edited images for visual quality and accuracy:
  1. Create an image evaluation with input images and expected characteristics
  2. Use image evaluators to assess composition, style, and format
  3. Track visual quality metrics across model updates and prompt changes

​
Getting Started

1

Configure Evaluators

Define your scoring criteria by adding evaluators—choose from the library or create custom ones.
2

Create an Evaluation

Create an evaluation manually or by importing a CSV, and attach your evaluators.
3

Run Evaluations

Execute your evaluation and view results in Test Runs.
4

Iterate and Improve

Use insights from test runs to refine your prompts, models, and evaluation criteria.
Last modified on August 28, 2026