Skip to main content
AI agents don’t just answer questions—they engage in complex, multi-turn conversations to achieve goals. Netra’s Simulation framework lets you test these interactions systematically, simulating realistic user behaviors to validate your agent’s performance before deployment.

Quick Start: Simulation

New to simulations? Get your first simulation running in minutes.

​
Why Simulation Matters

Traditional testing falls short for conversational agents. Simulations provide a comprehensive way to test multi-turn interactions with realistic user behaviors:

​
Core Building Blocks

The Simulation suite is built on three interconnected pillars:

​
Evaluators

Evaluators assess the entire conversation after it completes. Netra provides 8 preconfigured library evaluators in two categories: All evaluators use LLM-as-Judge with a default pass threshold of >= 0.6.

​
Datasets

Datasets are collections of simulation scenarios that define multi-turn conversation goals.

​
Test Runs

Test Runs execute your simulation scenarios, providing detailed conversation transcripts and evaluation results.

​
Use Cases

​
Goal Achievement Testing

Validate that your agent can successfully complete user objectives:
  1. Create scenarios with specific goals (e.g., “Get a refund from customer support”)
  2. Define what facts the agent must communicate
  3. Run simulations and verify goal achievement across different personas
  4. Analyze conversation transcripts to understand failure points

​
Persona-Based Testing

Test agent performance with different user types:
  1. Create datasets with various personas (frustrated, confused, friendly)
  2. Run the same scenario across all personas
  3. Compare results to identify which personas your agent handles poorly
  4. Refine your agent based on insights

​
Getting Started

1

Configure Evaluators

Set up evaluators to define your scoring criteria — choose from the library or create custom ones.
2

Create a Dataset

Build a multi-turn dataset with simulation scenarios, user personas, and facts to verify.
3

Run Simulations

Execute your dataset and view conversation transcripts and results in Test Runs.
4

Analyze and Improve

Use insights from simulations to refine your agent’s behavior.
  • Evaluators - Configure scoring logic and criteria
  • Datasets - Create multi-turn simulation scenarios
  • Test Runs - Analyze simulation results and conversation transcripts
  • Traces - Understand how simulations connect to trace data
Last modified on March 17, 2026