Agents

Agents in Netra represent your external voice applications: phone bots, voice assistants, or any LiveKit, Vapi, or custom WebSocket endpoint. Connect them once, then trigger automated runs against your Evaluations directly from the UI. No SDK integration required—Netra places simulated calls to your agent, records the conversation, and runs Evaluators to score every interaction.

Why Agents Matter

Testing voice agents manually doesn’t scale. Connecting your agent to Netra unlocks automated, repeatable evaluation:

Challenge	How Agents Help
Manual calling is slow	Run entire evaluations against your agent with a single click
Inconsistent test coverage	Every evaluation item is tested systematically with the same evaluators
No audio evaluation	Netra records, transcribes, and evaluates voice conversations end-to-end
Disconnected tracing	Agent responses are automatically traced and linked to evaluation results
Environment-specific bugs	Test against staging, production, or any registered endpoint

How It Works

Netra provides a way to interact with your connected voice agent:

Run

Trigger a full evaluation against your agent. Netra places simulated calls for every scenario, records the conversation, runs evaluators, and reports results.

Agents Dashboard

Navigate to Library → Agents from the left navigation panel. The dashboard displays all configured agents for the current project.

Each agent card shows:

Field	Description
Name	Agent display name
Platform	LiveKit, Vapi, or Custom
Endpoint	Server URL or API reference
Created	When the agent was configured

Configuring an Agent

Click Add Agent in the top right corner to configure a new voice agent connection.

Name Your Agent

Enter a descriptive name (e.g., “Customer Support Voice Bot - Staging”). This name identifies the agent when selecting it for runs.

Select Platform

Choose the platform your voice agent runs on:

Platform	Description
LiveKit	Open-source real-time communication. Provide your LiveKit server URL and credentials.
Vapi	Voice AI platform. Provide your Vapi API key and assistant reference.
Websocket	Any agent you host yourself. Provide a WebSocket URL Netra connects to, and POST an end-of-call report back. See Websockets.

Configure Connection

The configuration fields depend on your selected platform:LiveKit Configuration:

Field	Required	Description
Server URL	Yes	WebSocket URL (wss://) for your LiveKit server
API Key	Yes	LiveKit API key for authentication
API Secret	Yes	LiveKit API secret for token generation
Agent Reference	Yes	The agent or room name to connect to

Vapi Configuration:

Field	Required	Description
API Key	Yes	Vapi API key for authentication
Assistant ID	Yes	The Vapi assistant to use for calls

Websocket Configuration:

Field	Required	Description
WebSocket URL	Yes	`wss://` endpoint Netra connects to, one per agent
Auth	Yes	A token inside the request header
Sample rate	Yes	The rate your side expects. Netra resamples its caller to it

Your backend POSTs one end-of-call report per call to Netra’s webhook, authenticated with the Netra API key — it is static, the same for every call, which is why the report carries call.id to identify itself. The full protocol is in Websockets.

Coming soon. Custom agents are not yet configurable in the dashboard. The protocol is settled if you want to build against it now.

Add Application Details

Describe what your agent does. This context helps Netra simulate realistic callers when running tests:

Field	Description
Main purpose	What the application is for
User types	Who interacts with the agent
Sensitive data	Information the agent must handle carefully or never request
Dangerous actions	Actions that should always be refused or escalated
Forbidden topics	Subjects the agent must not discuss
System prompt	The agent’s instructions, if you want them on record

Save

Click Create Agent to save the configuration. The agent becomes available for runs and appears on the Agents dashboard.

Triggering a Run

This is the primary workflow for running an entire Evaluation against a configured voice agent from the UI.

Prerequisites

Before triggering a run, ensure you have:

At least one Agent configured in the current project
An Evaluation with voice scenarios
Evaluators attached to the evaluation for scoring

Starting a Run

Open Your Evaluation

Navigate to Evaluate → Evaluations and open the voice evaluation you want to run.

Click Run

In the evaluation detail page toolbar, click the Run button.

Configure the Run

The Run modal appears with the following fields:

Field	Required	Description
Agent	Yes	Select a configured voice agent from the dropdown
Run Name	Yes	Auto-generated as `{Evaluation Name} - {Date}`, editable

The modal also displays read-only context: evaluation name, scenario count, and evaluator count.

Confirm and Run

Click Run to start the simulation. On success, you are redirected to the Test Run detail page to monitor progress.

Monitoring Results

After triggering a run, you land on the Test Run detail page. The page updates as calls complete.

Status Summary

The summary card at the top shows aggregate metrics:

Metric	Description
Passed / Failed	Number of scenarios that passed or failed evaluation
Not Available	Scenarios where evaluation could not complete
Total Cost	Aggregate cost across all calls
Average Latency	Mean response time from your agent
Duration	Total wall-clock time for the run

Use the refresh button to update the summary while the run is in progress.

Run Statuses

Status	Meaning
Running	Calls are still being placed and evaluated
Completed	All calls have finished and evaluations are scored
Failed	All calls failed—check agent configuration and endpoint availability
Cancelled	The run was manually stopped

Per-Item Results

Each scenario in the results table shows:

Column	Description
Input	The original scenario goal and caller instructions
Expected Output	The ground truth from the evaluation (if provided)
Agent Output	The transcribed conversation from the call
Run Status	Whether the call succeeded or failed
Eval Status	Whether the scenario passed or failed evaluations
Trace	Link to the execution trace for debugging

Click any row to view detailed evaluation results, the full conversation transcript with synced audio playback, or the complete execution trace.

Simulation Overview — Understand the full simulation framework
Evaluations — Create voice test scenarios for your agents
Personas — Create reusable caller personas with voice traits
Evaluators — Configure audio and conversation scoring logic
Test Runs — View transcripts and synced call audio
Websockets — Connect an agent you host yourself
Quick Start: Simulation — Get your first simulation running in minutes

Get Started

Observability

Insights

Evaluations

Simulations

Evaluators

Red Teaming

Prompt Management

Monitoring & Dashboard

Account settings

Migrations

Why Agents Matter

How It Works

Run

Agents Dashboard

Configuring an Agent

Triggering a Run

Prerequisites

Starting a Run

Monitoring Results

Status Summary

Run Statuses

Per-Item Results

​Why Agents Matter

​How It Works

Run

​Agents Dashboard

​Configuring an Agent

​Triggering a Run

​Prerequisites

​Starting a Run

​Monitoring Results

​Status Summary

​Run Statuses

​Per-Item Results

​Related

Why Agents Matter

How It Works

Agents Dashboard

Configuring an Agent

Triggering a Run

Prerequisites

Starting a Run

Monitoring Results

Status Summary

Run Statuses

Per-Item Results

Related