> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getnetra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Quick Start: Evaluation

> Run your first AI evaluation in Netra in minutes. Create a dataset, configure evaluators, and measure your LLM output quality with automated test runs.

This guide walks you through setting up [evaluations](/Evaluation/Evaluation-overview) to measure your AI system's accuracy, quality, and reliability.

## 1. Prerequisites

Before setting up evaluations, ensure you have:

* [Netra SDK installed and initialized](/quick-start/QuickStart_Tracing)
* At least one traced LLM call in your dashboard
* Your API key configured

## 2. Create a Dataset

[Datasets](/Evaluation/Datasets) are collections of test cases that define inputs and expected outputs for your AI system.

### Option A: Create from Traces (Recommended)

Convert real-world interactions into test cases:

<Steps>
  <Step title="Navigate to Traces">
    Go to **Observability → Traces** and find a trace you want to use as a test case.
  </Step>

  <Step title="Add to Dataset">
    Click the **Add to Dataset** button on the trace.
  </Step>

  <Step title="Configure the Test Case">
    * Enter a dataset name (e.g., "Customer Support QA") - Add optional tags for
      organization - Review the input prompt - Provide the expected output - Click
      **Next**
  </Step>

  <Step title="Select Evaluators">
    Choose evaluators to score your AI's performance (see Step 3).
  </Step>
</Steps>

### Option B: Create Manually

<Steps>
  <Step title="Open Dataset Dashboard">
    Navigate to **Evaluation → Datasets** and click **Create Dataset**.
  </Step>

  <Step title="Configure Dataset">
    * Enter a dataset name - Select **Single Turn** for request/response pairs -
      Choose **Add manually**
  </Step>

  <Step title="Add Test Cases">
    For each test case, provide:

    * **Input**: The prompt or question
    * **Expected Output**: The ideal response
    * **Metadata** (optional): Additional context
  </Step>
</Steps>

## 3. Configure Evaluators

[Evaluators](/Evaluation/Evaluators) score your AI's outputs against defined criteria. Netra offers two types:

### LLM as Judge

Best for subjective quality assessment:

* **Answer Correctness**: Does the response match the expected answer?
* **Relevance**: Is the response relevant to the question?
* **Hallucination Detection**: Does the response contain fabricated information?
* **Toxicity**: Is the content safe and appropriate?

### Code Evaluators

Best for deterministic checks:

* **JSON Validation**: Verify JSON structure and schema
* **Regex Matching**: Pattern-based validation
* **Custom Logic**: Write JavaScript or Python for specific rules

<Steps>
  <Step title="Add Evaluators">
    When creating your dataset, click **Next** to reach the evaluator selection screen.
  </Step>

  <Step title="Select from Library">
    Browse [pre-built evaluators](/Evaluation/Evaluators#library) in categories: -
    Quality - Performance - Agentic - Guardrails
  </Step>

  <Step title="Map Variables">
    Configure how evaluator variables map to your data:

    * **Dataset field**: Use values from your test cases
    * **Agent response**: Use the actual LLM output
    * **Execution data**: Use trace metadata
  </Step>
</Steps>

## 4. Run an Evaluation

Once your dataset is configured with evaluators:

<Steps>
  <Step title="Get Dataset ID">
    Open your dataset and copy the **Dataset ID** displayed at the top.
  </Step>

  <Step title="Trigger Evaluation">
    Run your AI system with the dataset inputs. Evaluations execute automatically
    when traces are created.
  </Step>

  <Step title="View Results">
    Navigate to **Evaluation → [Test Runs](/Evaluation/TestRuns)** to see your evaluation results.
  </Step>
</Steps>

## 5. Analyze Test Run Results

Click on a [test run](/Evaluation/TestRuns) to view detailed results:

### Summary Metrics

* **Total Cost**: Aggregate cost of all LLM calls
* **Average Latency**: Response time across test cases
* **Pass/Fail Rate**: Overall success rate

### Per-Test-Case Results

Each test case shows:

| Field            | Description                           |
| ---------------- | ------------------------------------- |
| Input            | The prompt sent to the AI             |
| Expected Output  | Your defined ideal response           |
| Task Output      | The actual AI response                |
| Run Status       | Shows Running status                  |
| Eval Status      | Pass/Fail indicator                   |
| Evaluator Scores | Individual scores from each evaluator |
| View Trace       | Link to the full execution trace      |

## Troubleshooting

| Issue                  | Solution                                                                |
| ---------------------- | ----------------------------------------------------------------------- |
| No test runs appearing | Ensure your dataset has evaluators configured and traces are being sent |
| Evaluator errors       | Test your evaluator in the Playground before adding to datasets         |
| Unexpected failures    | Check variable mappings in evaluator configuration                      |

## Next Steps

<CardGroup cols={2}>
  <Card title="Datasets" icon="table" href="/Evaluation/Datasets">
    Learn advanced dataset management
  </Card>

  <Card title="Evaluators" icon="scale-balanced" href="/Evaluation/Evaluators">
    Create custom evaluation logic
  </Card>

  <Card title="Test Runs" icon="flask-vial" href="/Evaluation/TestRuns">
    Deep dive into test run analysis
  </Card>

  <Card title="Evaluation Overview" icon="gauge-high" href="/Evaluation/Evaluation-overview">
    Understand the full evaluation framework
  </Card>
</CardGroup>