> ## Documentation Index > Fetch the complete documentation index at: https://docs.getnetra.ai/llms.txt > Use this file to discover all available pages before exploring further. # Evaluating Agent Decisions > Evaluate AI agent decisions with Netra. Measure tool selection accuracy, escalation logic, and workflow completion using structured evaluation datasets. A traced agent tells you what happened — which tools were called, how long each step took, and what the LLM generated. Evaluation tells you whether the agent made the right decisions. Without structured scoring, you can't tell if the agent is selecting the wrong tools, over-escalating simple requests, or stopping before the workflow is complete. These failures don't throw errors — they just produce worse outcomes. This cookbook walks you through Netra's evaluation workflow: creating evaluators for agent-specific quality dimensions, building test datasets from your traces, running test suites, and interpreting results to improve your agent. **Prerequisite:** You need a Netra API key ([Get started here](/quick-start/Overview)) and an AI agent to evaluate. The test cases below use the customer support agent from the [Tracing LangChain Agents](/Cookbooks/observability/tracing-langchain-agents) cookbook as a reference. ## What You'll Learn Create structured test cases with inputs, expected outputs, and metadata for your evaluators Set up evaluators for tool correctness, escalation accuracy, and workflow completion Execute evaluations via the SDK and collect quality metrics Interpret scores, debug failures using trace integration, and improve your agent *** ## Why Agent Decisions Need Evaluation Agent evaluation differs from simple LLM evaluation. Agents make multi-step decisions that compound — a 95% accurate tool selection across 3 steps means only 86% of full workflows succeed (0.95^3): | Failure Mode | What Goes Wrong | Why You Can't Spot-Check It | | ------------------------ | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | **Wrong tool selection** | Agent uses `search_kb` when it should use `check_order_status` | The answer may still sound reasonable despite using the wrong data source | | **Over-escalation** | Agent escalates a simple FAQ to a human operator | Each escalation looks cautious and safe in isolation — you need aggregate metrics to see the pattern | | **Under-escalation** | Agent tries to handle a frustrated customer instead of escalating | Only visible when you compare the agent's decision against the expected action | | **Incomplete workflow** | Agent looks up the ticket but never checks the related order | The partial answer addresses part of the question, so it looks acceptable on a quick read | Netra's evaluation framework addresses this with [Datasets](/Evaluation/Datasets) (test cases with inputs, expected outputs, and metadata), [Evaluators](/Evaluation/Evaluators) (library and custom code-based scoring for tool usage, escalation, and completion), and [Test Runs](/Evaluation/TestRuns) (execution results with pass/fail rates, scores, and linked traces). The workflow is: create evaluators, build test cases, run, and review. See the [Evaluation Overview](/Evaluation/Evaluation-overview) for a deeper look at the framework. *** Now, let's walk through the process of evaluating agent decisions: ## Step 1: Create Evaluators You need three evaluators — one from the library and two custom LLM as Judge evaluators. ### Tool Correctness (Library) Go to **Evaluation → Evaluators**, switch to the **Library** tab, and add **Tool Correctness** from the Tool Use category. | Evaluator | What It Measures | | -------------------- | ---------------------------------------------------------------------------------------- | | **Tool Correctness** | Did the agent call the right tools, avoid forbidden tools, and use the correct sequence? | ### Escalation Accuracy (LLM as Judge) Click **Add Evaluator** and create an LLM as Judge evaluator. | Setting | Value | | ----------------- | ------------ | | **Type** | LLM as Judge | | **Output Type** | Numerical | | **Pass Criteria** | score >= 0.8 | Use the following prompt template: ``` You are evaluating whether a customer support agent made the correct escalation decision. Metadata for this test case: {{metadata}} Agent's response: {{agent_response}} Evaluate the escalation decision based on the "should_escalate" field in the metadata: - If should_escalate is true, the agent must have escalated (e.g., transferred to a human, mentioned a specialist, offered to connect to a manager). - If should_escalate is false, the agent must NOT have escalated. Score: - 1.0 if the agent's escalation decision is correct - 0.5 if the agent escalated unnecessarily (false positive — less severe) - 0.0 if the agent failed to escalate when required (false negative — severe) ``` ### Workflow Completion (LLM as Judge) Create another LLM as Judge evaluator that validates whether the agent completed all required steps. | Setting | Value | | ----------------- | ------------ | | **Type** | LLM as Judge | | **Output Type** | Boolean | | **Pass Criteria** | true | Use the following prompt template: ``` You are evaluating whether a customer support agent completed all required workflow steps. Metadata for this test case: {{metadata}} Agent's response: {{agent_response}} Check the "required_steps" field in the metadata. If required_steps is empty or missing, return true. Otherwise, verify that the agent's response addresses every step listed. Steps can be addressed using different wording — check for semantic equivalence, not exact keyword matches. Return true if ALL required steps are covered, false otherwise. ``` You can test each evaluator in the **Playground** before using it in a dataset. See [Evaluators](/Evaluation/Evaluators) for the full reference.