Skip to main content
Agents are the core entities you’re testing in simulations. They define what your AI system can do (abilities) and what it should avoid (constraints). By clearly defining these parameters, you create a foundation for realistic, goal-oriented simulations.

​
Why Agents Matter

Agents provide the blueprint for your AI system’s behavior:
BenefitDescription
Clear CapabilitiesDefine exactly what your agent can do, preventing scope creep
Boundary EnforcementSet explicit constraints to ensure safe, compliant behavior
ReusabilityCreate once, use across multiple simulation datasets
Version ControlTrack agent definition changes over time as capabilities evolve

​
Agents Dashboard

Navigate to Evaluation → Agents from the left navigation panel to access your agents. Agents Dashboard You will see cards displaying the following fields, each providing key information about your agents at a glance.
ColumnDescription
Agent NameUnique identifier for the agent
Created AtTimestamp for version tracking
ActionsQuick access to edit or delete agents

​
Creating an Agent

Click the Create Agent button in the top right corner of the Agents page.
1

Name Your Agent

Provide a descriptive name that identifies what this agent does.Example: “Customer Support Agent” or “Technical Documentation Assistant”
2

Define Abilities

Describe what your agent can do. This should include:
  • Role: What is the agent’s primary function?
  • Capabilities: What tasks can it perform?
  • Tools: What external tools or APIs can it access?
  • Knowledge: What information does it have access to? Agent Abilities Field
Example:
You are a customer support agent for an e-commerce platform.

You can:
- Answer questions about orders, shipping, and returns
- Process refund requests for orders within 30 days
- Access customer order history and tracking information
- Escalate complex issues to human supervisors
- Provide product recommendations based on customer needs
3

Define Constraints

Specify what your agent should NOT do. This is optional but highly recommended for safety and compliance.Constraints should include:
  • Security restrictions: What information should never be shared?
  • Scope limitations: What topics are out of bounds?
  • Escalation criteria: When should the agent defer to humans?
  • Compliance requirements: What regulatory boundaries must be respected? Agent Constraints Field
Example:
You must NOT:
- Process refunds for orders older than 30 days without supervisor approval
- Share customer payment information or full credit card numbers
- Make commitments about shipping dates you cannot verify
- Provide medical or legal advice
- Override established company policies without escalation
4

Create Agent

Click Create Agent to save your agent configuration.

​
Best Practices

​
Writing Effective Abilities

  • Be specific: “Process refunds up to $100” is better than “Handle refunds”
  • Include context: Mention available tools, knowledge bases, and data access
  • Use clear language: Write as if you’re briefing a human agent
  • Group related capabilities: Organize abilities into logical sections (information access, actions, escalation)

​
Writing Effective Constraints

  • Start with “must NOT”: Make prohibitions explicit and unambiguous
  • Provide escalation criteria: Define when to defer to humans or other systems
  • Include regulatory requirements: Call out compliance boundaries (PII, medical advice, financial regulations)
  • Include edge cases: Think about what could go wrong and prevent it proactively

​
Organizing Agents

  • Use descriptive names: “Refund Support Agent” is clearer than “Agent 1”
  • Create specialized agents: Multiple focused agents are better than one overly broad agent
  • Document changes: When updating abilities or constraints, note what changed and why
  • Test regularly: Run simulations after updating agents to verify behavior changes

​
Using Agents in Simulations

Once created, agents become available when building datasets:
  1. Create or edit a multi-turn dataset
  2. In the scenario configuration step, select your agent
  3. The agent’s abilities and constraints will be used to guide its behavior during simulations
  4. Run simulations and view how your agent performs in Test Runs

​
Example Agent Configurations

​
Customer Support Agent

Abilities: 
You are a customer support agent for TechGear, an electronics retailer.

You can:
- Answer questions about products, orders, shipping, and returns
- Look up order status using order numbers
- Process returns and refunds for orders within 30 days
- Provide troubleshooting steps for common device issues
- Offer product recommendations based on customer needs
- Access knowledge base articles about products and policies
Constraints: 
You must NOT:
- Process refunds exceeding $200 without manager approval
- Share customer credit card numbers or sensitive payment data
- Make shipping promises you cannot verify in the system
- Provide technical support for products outside our catalog
- Override return policies without documented approval

​
Technical Documentation Assistant

Abilities: 
You are a technical documentation assistant for DevTools API.

You can:
- Answer questions about API endpoints, parameters, and authentication
- Provide code examples in Python, JavaScript, and cURL
- Explain error codes and troubleshooting steps
- Reference official documentation and changelogs
- Suggest best practices for API integration
- Generate sample request/response payloads
Constraints: 
You must NOT:
- Provide API keys or authentication credentials
- Guarantee specific performance metrics or SLAs
- Recommend workarounds that violate terms of service
- Debug customer production code line-by-line
- Provide legal interpretations of licensing terms
Last modified on February 11, 2026