Skip to main content
This FAQ helps developers, platform teams, and decision-makers understand how Netra works across setup, observability, evaluation, simulation, integrations, and security.

​
Get Started

Account setup, SDK installation, and your first traces.
Netra is a unified platform for AI application observability, evaluation, and simulation. It helps teams trace LLM calls, evaluate output quality, simulate multi-turn conversations, and monitor production AI systems — all from a single dashboard. Netra is built on OpenTelemetry standards and integrates with leading LLM providers, AI frameworks, and vector databases.
You need:
  • A Netra account (US or EU data region)
  • An API key generated from the Netra dashboard
  • The netra-sdk package (Python >=3.10, <3.14 or Node.js 18+)
See the Quick Start Overview for a step-by-step walkthrough.
  1. Go to Organisation → Projects in the side navigation panel
  2. Click Create Project and provide a project name
  3. Confirm by clicking Create Project
Your API key is scoped to a project. See Project Settings for more details.
Most teams see their first traces within minutes. Install the SDK, initialize Netra.init() with your API key, and run any LLM workflow — traces appear in the dashboard immediately. Follow the Quick Start to get started.
Once the SDK is initialized, auto-instrumentation captures traces from all supported libraries in your environment — no code changes required. Your first trace is typically visible in the dashboard as soon as you run your workflow. When you need more structure, add decorators (@workflow, @agent, @task) or manual spans for fine-grained control. See the Tracing Quick Start for a complete example.

​
Observability

Traces, spans, agents, sessions, costs, and latency monitoring.
A trace captures the complete execution of a request through your AI application: LLM calls (prompts, completions, model parameters), tool invocations, retrieval steps, token usage, latency per step, costs, errors, and any custom metadata you attach.
A trace represents the full journey of a single request through your system. Spans are the individual operations within that trace — an LLM call, a tool execution, or a retrieval step. Spans are organized hierarchically (parent-child), so you can see exactly which step triggered which sub-step.
Netra offers three ways to instrument your application:
  1. Auto-instrumentation — Zero code changes. Netra automatically traces supported libraries (OpenAI, LangChain, Pinecone, etc.)
  2. Decorators — Add @workflow, @agent, or @task to your functions for structured tracing with minimal code
  3. Manual tracing — Full control using context managers (Python) or SpanWrapper for custom spans
  • Agents: Use the @agent decorator to track agent execution, tool usage patterns, and decision flows
  • Sessions: Call set_session_id to group related interactions into a single session
  • Users: Call set_user_id to attribute traces to individual users
Netra automatically captures token usage and costs from supported LLM providers. You can view cost breakdowns by model, tenant, user, or time range in custom dashboards. For unsupported providers, attach cost data as custom span attributes.

​
Evaluation

Datasets, evaluators, test runs, and automated quality checks.
An evaluation measures your AI system’s output quality against defined criteria. Netra supports two evaluator types: LLM-as-Judge evaluators (for subjective quality, semantic correctness, and custom prompts) and Code evaluators (for deterministic checks like JSON schema validation, regex matching, and business logic).
Datasets are collections of test cases that define what to evaluate. Each test case includes an input prompt and an expected output. You can create datasets by:
  • Converting production traces into test cases with one click
  • Manually adding test cases in the dashboard
  • Organizing by feature, model version, or release
Netra provides a library of preconfigured evaluators across several categories:
  • Quality: Coherence, Factual Accuracy, Answer Relevance
  • Guardrails: Toxicity, Bias, PII Detection
  • Agentic: Tool Call Accuracy, Goal Fulfillment
You can also create custom evaluators using LLM prompts or code logic. Test them in the Evaluator Playground before deploying.
Yes. When Auto-Evaluation is enabled at the organization level, Netra runs Coherence, Factual Accuracy, and Toxicity evaluators on every incoming trace automatically. These background checks continue until your configured usage limit is reached. See Evaluation Overview for setup details.
Navigate to Evaluation → Test Runs to see all evaluation results. Each test run shows pass/fail rates, individual evaluator scores, expected vs. actual output comparisons, and links to execution traces for debugging failures.

​
Simulation

Multi-turn conversation testing with configurable personas and goals.
Simulation lets you test your AI agents with realistic, multi-turn conversations. You define a goal, a user persona, and facts the agent should communicate — Netra then simulates the conversation and scores it automatically using LLM-as-Judge evaluators.
Create a simulation dataset in the dashboard:
  1. Set a conversation goal (e.g., “Get a refund for a damaged product”)
  2. Choose a user persona (neutral, friendly, frustrated, confused, or custom)
  3. Add user data and facts the agent should communicate
  4. Select and configure evaluators to score the conversation
Netra provides five persona types for simulation scenarios:
  • Neutral — Baseline performance testing
  • Friendly — Tests professionalism without pushback
  • Frustrated — Tests de-escalation and patience
  • Confused — Tests clarity and explanation quality
  • Custom — Define your own persona for industry-specific scenarios
Netra uses eight preconfigured LLM-as-Judge evaluators in two categories:
  • Quality (6): Guideline Adherence, Conversation Completeness, Profile Utilization, Conversational Flow, Conversation Memory, Factual Accuracy
  • Agentic (2): Goal Fulfillment, Information Elicitation
Each evaluator produces a score between 0 and 1. Scores at or above 0.6 pass.
Open the failed scenario in Simulation → Test Runs and use the three tabs:
  1. Conversation — Read the full multi-turn transcript to find where things went wrong
  2. Evaluation Results — See which evaluators failed and their scores
  3. Scenario Details — Verify the goal, persona, user data, and facts were configured correctly
Click View Trace on any turn to inspect the full execution flow, including LLM calls and tool invocations.

​
Integrations & SDKs

Supported languages, frameworks, providers, and auto-instrumentation.
Netra provides SDKs for both Python and TypeScript/JavaScript:
  • Python: netra-sdk on PyPI (Python >=3.10, <3.14)
  • TypeScript: netra-sdk on npm (Node.js 18+)
Both SDKs support auto-instrumentation, decorators, manual tracing, and context tracking. See the SDK Reference for the full API.
Netra integrates with 30+ services across four categories:
  • LLM Providers (14): OpenAI, Anthropic, Google Gemini, AWS Bedrock, Mistral, Groq, Cohere, and more
  • AI Frameworks (11): LangChain, LangGraph, LlamaIndex, CrewAI, Pydantic AI, DSPy, and more
  • Vector Databases (8): Pinecone, ChromaDB, Qdrant, Weaviate, Milvus, and more
  • Speech Services (3): Deepgram, ElevenLabs, Cartesia
See the full list at Integrations Overview.
Auto-instrumentation automatically traces all supported library calls once Netra.init() is called. You can either specify which libraries to instrument (e.g., InstrumentSet.OPENAI) or omit the instruments parameter to let Netra auto-detect all supported libraries in your environment.
You can send version identifiers such as prompt_version or model_version as metadata attributes on your spans, enabling comparisons and analysis across versions in the dashboard.
Yes. Netra can collect observability signals from gateway-based and webhook-driven architectures. Use manual tracing to instrument custom ingestion points that auto-instrumentation does not cover.

​
Dashboards & Alerts

Custom analytics, real-time monitoring, and proactive notifications.
Yes. Custom dashboards let you create real-time analytics views with six chart types (line, bar, pie, big number, and more). Available metrics include latency percentiles (P50, P90, P95, P99), total cost, token counts, error rates, and request counts. Filter by model, tenant, user, session, or environment.
Configure alert rules to get notified about anomalies in your AI system:
  • Scope: Trace-level or span-level monitoring
  • Metrics: Cost, latency, error rate, token count
  • Filters: Model, tenant ID, environment, service name
  • Delivery: Email or Slack (via API token or webhook)
For example, you can alert when any trace costs more than $0.50 or when P95 latency exceeds 3 seconds.

​
Multi-Tenancy

Per-customer isolation, usage attribution, and tenant-level monitoring for B2B SaaS.
A tenant represents an end customer of your application. When you build a multi-tenant AI product, each of your customers is modeled as a tenant so you can isolate their traces, metrics, and costs. Use one workspace per organization, one project per product, and a tenant_id for each customer.
Call set_tenant_id early in your request lifecycle (typically in middleware) to associate all subsequent traces with that customer:
  • Python: Netra.set_tenant_id("acme-corp")
  • TypeScript: Netra.setTenantId("acme-corp")
See the Tenants guide for setup details and best practices.
Yes. Each tenant’s traces, metrics, and costs are tagged with their unique tenant ID and can be viewed independently. The Tenants dashboard lets you filter by tenant to see only that customer’s data, including sessions, trace counts, and aggregated costs.
Yes. When configuring alert rules, you can add a Tenant ID filter so the alert only triggers for a specific customer. This is useful for per-tenant SLA monitoring and budget enforcement.

​
Security & Compliance

Data handling, privacy controls, and regulatory compliance.
Netra stores observability data including traces, span metadata, evaluation results, and simulation transcripts. You control what data is sent through SDK configuration options like trace_content (to enable or disable capturing prompt/completion content).
Yes. Retention policies can be configured based on your environment and compliance requirements.
Yes. Access is controlled using roles and permissions at the organization level. See Organisation Settings for configuration details.
Yes. Netra follows industry-standard security practices and complies with SOC 2, GDPR, and HIPAA standards.
Customer data is stored in secure, SOC 2-compliant data centers in the US, EU, and India, managed by top-tier cloud providers. You select your data region (US or EU) during account creation.
Yes. On-premises deployment is available under enterprise plans. Contact the Netra team for details.

​
Pricing & Billing

Plans, usage metering, and billing details.
Pricing is based on usage — traces, evaluations, and simulations — depending on your selected plan. See the pricing page for detailed information.
Yes. Netra provides monthly subscription plans and enterprise contracts with custom terms.
On-premises deployments are offered under enterprise pricing. Contact the Netra team for a quote.
Usage and billing details are available in the Netra dashboard under your organization settings.

​
Help & Troubleshooting

Common issues, debugging steps, and support.
Check the following:
  1. Your NETRA_API_KEY environment variable is set correctly
  2. Netra.init() is called before any LLM or framework calls
  3. The OTLP endpoint is reachable from your environment
  4. Your workflow is actually executing (check for errors in your application logs)
See the Tracing Quick Start for a working example.
Ensure tool functions are wrapped with the @task decorator or instrumented manually using SpanType.TOOL. Auto-instrumentation only captures calls to supported libraries — custom tool functions need explicit instrumentation.
Costs and token counts are automatically captured for supported LLM providers. If you are using an unsupported provider or a custom gateway, you need to attach token usage as custom span attributes via manual tracing.
Verify that:
  1. Your dataset has test cases with input and expected output
  2. At least one evaluator is attached to the dataset
  3. The test run has completed (check the status column)
Use the official support channel to reach the Netra team.
Include your project name, trace IDs, timestamps, SDK version (netra-sdk), and any relevant error logs or stack traces. This helps the team diagnose the issue quickly.
Last modified on March 17, 2026