Automations Overview

Automations notify you when something changes in your project. When a prompt gets a new version, a test run’s evaluation completes, or a red team run finishes, Netra sends a message to the destinations you choose: an email inbox, a Slack channel, your own HTTPS endpoint, or a GitHub Actions workflow.

React to prompt changes

Get notified when a new version is published or a production label moves.

Track test runs

Get an evaluation report when a run finishes, or an alert when a test run fails.

Follow red team runs

Know when red team runs finish or fail.

Trigger your pipelines

Start a GitHub Actions workflow or call your own service on every event.

Automations vs. Alert Rules

Both features send notifications to the same contact points, but they answer different questions.

	Automations	Alert Rules
Fires on	A lifecycle event (updated, evaluation completed, failed)	A metric crossing a threshold
Watches	Prompts, test runs, red team runs	Traces and spans (cost, latency, errors)
Example	”Tell #ml-team when a new prompt version is published"	"Tell on-call when p95 latency exceeds 5s”

Use Alert Rules for metric thresholds and automations for lifecycle events.

How Automations Work

Every automation has four parts:

Part	Description
Source	What the automation watches: a prompt, an evaluation’s test runs, or a red team config’s runs
Trigger events	Which events fire it, such as `evaluation_completed` or `test_run_failed`
Conditions	Optional filters that must all match, such as “Label is `production`”
Destinations	One or more contact points that receive the message

When an event happens, Netra checks every enabled automation in the project. If the source and event match and every condition holds, Netra queues one delivery per destination. Each delivery is tracked in the automation’s delivery history.

Sources and Events

Source	Events	Fires when
Prompt	`updated`	A new version is published, or a label is attached or removed
	`deleted`	A prompt is deleted
Test run	`test_run_triggered`	A test run starts
	`test_run_failed`	The test run as a whole fails, for example when every item fails to run
	`evaluation_completed`	Every item in the run has finished evaluating. The message includes how many evaluators and items passed.
Red team run	`redteam_run_completed`	A red team run finishes
	`redteam_run_failed`	A red team run fails

A prompt’s updated event covers several kinds of change. Add a Change condition to fire only on the ones you care about, such as New version published. See Conditions.

Destinations

Automations deliver to the project’s contact points. Four types are supported:

Type	What you receive
Email	A formatted email with the event summary. Test run and red team events also include a link to Netra.
Slack	A Slack message with the event details. Test run and red team events also include a View in Netra link.
Webhook	A signed JSON `POST` to your HTTPS endpoint
GitHub dispatch	A `repository_dispatch` event that triggers a GitHub Actions workflow

See Destinations for setup steps and the payload format.

Create an Automation

Set up triggers, conditions, and destinations

Destinations

Configure email, Slack, webhook, and GitHub dispatch

Get Started

Observability

Insights

Evaluations

Simulations

Evaluators

Red Teaming

Prompt Management

Monitoring & Dashboard

Automations

Account settings

Migrations

Automations Overview

React to prompt changes

Track test runs

Follow red team runs

Trigger your pipelines

Automations vs. Alert Rules

How Automations Work

Sources and Events

Destinations

See Also

Create an Automation

Destinations

React to prompt changes

Track test runs

Follow red team runs

Trigger your pipelines

​Automations vs. Alert Rules

​How Automations Work

​Sources and Events

​Destinations

​See Also

Create an Automation

Destinations

Automations vs. Alert Rules

How Automations Work

Sources and Events

Destinations

See Also