> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getnetra.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Image Evaluators

> Create image evaluators in Netra to assess visual quality, image-text alignment, and editing accuracy. Use multimodal judges or code-based validators for image AI.

Image evaluators assess the quality, accuracy, and relevance of AI-generated or edited images. They transform subjective visual assessments into measurable metrics—from composition quality and style consistency to format compliance and text-image alignment. Use them with [Image Evaluations](/Evaluations/image-evaluations/Datasets) to build automated visual quality pipelines.

## Why Image Evaluators Matter

Without systematic visual assessment, you cannot measure improvement or catch regressions in image generation and editing systems:

| Challenge            | How Image Evaluators Help                                            |
| -------------------- | -------------------------------------------------------------------- |
| Subjective quality   | Multimodal LLM judges provide consistent, scalable visual assessment |
| Format validation    | Code evaluators enforce dimensions, file types, and size constraints |
| Text-image alignment | Specialized evaluators verify images match prompts or descriptions   |
| Editing accuracy     | Comparison evaluators measure before/after quality and fidelity      |

## Image Evaluator Types

Netra offers several approaches to image evaluation, each suited to different use cases:

<CardGroup cols={2}>
  <Card title="Multimodal LLM as Judge" icon="brain">
    Vision-capable models evaluate subjective criteria like composition, style, and semantic accuracy. Best for nuanced visual quality assessment.
  </Card>

  <Card title="Image Analysis" icon="image">
    Rule-based evaluators that assess technical properties—dimensions, file size, format, and aspect ratio—without AI judgment.
  </Card>

  <Card title="Image-Text Coherence" icon="link">
    Measures alignment between image content and text descriptions. Verifies generated images match prompts or expected visual output.
  </Card>

  <Card title="Code Evaluator" icon="code">
    Custom JavaScript or Python logic for specialized visual checks—pixel analysis, color distribution, or domain-specific validation.
  </Card>
</CardGroup>

## Library Evaluators

Netra's evaluator library includes pre-built image evaluators organized by category. Navigate to **Library → Evaluators** and browse the **Multimodal** category.

| Evaluator                          | Type            | What It Measures                                                          |
| ---------------------------------- | --------------- | ------------------------------------------------------------------------- |
| **Image Analysis**                 | multimodal      | Evaluates how faithfully an AI-generated image represents the text prompt |
| **Image Aspect Ratio Check**       | multimodal-rule | Validates aspect ratio matches target (e.g., 16:9, 4:3)                   |
| **Image Dimensions Check**         | multimodal-rule | Validates exact pixel dimensions (e.g., 800x800)                          |
| **Image Editing Quality**          | multimodal      | Evaluates how well an edited image reflects intended changes              |
| **Image Helpfulness**              | multimodal-llm  | Measures how much images help users understand text                       |
| **Image Output Format Validation** | multimodal-rule | Validates file format (JPEG, PNG, GIF, WEBP, BMP, TIFF)                   |
| **Image Size Check**               | multimodal-rule | Verifies file size within threshold                                       |
| **Image-Text Coherence**           | multimodal-llm  | Evaluates how well images and text work together                          |
| **Image-Text Reference**           | multimodal-llm  | Checks whether text accurately describes what's shown in images           |

<Info>
  These evaluators are available in the Library under the **Multimodal** category. Click **Add** on any evaluator card to customize it and save to **My Evaluators**. The `multimodal-rule` evaluators perform deterministic checks without AI judgment, while `multimodal` and `multimodal-llm` evaluators use vision-capable models for subjective assessment.
</Info>

## Creating Custom Image Evaluators

Click **Add Custom** in the top right corner to open the creation wizard.

<Steps>
  <Step title="Choose Evaluator Type">
    Select **LLM as Judge** for vision-based assessment or **Code Evaluator** for deterministic checks.
  </Step>

  <Step title="Select Turn Scope">
    Choose **Image** as the turn scope. This configures the evaluator to accept image inputs alongside text.
  </Step>

  <Step title="Configure the Evaluator">
    For **LLM as Judge**:

    * Write a prompt that includes `{{image}}` and `{{expected_output}}` placeholders
    * The `{{image}}` variable will be populated with the image from your evaluation item
    * Select a vision-capable model (e.g., GPT-4o, Claude 3.5 Sonnet)

    For **Code Evaluator**:

    * Write a handler function that receives the image data and expected output
    * Return a score or boolean based on your validation logic

    **Example LLM prompt:**

    ```
    Analyze this image and evaluate its quality based on the following criteria:

    Image: {{image}}
    Expected Description: {{expected_output}}

    Rate the image on:
    1. Composition (0-10)
    2. Clarity (0-10)
    3. Style consistency (0-10)
    ```
  </Step>

  <Step title="Set Output and Pass Criteria">
    | Output Type   | Configuration                                        |
    | ------------- | ---------------------------------------------------- |
    | **Numerical** | Set a threshold (e.g., average score >= 7.0 to pass) |
    | **Boolean**   | Pass if all criteria meet minimum requirements       |
  </Step>

  <Step title="Test in Playground">
    * Upload a sample image or provide an image URL
    * Run the evaluator in real-time
    * Refine your prompt until results are consistent
  </Step>
</Steps>

### Code Evaluator Example

For deterministic image validation, use a Code Evaluator:

**JavaScript example:**

```javascript theme={null}
function handler(input, output, expectedOutput) {
  // output contains the image metadata
  const minWidth = 512;
  const minHeight = 512;
  const allowedFormats = ['image/jpeg', 'image/png', 'image/webp'];

  return (
    output.width >= minWidth &&
    output.height >= minHeight &&
    allowedFormats.includes(output.format)
  );
}
```

**Python example:**

```python theme={null}
def handler(input, output, expectedOutput):
    min_width = 512
    min_height = 512
    allowed_formats = ['image/jpeg', 'image/png', 'image/webp']

    return (
        output['width'] >= min_width and
        output['height'] >= min_height and
        output['format'] in allowed_formats
    )
```

## Variable Mapping for Image Evaluators

When configuring evaluators on [evaluations](/Evaluations/image-evaluations/Datasets), map variables to connect evaluator inputs to your image data:

| Variable                     | Description                                     | Source                                  |
| ---------------------------- | ----------------------------------------------- | --------------------------------------- |
| `{{input}}`                  | Text prompt describing what to generate or edit | Evaluation item input                   |
| `{{output_image}}`           | Generated or edited image URL                   | Task function return value              |
| `{{input_image}}`            | Original image (for editing tasks)              | Metadata `input_image` field            |
| `{{reference_images}}`       | Array of reference images for comparison        | Metadata `reference_images` field       |
| `{{text_caption}}`           | Description of the image                        | Metadata `text_caption` field           |
| `{{expected_dimensions}}`    | Expected pixel dimensions                       | Metadata `expected_dimensions` field    |
| `{{expected_aspect_ratio}}`  | Expected aspect ratio                           | Metadata `expected_aspect_ratio` field  |
| `{{expected_format}}`        | Expected file format                            | Metadata `expected_format` field        |
| `{{expected_max_file_size}}` | Maximum allowed file size                       | Metadata `expected_max_file_size` field |
| `{{input_edit_instruction}}` | Editing instruction applied to the image        | Metadata `input_edit_instruction` field |

<Info>
  The `output_image` variable is automatically populated from your task function's return value. All other image-related variables are sourced from the evaluation item's metadata fields.
</Info>

## Best Practices

### Writing Effective Image Evaluation Prompts

* **Be specific about criteria**: Define exactly what constitutes good composition, clarity, or style
* **Provide reference descriptions**: Include detailed expected output descriptions
* **Use structured scales**: Rate specific aspects (composition, color, style) rather than overall quality
* **Test with diverse images**: Validate with various image types, styles, and edge cases

### Choosing the Right Evaluator Type

| Use Case                                       | Recommended Evaluator          |
| ---------------------------------------------- | ------------------------------ |
| "Does this image look professional?"           | Image Analysis (multimodal)    |
| "Is the image at least 1024x1024?"             | Image Dimensions Check         |
| "Does the aspect ratio match 16:9?"            | Image Aspect Ratio Check       |
| "Does the image match this description?"       | Image-Text Coherence           |
| "Is the file size under 5MB?"                  | Image Size Check               |
| "Is this a valid PNG file?"                    | Image Output Format Validation |
| "Did the edit preserve the original style?"    | Image Editing Quality          |
| "Do the images help explain the text?"         | Image Helpfulness              |
| "Does the text describe what's in the images?" | Image-Text Reference           |

### Testing Before Deployment

Always use the Playground before adding evaluators to production evaluations:

* Test with representative sample images from your use case
* Include edge cases (low resolution, unusual formats, complex scenes)
* Verify pass/fail thresholds produce expected results
* Test with both successful and failure scenarios

## Related

* [Evaluators Overview](/Evaluators/overview) - Understand the full evaluator framework
* [Text Evaluators](/Evaluators/text-evaluators) - LLM-as-Judge, code, and rule-based text evaluators
* [Voice Evaluators](/Evaluators/voice-evaluators) - TTS, STT, and conversational evaluators
* [Evaluation Overview](/Evaluations/Evaluation-overview) - Evaluations, test runs, and the evaluation framework
* [Image Evaluations](/Evaluations/image-evaluations/Datasets) - Create image test cases for your evaluators
* [Image Test Runs](/Evaluations/TestRuns) - View evaluation results for image evaluations
* [Quick Start: Evaluation](/quick-start/QuickStart_Evals) - Get started with evaluations