# Set KPI and ROI

The Composabl benchmarking feature allows you to compare the performance of different multi-agent systems against key performance indicators (KPIs). This tool helps you evaluate your agent system's effectiveness, track its improvements, and calculate potential return on investment (ROI).

Benchmarks are generated only after training is complete. You won't see benchmark data while training progresses, even for long-running jobs. If a training job fails, no benchmark data will be generated.

Note: Benchmarking does not influence agent system training, goals, or rewards. It is purely an analysis tool that helps you answer the question: "Based on how this agent system was trained and how we believe it should perform in real life, how much money is this agent system generating?"

## Getting Started

### Define the KPI for Your Project

The key performance indicator (KPI) for your project is the top-level objective that you will use to compare and evaluate your multi-agent systems' performance.&#x20;

To define the KPI for your project, choose the sensor variable that represents the metric that best indicates performance success. For most processes, this is a metric with a clear business impact, such as product yield or energy use.

### Setting up the KPI

&#x20;To set up the KPI:

1. Navigate to the Benchmark page from the main dashboard
2. Click the "Settings" button in the top right of the KPI Performance Metrics section
3. In the "Set up KPI" section, configure the following:
   * **Sensor Name**: Select a sensor or preceptor variable to track (e.g., "Eps\_Yield")
   * **Target Value**: Set the desired target for this KPI (e.g., 0.33)
   * **Benchmark Unit**: Select the unit of measurement (e.g., "% of Max Theoretical Yield"). Note that this is for UI readability only and doesn't affect calculations.

<figure><img src="https://2216884851-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6X1hFdU8qyyv3sgW7kkX%2Fuploads%2F6pzgu1w2tcrbECCeMjop%2FSet%20KPI%20and%20ROI.png?alt=media&#x26;token=9ae63312-73fe-46f9-99f3-54a8909daf5b" alt=""><figcaption></figcaption></figure>

### Setting up ROI Calculations

Return on Investment (ROI) calculations help quantify the financial impact of your agent systems' performance. The ROI is calculated based on the monetary value of the difference between your target value and the actual value of the KPI generated by that agent system, minus training costs.

1. In the Settings modal, navigate to the "Enter ROI criteria" section
2. Define the conditions for calculating ROI:
   * **When**: Select the metric to track (e.g., "Eps\_Yield")
   * **Condition**: Choose how the metric changes (e.g., "increases by")
   * **Percentage**: Enter the percentage change that matters (e.g., 2%)
   * **Value**: Set the financial value of this change (e.g., $1,000,000)
   * **Period**: Select the time period (e.g., "per year")

For example: "When Eps\_Yield increases by 1% of max theoretical yield, it is worth $1,000,000 per year."

You can change KPI and ROI settings at any time after training completes, and the benchmark results will be automatically recalculated. This allows you to explore different business scenarios without retraining your agent systems.

## Reading the Benchmarking Dashboard

<figure><img src="https://2216884851-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F6X1hFdU8qyyv3sgW7kkX%2Fuploads%2F8yQ4fYy2tkqobDY2EFkg%2FCleanShot%202025-03-04%20at%2011.59.40.png?alt=media&#x26;token=57c8d768-abdc-4d50-a5a7-995cce851ccf" alt=""><figcaption></figcaption></figure>

### Understanding the Overview Panel

At the top of the benchmarking page, you'll find an Overview panel containing three key metrics:

* **Highest performing Agent System**: Displays the outcome of the highest performing agent system
* **Lowest performing Agent System**: Displays the outcome of the lowest performing agent system
* **ROI**: Shows the calculated financial return based on your KPI settings

Each metric displays both the value and the relevant unit (e.g., "% of Max Theoretical Yield").

### Bar Graph Visualization

The bar graph in the KPI Performance Metrics section provides a visual comparison of your agent systems:

* Each bar represents an agent system's performance on your selected KPI (all bars correspond to the same KPI)
* The height of the bar indicates the specific value of the KPI for that agent system (Y-axis)
* Horizontal dashed lines show benchmark averages
* The actual value of the KPI for each agent system is shown above the respective bar

### Performance Ranking Table

Below the graph is a detailed table showing:

* **Agent System Number**: Sequential ID for each agent system
* **Agent System Name**: Name or description of the agent system
* **ROI**: Calculated financial return based on the agent system's performance
  * Green arrows (↑) indicate positive ROI with the percentage and absolute value
  * Red arrows (↓) indicate negative ROI with the percentage and absolute value
* **Eps\_Yield (variable) per % of Max Theoretical Yield**: Performance metric with indicators for highest agent system performers
