Create Skill Agents

You can use Composabl to create skill agents that learn with deep reinforcement learning. Configure the Composabl teacher by setting goals, constraints, and success criteria for your skill agent. Composabl then turns these into reward functions and trains the skill agent.

Create a New Skill Agent

To create or edit a skill agent, follow these steps:

1. Navigate to the Skill Agents page

2. Click on a skill agent, or click + to create a new skill agent

3. Configure the skill agent

Use the dropdown menus to select the variables and then define the parameters for each goal, constraint, or success criterion you want to include.

Choose Implementation Method

To create a new skill agent that learns with deep reinforcement learning, select Teacher under the Implementation Method dropdown. The other option, Controller, is used for a programmed skill agent that is configured with the Composabl SDK and published to the Agent Orchestration Studio.

Configure Goals

Goals define what a skill agent should do. Goals apply to one of the sensor variables and are defined using one of three possible directives:

• Maximize: Maximize the value of the variable

• Minimize: Minimize the value of the variable

• Maintain: Keep the variable at a specified value or track a specified variable (such as a set point)

For example, for the industrial mixer, we want to maximize the yield of the product over the course of an entire episode, eps_yield.

Configure Constraints

Constraints set the boundaries for the skill agent. They define rules that the skill agent must follow during operation. Constraints are defined using one of two possible directives:

• Avoid: The skill agent learns to keep the variable from reaching a specified value or range through withholding rewards

• Terminate: When the actions of the skill agent lead to certain conditions within a variable, the skill agent has failed and must stop and start a new episode

For example, we want to avoid the temperature, T, getting above 400 degrees Kelvin in the industrial mixer use case.

Configure Success Criteria

Success criteria tell the skill agent when it's doing something right. They are defined using one of two possible directives:

• Approach: The skill agent learns to get close to a specified value by getting increased reward

• Succeed: When the success criteria are achieved, the session ends, and a new one begins so that the skill agent can keep practicing and learn to win every time