Orchestrate Skill Agents

Skill agents can be arranged in sequences or hierarchies, in skill groups, or as coordinated skills that output multiple decisions together. The orchestration structures reflect common design patterns that can be used to accelerate the design and creation of agent systems.

Orchestrate Skills in Hierarchies and Sequences

For some agent system designs, the task will be broken down into different skill agents that each control the system under certain conditions. For these agent systems, a special skill called an orchestrator chooses the right skill agent at the right time to accomplish the task. Orchestrators are the specialized supervisor skill agents that orchestrate the skill agents together, determining which skill agent to activate based on the conditions the system needs to respond to.

To add an orchestrator to an agent system, drag the orchestrator into your agent system above the skills layer.

You will then be prompted to configure the orchestrator.

Orchestrators can be trained with DRL just like the other skill agents, and you set goals for them the same way. Click on Configure Orchestrator to be taken to the menu to set your orchestrator’s goals. Note that the goals of the agent’s top-level orchestrator should be the same as the goals for the agent system as a whole.

You will also be prompted to choose between an additional set of options that correspond to two separate Machine Teaching design patterns.

• Fixed-order sequence: perform the skills in a set order. This is used in the , a design pattern that is useful for tasks that involve fixed sequences of actions.

• Variable order sequence: perform the skills in any order based on the orchestrator’s determination. This is used in the , a design pattern that is useful for tasks that require different control strategies to be used in different situations or conditions.

Orchestrate Skill Agents in Groups

Unlike agent system designs that use an orchestrator to assign control to skills one at a time, agent systems with skill groups use skills working together to make decisions.

Skill groups always consist of two or more skill agents. To create a skill group, simply drag the second skill under the first, and a skill group will automatically be created.

Skill groups are used for the , where one skill determines what the action should be and a second skill then “turns the knobs” to implement the decision.

A common plan-execute design is the set point pattern, where a DRL agent skill determines what the set point of a system should be, and then a traditional automation technology (a controller that uses model predictive control or optimization) determines what actions are necessary to get the system to the set point. For example, in the industrial mixer use case, the DRL plan skill agent decides what temperature the liquid in the tank should be. Then the MPC execute skill agent determines what temperature of coolant needs to be flowed around the tank to achieve the desired set point.

In the industrial mixer example, the DRL skill agent is able to train effectively because the actions of the MPC controller are predictable. That means that it can practice and learn knowing that variations in performance are due to its own actions.

In agent systems with multiple DRL skill agents arranged in plan-execute patterns, Composabl will always train the skills from the bottom to the top. In other words, the execute skill will have to achieve competence before the plan skill will start training. That allows each skill to effectively interpret the feedback from the system without confusion from each other.

Orchestrate Coordinated Skills

Some tasks require multiple skill agents to work together on a single decision, but in parallel rather than in sequence. Skill agents for these tasks use coordinated skills that learn to take action together toward a goal. Also known as Multi-Agent Training, coordinated skills are trained using a coach, rather than a teacher.

Examples of Coordinated Skill Agents

Traffic Optimization: Enhancing traffic flow and safety by teaching individual vehicles to navigate optimally and cooperate with each other.

Collaborative Robotics: Enabling robots to work together on tasks such as assembly in manufacturing or coordination in logistics.

Smart Grids: Optimizing energy distribution by having agents represent power plants, storage, and consumers to improve efficiency and stability.

Multiplayer Games: Creating adaptive and intelligent NPCs that can offer dynamic challenges to players in competitive or cooperative game settings.

Communication Networks: Improving network performance by optimizing resource allocation and traffic routing through agents representing network components.

Environmental Management: Balancing economic, ecological, and social goals in land use and resource management by simulating stakeholders as agents.

Healthcare Logistics: Strategizing resource allocation and treatment plans in scenarios like pandemics by considering the actions of hospitals, pharmacies, and patients as agents.

Supply Chain Optimization: Minimizing costs and delivery times in supply chains by coordinating agents representing various stages of the supply chain process.

Orchestrate Coordinated Skills with the SDK

Coordinated skill agents are not yet available in the UI. In the SDK, we have expanded the API to integrate Coordinated Skills through the add_coordinated_skill method on your agent. This method accepts a new class that gets configured, named CoordinatedSkill, just as with the Teacher or Controller classes we implement this class by inheriting from the Coach class.

The coordinated skill agent will now take the incoming observation and action spaces and pass it to the sub-skills as a shared environment observation and action taking. The sub-skills will then return their observations and actions, which will be passed back to the coordinated skill agent. The coordinated skill agent will then return the combined observations and actions to the agent system.

python
# ####################################################################################################
# Define the Coordinated Coach
# ####################################################################################################
class CoordinatedCoach(Coach):
    def __init__(self):
        self.counter = 0

    def compute_reward(self, transformed_ sensors, action, sim_reward):
        """
        Computes the reward for the given transformed observation and action
        :param transformed_ sensors: The transformed observation
        :param action: The actions dict
        :param sim_reward: The reward from the simulation
        :return: The reward, as a dictionary, with each key the sub-skill name and the value the reward
        """
        self.counter += 1
        return 1

    def compute_success_criteria(self, transformed_ sensors, action):
        # keep the episodes short to make testing quicker
        return self.counter > 100

    def compute_termination(self, transformed_ sensors, action):
        # keep the episodes short to make testing quicker
        return self.counter > 150

    def transform_action(self, composabl_ sensors, action):
        return action


# ####################################################################################################
# Construct your Agent
# ####################################################################################################
s1 = Skill("skill1", IncrementTeacher)
s2 = Skill("skill2", IncrementTeacher)

a = Agent()
a.add_coordinated_skill(CoordinatedSkill(
  "my-coordinated-skill",
  CoordinatedCoach,
  [s1, s2]
))