Agent Ontologies
From Pipelines to Networks: Understanding AI Agent Architectures
Orchestra is a lightweight multi-agent framework designed for reliability, capability, composability, and ease of use. It revolves around the concept of breaking challenging tasks into cascading hierarchies of sub-steps and subtasks performed by a network of AI agents.
So what actually is an “AI agent”?
Definitions vary, but it can be said that an AI Agent is a cognitive entity powered by a reasoning engine like a language model, situated within an information environment, capable of perceiving inputs, maintaining an internal state, and acting upon the environment to achieve its goals without human intervention.
The term “autonomous agents” has roots in the field of Artificial Life and Artificial Intelligence, and was examined thoroughly in Jeffrey S. Rosenschein’s dissertation at Standford University in 1985; Rational Interaction: Cooperation Among Intelligent Agents.
That said, the agency of these cognitive entities can vary. Reasoning engines can exist in simple loops that are deterministically linked together which don't meet all of the required criteria defining an "agent", or in large networks of communicating agents with greater degrees of autonomy.
Ontologies
Below is a brief walkthrough of some of the architectures that can be built with Orchestra, from more deterministic workflows to hierarchical multi-agent systems with agents that have greater degrees of agency.
Task-Tool Pipeline
This is a pipeline where tasks are assigned a language model and attributes, but not actually assigned a separate agent instance or self-determined tool use. Task outputs are then directly fed into functions, the outputs of which can be fed back into subsequent tasks for further application of reasoning and transformation of information. These are useful when you need an LLM-in-the-loop determining aspects of the flow of the application, but the flow itself is deterministic and entirely known beforehand. Less flexible, but highly predictable, it's somewhat agentic but it doesn't quite meeting all the criteria of an agent.
Imagine a document processing system that handles legal contracts. The pipeline starts with an LLM analyzing the contract to extract key search terms. These terms automatically flow into a legal database search, pulling relevant precedents and regulations. The search results are then progressively summarized by the LLM, translated into different languages for international clients, and automatically saved to the client's file system. While the LLM provides the reasoning, it's following a predetermined path. It supports smaller and more cost effective models and is the least likely to go off the rails. That said, it's not quite meeting the definition of an agent yet.
Task-Tool Pipeline with External Personas
This is a pipeline where the tasks are assigned an agent with a language model but not assigned tools. The task is defined and assigned an agent, and personas can be assigned to multiple tasks. Outputs are still directly fed into functions. This is useful when you have multiple tasks that can be assigned the same agent persona, but still want to maintain strict control of the flow of operations.
For example, a content creation pipeline where the same "editor" agent persona reviews multiple different types of content: blog post drafts, social media posts, and email newsletters. The agent maintains consistent voice/style across different tasks, but the workflow is still predetermined.
Chat Task with an Agent
This is a single task that supports conversation with a single agent. The agent can determine tool calls but there is no discrete, pre-determined task flow.
An example could be a research assistant agent that can engage in back-and-forth dialogue about a topic, using tools like web search or document lookup when needed, but without a predetermined sequence. The conversation flow is dynamic based on the user's needs, and functions are called when needed, according to the discretion of the agent. The conversation flows naturally, with the agent deciding which tools to use based on how the discussion evolves.
Agent Workflows
This is a workflow where the tasks are predetermined and assigned an agent with a language model given tools. Functions are called by agents under their own discretion. This is useful for when agents need flexibility of which function to call, but the sequence of tasks is still predetermined and fixed.
An example of a workflow could be a data analysis agent that decides which statistical tools to apply based on the dataset characteristics - it might run tests, then decide between methods, and select appropriate visualizations, making decisions at each step. The flow of tasks is still pre-determined, but the tools used are up to the agent.
Agent with Callable Pipelines
This is an agent that has tools that it can call that themselves are deterministic pipelines, ideal for repetitive actions and processes that need to be initiated at the right time with the right inputs.
For example, imagine an agent with the ability to call deterministic pipelines. Given different inputs it could decide which pipeline to initiate with the necessary inputs. The pipeline becomes callable with inputs just like a simple tool. It could then pass a URL of a research paper which then initiates a pipeline that downloads the content, splits it into chunks, translates each chunk and saves the accumulated translations to file, or it could initiate standardized workflows like "create new project", which would involve creating folders, setting up documentation, and sending notifications, executing a series of predetermined steps, while the agent maintains higher-level oversight, determining which pipeline to call, when, and their inputs.
Orchestras
Orchestras are defined by having multiple layers, or hierarchies, in the group of agents. Unlike most frameworks that just route messages to and from agents, or simply ask other agents questions, in Orchestra agents can actually create workflows dynamically by decomposing a complex task into subtasks, assign agents to subtasks, and controlling data flows between subtasks. It allows an agent role of Conductor to essentially orchestrate its assigned agents dynamically according to its inputs. Some workflows it issues may involve all of its agents, while others may only require a few tasks to be assigned to a smaller group of its agents.
Think of a software development project where a conductor agent acts like a technical project manager. When given a new feature request, they'll break it down – having a System Designer sketch out the architecture, an API Designer draft the interfaces, Developers handle the implementation, QA ensure quality, and Technical Writers document everything. The conductor orchestrates this complex sequence based on the requirements defined in its input, ensuring the flow follows a logical sequence, managing dependencies between teams, and adapting the workflow based on how the project evolves. They actively coordinate and adjust subtasks for worker agents and managing dependencies based on the project's needs.
Network of Agents
Remaining somewhat theoretical, is the multi-hierarchical network-of-agents. Representing a large network of interacting agents that are capable of carrying out tasks and communicating with each other bidirectionally, these multi-layered hierarchies of agents represents entire organizations or aspects of “mind”.
An example of a network-of-agents could be a newsroom where multiple hierarchies of agents work together: editorial teams (writers, editors, fact-checkers), production teams (formatters, image processors), and management teams (coordinators, quality control) all interact in complex patterns to produce publications. Based on a single input, the conductor layers conduct and orchestrate their own team members in a multi-hierarchical system representing entire organizations.
Orchestra
The range of agent designs in Orchestra - from basic task-tool pipelines to complex networks - shows how the framework can support different levels of agent independence and complexity. Simple designs offer predictability and efficiency for straightforward tasks, while advanced setups allow for adaptive problem-solving through coordinated agent teamwork.
Orchestra's way of organizing agents in hierarchies and breaking down tasks provides a foundation for building AI systems that can grow with your needs. Whether you're building simple LLM-powered workflows or managing complex multi-agent teams, the framework helps you balance capability and control, letting you match agent independence to your specific needs.
As multi-agent systems continue to develop, Orchestra's building-block approach makes it well-suited for exploring new ways agents can work together, while staying true to its core goals of reliability and ease of use. Check out the GitHub repo and the documentation to get started.