Introducing Orchestra
A New Cognitive Architecture
Orchestra is a multi-agent framework built on the principle that complex systems are best created through composition, like how a symphony emerges from the careful arrangement of individual musical elements. Orchestra represents a cognitive architecture designed to manage and distribute cognitive load effectively.
“Composability refers to a property of software that allows smaller pieces to be assembled into larger compositions. Composability depends on interoperability, but it takes the idea further by combining systems the way one builds using Lego bricks. Composing software is like making music or writing where larger creations like symphonies or novels are composed of smaller parts like strings of notes or words.”
— Chris Dixon, Read Write Own
Both people and agents benefit from breaking down complex challenges into composable pieces. This approach allows agents to tackle tough problems by distributing cognitive load across multiple steps and participants.
Reducing Cognitive Load
When faced with a complex task, the cognitive load can be significant, requiring the LLM to juggle multiple variables, contexts, and potential actions simultaneously. By spreading the cognitive load, Orchestra ensures that each step in the task execution process is focused and manageable.
The approach involves decomposing a large task into smaller, sequential steps or instances. Each step is designed to handle a specific aspect of the task, allowing the LLM to concentrate its reasoning and decision-making capabilities on a single responsibility at a time. This reduces the overall cognitive burden and enhances the clarity and coherence of the LLM's responses.
Cognitive load doesn’t just overwhelm models into repetition or hallucination, it also detracts from the quality of an output for LLMs that aren’t totally overwhelmed.
This method allows the LLM to build upon previous results, maintain a clear execution history, and make informed decisions based on well-defined contexts. By implementing a cascade of steps, Orchestra leverages structured thinking patterns akin to human problem-solving strategies. The result is a more reliable and predictable performance, with improved accuracy and reduced risk of errors.
Orchestration: Conduct and Compose
Orchestra's orchestration capabilities are unique in that they are implemented as tools that can be assigned to any agent, allowing for flexible orchestration patterns. The framework is built around the concept of task decomposition and coordination, where agents can break down complex problems into smaller, manageable tasks on the fly. These orchestration capabilities are implemented as tools that can be assigned to any agent.
The Conduct Tool
The Conduct tool provides a mechanism for enabling agents to delegate tasks to other agents through a tool-based architecture. This delegation capability isn't restricted to a central orchestrator; instead, it's implemented as a tool that can be granted to any agent. This allows for hierarchical delegation patterns, where agents can act as both executors, fulfilling tasks themselves with their tools, and conductors, orchestrating other agents to carry out more complex cross-domain tasks. The act of conducting involves breaking down a task into subtask, assigning the right agents for each job, and specifying which subtask outputs are inputs for downstream tasks. This allows the orchestrator to create complex workflows with one tool-call, and receive the outputs. If necessary, it can reissue a flow of tasks after receiving the results in an iterative and recursive process.
The Compose Tool
The Compose tool represents a higher-level planning layer that pairs with the Conduct tool. Rather than immediately jumping into task delegation, the Compose tool enables agents to first create comprehensive plans when dealing with complex objectives. By separating the planning phase from execution, agents can focus on high-level strategy and optimal task organization. The compose tool itself is only necessary when the task at hand involves a level of complexity, and its separation from the conduct tool allows the orchestrator agents to determine themselves if the request necessitates high-level and comprehensive planning. For tasks that are straightforward delegations (one agent doing a task or two), the orchestrator can directly instruct agents to carry out a task. But for tasks that may involve multiple agents in a more complex request, the orchestrator agent can call on the conduct tool to enrich and expand the orchestration itself by composing before conducting.
Modular Orchestration Patterns
The tool-based orchestration system in Orchestra serves as a building block for creating complex agent hierarchies and organizational structures. Organizations can be modeled with department-level orchestrators managing their specific agent teams, while higher-level orchestrators coordinate across departments. This modularity allows for the creation of specialized orchestration patterns tailored to specific domains or organizational needs.
Agent Communication and Coordination
Agents can communicate bi-directionally, share context, and collaboratively work on complex problems. They can spawn sub-tasks, coordinate responses, and maintain both local and shared state. The orchestrator doesn’t have to carry all of the context from one agents output to the next because it can assign the outputs of one agents work directly as an input in a subsequent task, preventing itself from becoming a bottleneck in the data flow between delegated tasks.
This is true orchestration—not just routing messages, but conducting a symphony of agents working in harmony.
Task Execution
Orchestra's approach to LLM task execution is distinct from traditional function-calling methods commonly used in other frameworks, which impose a significant cognitive load on the LLM. Instead, Orchestra employs a phased execution model that clearly separates concerns through its task execution engine.
Tool Loop Phase
During this phase, tool usage instructions and available tools are clearly delineated, and the original task instruction is insulated with distinct delimiters. The LLM is given only two options: make tool calls or exit the loop. As tool results accumulate within the same context, a clear execution history is built.
Final Response Phase
Once tool usage is complete, Orchestra transitions to the final response phase. Tool instructions and delimiters are removed from the context, and the original instruction is presented cleanly alongside the tool results. This allows the LLM to focus solely on generating the final response without the distraction of tool usage instructions.
This separation of phases reduces the cognitive load on the LLM, as it never has to consider tool usage and final response generation simultaneously. The context becomes clearer, with tool results presented cohesively rather than scattered across function call messages. Decision-making is more focused, leading to more reliable tool usage, better-contextualized responses, clearer reasoning patterns, reduced likelihood of hallucination or confusion, and more predictable agent behavior.
Simplified Tool Integration with Docstrings
Orchestra is built around the idea of managing mental load to support more reliable and predictable agent behavior. The task execution model takes advantage of docstrings to construct prompts for tool usage, offering a more natural and simplified method for instructing the LLM on how to call functions as opposed to the more rigid and complex JSON schemas required by traditional function-calling APIs.
Docstrings are inherently written in natural language, making them more readable and easier to understand for both humans and LLMs. By using docstrings, Orchestra conveys the necessary information about a function's purpose, parameters, and expected behavior without the overhead of detailed JSON schemas. This streamlined communication ensures that the LLM can execute functions efficiently, without being bogged down by the intricacies of a rigid schema.
Orchestra's ability to work with standard Python docstrings means that functions from various sources—whether they're LangChain tools, CrewAI components, Composio utilities, or direct SDK functions—can be integrated into Orchestra agents with minimal or no modification. This compatibility-first approach opens up a wide ecosystem of tools and functions that can be immediately utilized within the framework, reducing the development overhead typically associated with tool integration. Instead of writing custom schemas, adapters, or wrappers for each tool, developers can focus on the functionality and logic of their applications.
Tool-Call Summaries and Rationales
One advantage of Orchestra's approach over rigid function-calling APIs is the ability to extend and customize the schema for tool interactions. For example, Orchestra implements a tool_summaries feature that allows the LLM to provide explanatory rationales for its tool usage decisions. When enabled, each tool call includes a clear explanation of why the tool is being called at that moment, along with function name and parameters.
This kind of metacognitive annotation isn't possible with standard function-calling APIs, which are constrained to a fixed schema of name, parameters, and results. By managing its own tool-calling structure, Orchestra can introduce novel fields and features that enhance transparency and user understanding. Users can understand the LLM's reasoning process in real-time, making the agent's decision-making process more interpretable and trustworthy.
Orchestra represents a shift in how AI agents collaborate and solve complex problems. By focusing on composability, task decomposition, and flexible orchestration through assignable tools, it enables the creation of workflows that mirror human problem-solving strategies.
Visit orchestra.org, view the source on GitHub, and follow @orchestraorg.