Hierarchical Decomposition
A multi-level agent hierarchy where manager agents decompose complex objectives into progressively smaller subtasks, delegating to sub-managers and leaf workers in a tree structure.
function HierarchicalDecomposition(objective, depth=0, max_depth=3):
if depth >= max_depth or is_atomic(objective):
// Leaf node: execute directly
return Worker.execute(objective)
// Manager: decompose into subtasks
subtasks = Manager.decompose(objective)
// Delegate to children (parallel where possible)
results = parallel_map(subtasks, fn(subtask):
return HierarchicalDecomposition(
subtask, depth + 1, max_depth
)
)
// Validate and aggregate results
validated = Manager.validate(results)
return Manager.aggregate(validated)
function Manager.decompose(objective):
return LLM(
"Break this objective into 2-5 independent subtasks: "
+ objective
)When to Use
- Large-scale tasks that naturally decompose into multiple independent workstreams
- Tasks requiring both breadth (many subtasks) and depth (complex subtasks)
- When different levels of abstraction require different agent capabilities
- Systems that need to parallelize work across many workers for throughput
- Enterprise workflows mirroring organizational hierarchies
When NOT to Use
- Tasks where the decomposition is harder than the task itself
- Small or medium tasks where a flat supervisor pattern suffices
- When tight feedback loops between components are required
- Cost-sensitive scenarios — hierarchy multiplies LLM calls exponentially
- Creative tasks where emergent holistic quality matters more than component quality
Multi-Level Agent Hierarchies
The Hierarchical Decomposition pattern extends the supervisor pattern into a tree structure. Instead of a single supervisor managing all workers, you build a hierarchy of managers — a top-level executive agent decomposes the objective into major workstreams, mid-level managers further decompose each workstream into specific tasks, and leaf-node workers execute the atomic tasks.
This pattern directly mirrors how large organizations operate. A CEO sets strategic objectives, VPs decompose them into departmental goals, team leads create specific work items, and individual contributors execute them. Each level of the hierarchy operates at a different level of abstraction — higher levels deal with strategy and coordination, lower levels deal with execution and details.
The architecture consists of:
- Executive Agent (Root): Receives the top-level objective and decomposes it into 2-5 major workstreams. Uses the most capable model available because its decisions have the highest impact.
- Manager Agents (Intermediate Nodes): Each manages a workstream. They further decompose their assigned workstream into specific tasks, delegate to workers or sub-managers, aggregate results, and report back to their parent.
- Worker Agents (Leaf Nodes): Execute atomic tasks. They are specialized, narrowly scoped, and optimized for speed and cost. They report results back to their parent manager.
The depth of the hierarchy depends on the complexity of the task. Most practical systems use 2-3 levels; deeper hierarchies add coordination overhead that outweighs the benefits of further decomposition.
Task Decomposition Strategies
The quality of the hierarchical system depends entirely on the quality of task decomposition at each level. Several strategies can be applied:
Functional decomposition: Break the task by function or capability — one branch handles research, another handles writing, another handles validation. This maps naturally to teams with different expertise and works well when the task has clearly separable functional concerns.
Data-parallel decomposition: When the same operation must be applied to many data items (analyze 50 documents, process 100 customer records), the manager splits the data among workers who execute in parallel. This is the simplest form of decomposition and scales linearly.
Pipeline decomposition: Break the task into sequential stages — research, draft, review, revise, finalize. Each stage is managed by a sub-manager with its own workers. The output of one stage feeds into the next. This works well for content creation, data processing, and any task with natural phases.
Recursive decomposition: The manager applies the same decomposition logic recursively until subtasks are small enough for a single worker. This is elegant but requires careful termination criteria to prevent over-decomposition (tasks so small that the coordination overhead exceeds the work itself).
In practice, most systems combine strategies: functional decomposition at the top level (research branch, analysis branch, writing branch), with data-parallel decomposition within each branch (split documents across workers), and pipeline decomposition for the final assembly.
Result Aggregation and Quality Control
As results flow up the hierarchy, each manager must aggregate outputs from its children into a coherent result for its parent. This aggregation is where much of the value — and risk — of the hierarchical pattern lies.
Simple concatenation: For data-parallel tasks, results are concatenated or merged. A manager that split 50 documents across 5 workers simply collects all 50 analyses. This is fast but may produce inconsistencies if workers interpreted the task differently.
Synthesis: For functional or pipeline decomposition, the manager must synthesize diverse outputs into a unified result. A research manager combines findings from multiple search workers into a coherent research brief. This requires an LLM call and adds latency, but produces much higher quality output.
Quality validation: Before aggregating, the manager should validate each worker's output. Does it address the assigned subtask? Is it in the expected format? Is the quality sufficient? Low-quality outputs can be sent back for revision or routed to an alternative worker.
Conflict resolution: When workers produce contradictory results (different workers find conflicting data), the manager must resolve the conflict — through additional research, by weighing source reliability, or by presenting both perspectives to the parent with a recommendation.
The aggregation strategy should be defined explicitly at design time. Ad-hoc aggregation (letting the manager LLM figure it out) works for prototypes but produces unpredictable results in production.
Scaling Considerations
Hierarchical decomposition introduces significant scaling considerations that must be addressed for production systems:
Fan-out control: Each manager should delegate to 2-7 children (following cognitive load principles). More than 7 children overwhelms the manager's ability to track and coordinate. If a workstream requires more parallelism, add another level of hierarchy rather than widening the fan-out.
Depth vs. breadth: Deeper hierarchies provide finer-grained control but add communication overhead. Each level adds latency (at least one LLM call for decomposition and one for aggregation). In practice, 2-3 levels handle most tasks. If you need more than 4 levels, reconsider whether hierarchical decomposition is the right pattern.
Cost modeling: Total cost = sum of all LLM calls across all nodes. A 3-level hierarchy with fan-out of 4 at each level produces 1 + 4 + 16 = 21 nodes, each making at least 2 LLM calls (receive task + produce result), for a minimum of 42 LLM calls. Plus aggregation calls at each manager. Cost grows exponentially with depth. Use cost-effective models for leaf workers and reserve expensive models for root and manager decisions.
Parallelism: Independent subtasks at the same level can execute in parallel. A well-designed hierarchy maximizes parallelism — siblings don't depend on each other, only on their parent. Use async execution for sibling workers and gather results with appropriate timeouts.
Failure isolation: A failed worker should only affect its parent's subtask, not the entire hierarchy. Managers should handle child failures locally (retry, fallback, graceful degradation) and only escalate to their parent when local recovery is impossible.
When Hierarchy Helps vs. Hurts
Hierarchical decomposition is powerful for certain problem classes but counterproductive for others:
Strong fit: Large-scale content generation (write a 50-page report), comprehensive research (analyze a market with multiple segments), complex data processing (ETL pipeline with validation), and enterprise workflows that mirror organizational structure. In these cases, the hierarchy provides natural structure, parallelism, and manageability.
Poor fit: Tasks that require tight feedback loops between components (the output of one part changes the requirements of another), creative tasks where the whole is not the sum of the parts (writing a novel — you can't decompose it into independently written chapters), and tasks where the decomposition itself is the hard problem (if you don't know how to break it down, adding hierarchy doesn't help).
A useful heuristic: if a human team would solve the problem using a hierarchical work breakdown structure (WBS), then hierarchical agent decomposition is likely a good fit. If humans would solve it through iterative collaboration and creative exploration, consider peer collaboration or agent teams instead.