Planning & Reasoning in Agents

Planning and reasoning are what separate an agent that stumbles through a task from one that solves it deliberately. This tutorial covers the techniques — decomposition, hierarchical planning, chain-of-thought, tree-of-thought — and, as always, is honest about their cost and when they earn it.

Task decomposition#

A model handles several focused steps more reliably than one sprawling request. Decomposition also gives you an inspectable plan:

record SubTask(String description, List<String> dependsOn) {}
 
public Result solve(String goal) {
    // The model breaks the goal into sub-tasks with dependencies.
    List<SubTask> plan = planner.decompose(goal);   // structured output
 
    // Execute in dependency order, feeding results forward.
    Map<String, String> results = new LinkedHashMap<>();
    for (SubTask task : topologicalOrder(plan)) {
        results.put(task.description(), executor.execute(task, results));
    }
    return synthesise(goal, results);
}

Chain-of-thought#

Prompting the model to reason through steps before answering improves accuracy on multi-step problems:

String prompt = """
        Work through this step by step, showing your reasoning, then give the
        final answer on a line starting with "ANSWER:".
 
        Problem: %s
        """.formatted(problem);
 
String response = model.chat(prompt);
String answer = extractAfter(response, "ANSWER:");

The reasoning is not for the user — you extract just the answer. The intermediate steps are what improve the result.

Hierarchical planning#

For large tasks, plan at a high level, then plan each high-level step in detail. This mirrors how people tackle big projects:

It keeps each planning call focused — the high-level planner reasons about the shape, the detailed planners about mechanics — and it lets you re-plan a single step without redoing the whole plan.

Tree-of-thought#

When a single line of reasoning often fails, explore several branches and evaluate them:

public String solveHard(String problem) {
    // Generate several candidate approaches.
    List<String> branches = generator.propose(problem, 3);
 
    // Evaluate each, pursue the most promising, prune the rest.
    return branches.stream()
            .map(branch -> new Scored(branch, evaluator.score(branch, problem)))
            .max(Comparator.comparingDouble(Scored::score))
            .map(best -> executor.pursue(best.branch(), problem))
            .orElseThrow();
}

Reflection as reasoning#

Reflection — critiquing and revising output — is a form of reasoning that catches errors after the fact:

String draft = solver.solve(problem);
Critique critique = critic.evaluate(draft, problem);
if (!critique.isSound()) {
    draft = solver.revise(draft, critique);
}

It pairs well with planning: plan, execute, then reflect on whether the result actually met the goal. See agent architecture patterns.

Choosing a technique#

The through-line: match the reasoning cost to the task difficulty. Most tasks need far less than the elaborate techniques suggest, and over-applying them wastes money and adds latency for no gain.

Next#

• Memory systems for agents
• Agentic RAG advanced patterns
• Building autonomous coding agents