Published on

AI Agent Architecture Patterns — ReAct, Plan-Execute, and Reflection Loops

Authors

Introduction

AI agents have evolved beyond simple LLM calls into sophisticated reasoning systems. Understanding the fundamental architecture patterns is critical for building production systems that are debuggable, reliable, and performant. This post covers ReAct, Plan-Execute-Observe, reflection loops, and the infrastructure needed to keep agents from hallucinating forever.

ReAct: Reasoning + Acting

ReAct (Reasoning + Acting) is the most widely implemented agent pattern. The loop alternates between the agent reasoning about what to do next and then executing a tool or action.

interface AgentState {
  messages: Array<{ role: string; content: string }>;
  toolCalls: ToolCall[];
  iterations: number;
  maxIterations: number;
}

interface ToolCall {
  id: string;
  name: string;
  input: Record<string, unknown>;
  result?: string;
  error?: string;
}

class ReactAgent {
  private maxIterations: number = 10;

  async run(userQuery: string): Promise<string> {
    const state: AgentState = {
      messages: [{ role: 'user', content: userQuery }],
      toolCalls: [],
      iterations: 0,
      maxIterations: this.maxIterations,
    };

    const systemPrompt = `You are a helpful AI assistant with access to tools.
When you need to take an action, respond with tool calls in JSON format.
Think through your reasoning step by step before deciding which tool to use.`;

    while (state.iterations < state.maxIterations) {
      state.iterations++;

      // Get LLM response with reasoning
      const response = await this.llmCall(systemPrompt, state.messages);

      // Extract tool calls from response
      const toolCalls = this.extractToolCalls(response);

      if (toolCalls.length === 0) {
        // No more tools needed, return response
        return response;
      }

      // Execute tools
      for (const toolCall of toolCalls) {
        try {
          const result = await this.executeTool(toolCall.name, toolCall.input);
          toolCall.result = result;
          state.messages.push({
            role: 'assistant',
            content: `Called ${toolCall.name}: ${JSON.stringify(toolCall.input)}`,
          });
          state.messages.push({
            role: 'user',
            content: `Tool result: ${result}`,
          });
        } catch (error) {
          toolCall.error = (error as Error).message;
          state.messages.push({
            role: 'user',
            content: `Tool error: ${(error as Error).message}. Try a different approach.`,
          });
        }
      }

      state.toolCalls.push(...toolCalls);
    }

    throw new Error(`Agent exceeded max iterations (${this.maxIterations})`);
  }

  private async llmCall(system: string, messages: any[]): Promise<string> {
    const response = await fetch('https://api.anthropic.com/v1/messages', {
      method: 'POST',
      headers: {
        'x-api-key': process.env.ANTHROPIC_API_KEY!,
        'content-type': 'application/json',
      },
      body: JSON.stringify({
        model: 'claude-opus-4-1',
        max_tokens: 2048,
        system,
        messages,
      }),
    });

    const data = (await response.json()) as any;
    return data.content[0].text;
  }

  private extractToolCalls(response: string): ToolCall[] {
    const pattern = /\{.*?"name":\s*"([^"]+)".*?\}/g;
    const tools: ToolCall[] = [];

    let match;
    while ((match = pattern.exec(response)) !== null) {
      try {
        const toolJson = JSON.parse(match[0]);
        tools.push({
          id: `tool-${Date.now()}-${Math.random()}`,
          name: toolJson.name,
          input: toolJson.input || {},
        });
      } catch {
        // Skip malformed tool calls
      }
    }

    return tools;
  }

  private async executeTool(name: string, input: Record<string, unknown>): Promise<string> {
    // Tool execution implementation
    return `Result from ${name}`;
  }
}

ReAct's strength is its simplicity: think, act, observe the result, repeat. The weakness is that agents can get stuck in loops taking the same action repeatedly without making progress.

Plan-Execute-Observe Pattern

Plan-Execute-Observe explicitly separates planning from execution, making the agent more structured and debuggable.

interface Plan {
  steps: PlanStep[];
  reasoning: string;
}

interface PlanStep {
  id: string;
  description: string;
  tool: string;
  expectedInput: Record<string, unknown>;
  status: 'pending' | 'executing' | 'completed' | 'failed';
  result?: string;
  error?: string;
}

class PlanExecuteAgent {
  async run(userQuery: string): Promise<string> {
    // Step 1: Planning
    const plan = await this.generatePlan(userQuery);
    console.log(`Generated plan with ${plan.steps.length} steps`);

    // Step 2: Execution
    for (const step of plan.steps) {
      if (step.status === 'pending') {
        step.status = 'executing';

        try {
          const result = await this.executeTool(step.tool, step.expectedInput);
          step.result = result;
          step.status = 'completed';
        } catch (error) {
          step.error = (error as Error).message;
          step.status = 'failed';

          // Decide whether to continue or bail
          const shouldContinue = await this.shouldContinueAfterFailure(
            plan,
            step,
            error as Error,
          );

          if (!shouldContinue) {
            throw new Error(`Plan execution failed at step: ${step.description}`);
          }
        }
      }
    }

    // Step 3: Observation & Synthesis
    const finalResponse = await this.synthesizeResults(plan, userQuery);
    return finalResponse;
  }

  private async generatePlan(query: string): Promise<Plan> {
    const prompt = `You are a planning agent. Break down this task into steps.
Return a JSON plan with an array of steps, each with id, description, tool name, and input.

Task: ${query}`;

    const response = await this.llmCall(prompt);
    try {
      return JSON.parse(response);
    } catch {
      throw new Error('Failed to parse plan from LLM response');
    }
  }

  private async shouldContinueAfterFailure(
    plan: Plan,
    failedStep: PlanStep,
    error: Error,
  ): Promise<boolean> {
    // Check if we can skip this step or retry with different params
    const hasAlternatives = plan.steps.some(
      (s) =>
        s.id !== failedStep.id && s.status === 'pending' && s.tool !== failedStep.tool,
    );

    return hasAlternatives;
  }

  private async synthesizeResults(plan: Plan, originalQuery: string): Promise<string> {
    const results = plan.steps
      .filter((s) => s.status === 'completed')
      .map((s) => `${s.description}: ${s.result}`)
      .join('\n');

    const prompt = `Original request: ${originalQuery}\n\nExecution results:\n${results}\n\nProvide a final answer.`;

    return this.llmCall(prompt);
  }

  private async llmCall(prompt: string): Promise<string> {
    // LLM call implementation
    return '';
  }

  private async executeTool(name: string, input: Record<string, unknown>): Promise<string> {
    // Tool execution
    return '';
  }
}

This pattern makes debugging much easier: you can see exactly what the plan was and where it succeeded or failed. It also makes it easier to add checkpoints and human review steps.

Reflection and Critique Loops

Reflection allows agents to evaluate their own output and iteratively improve it. This is particularly useful for code generation, writing, and analysis tasks.

interface ReflectionResult {
  original: string;
  critique: string;
  revision: string;
  isAcceptable: boolean;
  iterations: number;
}

class ReflectingAgent {
  private maxReflections: number = 3;

  async generateWithReflection(prompt: string): Promise<ReflectionResult> {
    let current = await this.generate(prompt);
    let iterations = 0;

    while (iterations < this.maxReflections) {
      iterations++;

      // Critique the current output
      const critique = await this.critique(prompt, current);

      // Check if output is acceptable
      if (this.isAcceptable(critique)) {
        return {
          original: current,
          critique,
          revision: current,
          isAcceptable: true,
          iterations,
        };
      }

      // Ask for revision
      const revised = await this.revise(prompt, current, critique);

      if (revised === current) {
        // No improvement, bail out
        break;
      }

      current = revised;
    }

    return {
      original: current,
      critique: 'Max reflections reached',
      revision: current,
      isAcceptable: false,
      iterations,
    };
  }

  private async generate(prompt: string): Promise<string> {
    return this.llmCall(
      `${prompt}\n\nProvide a high-quality, detailed response.`,
    );
  }

  private async critique(originalPrompt: string, output: string): Promise<string> {
    const critiquePrompt = `Original request: ${originalPrompt}

Provided response:
${output}

Critique this response. Point out:
- Completeness: does it fully address the request?
- Accuracy: is the information correct?
- Clarity: is it easy to understand?
- Structure: is it well-organized?

Be specific about what could be improved.`;

    return this.llmCall(critiquePrompt);
  }

  private isAcceptable(critique: string): boolean {
    // Simple heuristic: if critique mentions "excellent", "complete", or "no issues"
    const acceptable = /(excellent|complete|no issues|well-structured|comprehensive)/i.test(
      critique,
    );
    return acceptable;
  }

  private async revise(
    originalPrompt: string,
    currentOutput: string,
    critique: string,
  ): Promise<string> {
    const revisionPrompt = `Original request: ${originalPrompt}

Current response:
${currentOutput}

Feedback on current response:
${critique}

Please revise the response to address the feedback above. Make it better.`;

    return this.llmCall(revisionPrompt);
  }

  private async llmCall(prompt: string): Promise<string> {
    // LLM implementation
    return '';
  }
}

Reflection loops are powerful but expensive, so use them selectively. They work best for tasks where quality matters more than speed.

Agent State Management

Managing agent state is critical for debugging, resuming interrupted tasks, and monitoring.

interface AgentCheckpoint {
  id: string;
  timestamp: number;
  state: AgentState;
  metadata: {
    totalTokensUsed: number;
    totalToolCalls: number;
    currentIteration: number;
  };
}

class StatefulAgent {
  private checkpointStore: Map<string, AgentCheckpoint> = new Map();

  async runWithCheckpointing(
    sessionId: string,
    userQuery: string,
  ): Promise<{ result: string; checkpoints: AgentCheckpoint[] }> {
    let state: AgentState = this.getOrCreateState(sessionId);
    const checkpoints: AgentCheckpoint[] = [];

    try {
      while (state.iterations < state.maxIterations) {
        state.iterations++;

        // Create checkpoint before each iteration
        const checkpoint = this.createCheckpoint(sessionId, state);
        checkpoints.push(checkpoint);
        this.checkpointStore.set(checkpoint.id, checkpoint);

        // Execute iteration
        const toolCalls = await this.executeIteration(state, userQuery);

        if (toolCalls.length === 0) {
          return {
            result: state.messages[state.messages.length - 1].content,
            checkpoints,
          };
        }

        state.toolCalls.push(...toolCalls);
      }
    } catch (error) {
      // On error, save final checkpoint for recovery
      const errorCheckpoint = this.createCheckpoint(sessionId, state);
      checkpoints.push(errorCheckpoint);
      throw error;
    }

    throw new Error(`Agent max iterations exceeded`);
  }

  private createCheckpoint(sessionId: string, state: AgentState): AgentCheckpoint {
    return {
      id: `${sessionId}-checkpoint-${state.iterations}`,
      timestamp: Date.now(),
      state: JSON.parse(JSON.stringify(state)), // Deep copy
      metadata: {
        totalTokensUsed: this.estimateTokens(state.messages),
        totalToolCalls: state.toolCalls.length,
        currentIteration: state.iterations,
      },
    };
  }

  private getOrCreateState(sessionId: string): AgentState {
    // Load from storage if exists, else create new
    return {
      messages: [],
      toolCalls: [],
      iterations: 0,
      maxIterations: 10,
    };
  }

  private async executeIteration(state: AgentState, query: string): Promise<ToolCall[]> {
    // Iteration logic
    return [];
  }

  private estimateTokens(messages: any[]): number {
    return messages.reduce((sum, msg) => sum + Math.ceil(msg.content.length / 4), 0);
  }
}

Checkpointing enables recovery, debugging, and cost tracking. Always save state before expensive operations.

Preventing Infinite Loops

The most common agent failure is getting stuck in a loop. Multi-layered prevention is essential.

interface LoopDetectionMetrics {
  recentToolCalls: ToolCall[];
  toolCallCounts: Map<string, number>;
  messagePatterns: string[];
  hasLoop: boolean;
  reason?: string;
}

class LoopDetector {
  private windowSize: number = 5;

  detectLoop(state: AgentState): LoopDetectionMetrics {
    const metrics: LoopDetectionMetrics = {
      recentToolCalls: state.toolCalls.slice(-this.windowSize),
      toolCallCounts: new Map(),
      messagePatterns: [],
      hasLoop: false,
    };

    // Check 1: Same tool called repeatedly
    const recentTools = metrics.recentToolCalls.map((tc) => tc.name);
    for (const tool of recentTools) {
      metrics.toolCallCounts.set(tool, (metrics.toolCallCounts.get(tool) || 0) + 1);
    }

    const toolCallMax = Math.max(...metrics.toolCallCounts.values());
    if (toolCallMax >= 3 && recentTools.length >= 3) {
      metrics.hasLoop = true;
      metrics.reason = `Tool '${recentTools[0]}' called ${toolCallMax} times in last ${this.windowSize} iterations`;
      return metrics;
    }

    // Check 2: Identical inputs to same tool
    const recentByTool = new Map<string, any[]>();
    for (const tc of metrics.recentToolCalls) {
      if (!recentByTool.has(tc.name)) {
        recentByTool.set(tc.name, []);
      }
      recentByTool.get(tc.name)!.push(tc.input);
    }

    for (const [tool, inputs] of recentByTool.entries()) {
      const uniqueInputs = new Set(inputs.map((i) => JSON.stringify(i)));
      if (uniqueInputs.size === 1 && inputs.length >= 2) {
        metrics.hasLoop = true;
        metrics.reason = `Tool '${tool}' called with identical inputs`;
        return metrics;
      }
    }

    // Check 3: Message history shows no progress
    const lastMessages = state.messages.slice(-10);
    const patterns = lastMessages.map((m) => m.content.substring(0, 50));
    const uniquePatterns = new Set(patterns);
    if (uniquePatterns.size < 3 && lastMessages.length >= 10) {
      metrics.hasLoop = true;
      metrics.reason = `Low message diversity suggests no progress`;
      return metrics;
    }

    return metrics;
  }
}

class SafeAgent {
  private loopDetector = new LoopDetector();

  async runSafely(userQuery: string): Promise<string> {
    const state: AgentState = {
      messages: [{ role: 'user', content: userQuery }],
      toolCalls: [],
      iterations: 0,
      maxIterations: 10,
    };

    while (state.iterations < state.maxIterations) {
      // Check for loops
      const loopMetrics = this.loopDetector.detectLoop(state);
      if (loopMetrics.hasLoop) {
        console.log(`Loop detected: ${loopMetrics.reason}`);
        throw new Error(`Agent stuck in loop: ${loopMetrics.reason}`);
      }

      // Continue with normal iteration
      state.iterations++;

      // ... rest of iteration logic
    }

    throw new Error(`Max iterations exceeded`);
  }
}

Effective loop prevention requires multiple signals: iteration limits, tool call frequency analysis, input uniqueness checks, and message diversity monitoring.

Agent Tracing for Debugging

Production agents need comprehensive tracing to diagnose failures.

interface Trace {
  traceId: string;
  agentName: string;
  startTime: number;
  endTime?: number;
  spans: TraceSpan[];
  finalResult?: string;
  error?: string;
}

interface TraceSpan {
  spanId: string;
  name: string;
  startTime: number;
  endTime: number;
  status: 'success' | 'error' | 'timeout';
  input?: unknown;
  output?: unknown;
  metadata: Record<string, unknown>;
}

class TracedAgent {
  private tracer = new AgentTracer();

  async runWithTracing(userQuery: string): Promise<string> {
    const trace = this.tracer.startTrace('react-agent');

    try {
      const thinkSpan = this.tracer.startSpan(trace, 'think');
      const response = await this.llmCall(userQuery);
      this.tracer.endSpan(thinkSpan, { output: response });

      const toolSpan = this.tracer.startSpan(trace, 'execute-tool');
      const toolResult = await this.executeTool('search', { query: userQuery });
      this.tracer.endSpan(toolSpan, { output: toolResult });

      this.tracer.endTrace(trace, { result: toolResult });
      return toolResult;
    } catch (error) {
      this.tracer.endTrace(trace, { error: (error as Error).message });
      throw error;
    }
  }
}

class AgentTracer {
  private traces: Map<string, Trace> = new Map();

  startTrace(agentName: string): Trace {
    const trace: Trace = {
      traceId: `trace-${Date.now()}-${Math.random()}`,
      agentName,
      startTime: Date.now(),
      spans: [],
    };

    this.traces.set(trace.traceId, trace);
    return trace;
  }

  startSpan(trace: Trace, name: string): TraceSpan {
    const span: TraceSpan = {
      spanId: `span-${Math.random()}`,
      name,
      startTime: Date.now(),
      endTime: 0,
      status: 'success',
      metadata: {},
    };

    return span;
  }

  endSpan(span: TraceSpan, data: { output?: unknown; error?: string }): void {
    span.endTime = Date.now();
    span.output = data.output;
    if (data.error) {
      span.status = 'error';
    }
  }

  endTrace(trace: Trace, result: { result?: string; error?: string }): void {
    trace.endTime = Date.now();
    trace.finalResult = result.result;
    trace.error = result.error;
  }

  exportTrace(traceId: string): Trace | undefined {
    return this.traces.get(traceId);
  }
}

Detailed tracing is invaluable for understanding why agents make decisions and where failures occur.

When Agents Beat Single Prompts

Understanding when to use agents vs simple prompts is crucial for cost and latency optimization.

Use agents when:

  • The task requires multiple steps or tools
  • You need to handle uncertainty (agent can retry/pivot)
  • The problem requires reasoning over time
  • You need audit trails of decision-making

Use single prompts when:

  • The task is straightforward (classification, formatting, summarization)
  • Latency is critical (agents add multiple LLM calls)
  • Cost is the primary constraint (each iteration costs tokens)
  • The task doesn't require external information
class AdaptiveAgent {
  async solve(problem: string): Promise<string> {
    // Quickly evaluate if a simple prompt suffices
    const complexity = await this.evaluateComplexity(problem);

    if (complexity < 0.3) {
      // Simple prompt is enough
      return this.simplePrompt(problem);
    }

    if (complexity < 0.7) {
      // Use single-turn agent with tools
      return this.singleTurnAgent(problem);
    }

    // Multi-turn agent for complex problems
    return this.multiTurnAgent(problem);
  }

  private async evaluateComplexity(problem: string): Promise<number> {
    // Heuristics: number of questions, imperative phrases, length, etc.
    const questionCount = (problem.match(/\?/g) || []).length;
    const imperatives = (problem.match(/\b(find|retrieve|calculate|analyze|compare)\b/gi) || [])
      .length;
    const length = problem.length;

    return Math.min(1, (questionCount + imperatives) * 0.1 + length / 1000);
  }

  private async simplePrompt(problem: string): Promise<string> {
    return 'Simple response';
  }

  private async singleTurnAgent(problem: string): Promise<string> {
    return 'Single-turn response';
  }

  private async multiTurnAgent(problem: string): Promise<string> {
    return 'Multi-turn response';
  }
}

Checklist

  • Use iteration limits and loop detection in all agents
  • Implement checkpoint/resume for long-running agents
  • Add comprehensive tracing for debugging
  • Monitor token usage and cost per agent run
  • Test agents with adversarial queries that could cause loops
  • Version your agent prompts alongside code changes
  • Implement reflection loops selectively for high-quality outputs

Conclusion

AI agent architecture has standardized around ReAct, Plan-Execute-Observe, and reflection patterns. The key to production success is preventing infinite loops, managing state carefully, and maintaining comprehensive traces. Choose the right pattern for your problem, implement safeguards religiously, and always be able to explain why your agent took each action.