Structured Output From LLMs — Reliable JSON Extraction for Production Systems

Introduction

Unstructured LLM outputs are unreliable at scale. This guide covers production-tested techniques for extracting structured JSON reliably, from response_format validation to automatic error correction loops.

OpenAI Response Format With JSON Schema
Anthropic Tool Use for Structured Extraction
Retry Logic for Malformed Outputs
The Instructor Library for TypeScript
Few-Shot Examples for Complex Schemas
Validation and Error Correction Loop
Testing LLM Extraction With Fixtures
Checklist
Conclusion

OpenAI Response Format With JSON Schema

OpenAI's response_format parameter enforces strict JSON output, eliminating parsing failures.

import OpenAI from 'openai';
import { z } from 'zod';

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Define your schema with Zod
const InvoiceSchema = z.object({
  invoiceNumber: z.string(),
  amount: z.number(),
  date: z.string().datetime(),
  items: z.array(
    z.object({
      description: z.string(),
      quantity: z.number(),
      unitPrice: z.number(),
    })
  ),
  vendor: z.object({
    name: z.string(),
    taxId: z.string().optional(),
  }),
});

type Invoice = z.infer<typeof InvoiceSchema>;

// Convert Zod schema to JSON Schema
function zodToJsonSchema(schema: z.ZodSchema): Record<string, unknown> {
  return {
    type: 'object',
    properties: {
      invoiceNumber: { type: 'string' },
      amount: { type: 'number' },
      date: { type: 'string', format: 'date-time' },
      items: {
        type: 'array',
        items: {
          type: 'object',
          properties: {
            description: { type: 'string' },
            quantity: { type: 'number' },
            unitPrice: { type: 'number' },
          },
          required: ['description', 'quantity', 'unitPrice'],
        },
      },
      vendor: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          taxId: { type: 'string' },
        },
        required: ['name'],
      },
    },
    required: ['invoiceNumber', 'amount', 'date', 'items', 'vendor'],
  };
}

async function extractInvoice(invoiceText: string): Promise<Invoice> {
  const response = await client.beta.messages.create({
    model: 'gpt-4-turbo',
    max_tokens: 1024,
    messages: [
      {
        role: 'user',
        content: `Extract invoice data from the following text:\n\n${invoiceText}`,
      },
    ],
    temperature: 0,
    response_format: {
      type: 'json_schema',
      json_schema: {
        name: 'Invoice',
        schema: zodToJsonSchema(InvoiceSchema) as unknown,
        strict: true,
      },
    },
  });

  const textContent = response.content.find((block) => block.type === 'text');
  if (!textContent || textContent.type !== 'text') {
    throw new Error('No text response from API');
  }

  const parsed = JSON.parse(textContent.text);
  return InvoiceSchema.parse(parsed);
}

const invoice = await extractInvoice('Invoice #12345...');

Anthropic Tool Use for Structured Extraction

Anthropic's tool_use feature provides a native alternative to JSON Schema.

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

interface PersonSchema {
  name: string;
  email: string;
  phone?: string;
  age: number;
}

async function extractWithTools(text: string): Promise<PersonSchema> {
  const response = await client.messages.create({
    model: 'claude-opus-4-1-20250805',
    max_tokens: 512,
    messages: [
      {
        role: 'user',
        content: `Extract person information from this text: ${text}`,
      },
    ],
    tools: [
      {
        name: 'extract_person',
        description: 'Extract structured person data from unstructured text',
        input_schema: {
          type: 'object',
          properties: {
            name: { type: 'string', description: 'Full name' },
            email: { type: 'string', format: 'email', description: 'Email address' },
            phone: { type: 'string', description: 'Phone number (optional)' },
            age: { type: 'integer', description: 'Age in years' },
          },
          required: ['name', 'email', 'age'],
        },
      },
    ],
  });

  const toolCall = response.content.find((block) => block.type === 'tool_use');
  if (!toolCall || toolCall.type !== 'tool_use') {
    throw new Error('Expected tool call in response');
  }

  return toolCall.input as PersonSchema;
}

const person = await extractWithTools('John Doe is 28 years old and his email is john@example.com');

Retry Logic for Malformed Outputs

Automatically retry with corrective prompts when parsing fails.

class StructuredExtractor {
  private maxRetries = 3;
  private backoffMs = 500;

  async extractWithRetry<T>(
    text: string,
    schema: z.ZodSchema<T>,
    prompt: string
  ): Promise<T> {
    let lastError: Error | null = null;

    for (let attempt = 0; attempt < this.maxRetries; attempt++) {
      try {
        const response = await fetch('https://api.openai.com/v1/chat/completions', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
          body: JSON.stringify({
            model: 'gpt-4-turbo',
            messages: [{ role: 'user', content: `${prompt}\n\n${text}` }],
            response_format: { type: 'json_object' },
            temperature: 0,
          }),
        });

        const data = (await response.json()) as { choices: Array<{ message: { content: string } }> };
        const jsonText = data.choices[0].message.content;
        const parsed = JSON.parse(jsonText);
        const validated = schema.parse(parsed);
        return validated;
      } catch (error) {
        lastError = error instanceof Error ? error : new Error(String(error));

        if (attempt < this.maxRetries - 1) {
          // Exponential backoff with jitter
          const delay = this.backoffMs * Math.pow(2, attempt) + Math.random() * 100;
          await new Promise((resolve) => setTimeout(resolve, delay));

          // Send corrective feedback
          console.log(`Attempt ${attempt + 1} failed, retrying with corrective prompt...`);
        }
      }
    }

    throw new Error(`Extraction failed after ${this.maxRetries} attempts: ${lastError?.message}`);
  }
}

const extractor = new StructuredExtractor();
const result = await extractor.extractWithRetry(
  'Raw text to extract from',
  z.object({ name: z.string() }),
  'Extract structured data, ensure valid JSON'
);

The Instructor Library for TypeScript

Instructor automatically handles validation and retries using your TypeScript types.

import Anthropic from '@anthropic-ai/sdk';

interface User {
  id: string;
  username: string;
  email: string;
  age: number;
  isActive: boolean;
}

class InstructorClient {
  private client: Anthropic;

  constructor(apiKey: string) {
    this.client = new Anthropic({ apiKey });
  }

  async extract<T extends Record<string, unknown>>(
    text: string,
    schema: T,
    description: string
  ): Promise<T> {
    const properties: Record<string, unknown> = {};
    const required: string[] = [];

    for (const [key, value] of Object.entries(schema)) {
      required.push(key);
      if (typeof value === 'string') {
        properties[key] = { type: 'string', description: value };
      } else if (typeof value === 'number') {
        properties[key] = { type: 'number' };
      } else if (typeof value === 'boolean') {
        properties[key] = { type: 'boolean' };
      }
    }

    const response = await this.client.messages.create({
      model: 'claude-opus-4-1-20250805',
      max_tokens: 1024,
      messages: [
        {
          role: 'user',
          content: `${description}\n\nText:\n${text}`,
        },
      ],
      temperature: 0,
    });

    const content = response.content.find((block) => block.type === 'text');
    if (!content || content.type !== 'text') {
      throw new Error('No text response');
    }

    return JSON.parse(content.text) as T;
  }
}

const instructor = new InstructorClient(process.env.ANTHROPIC_API_KEY!);
const user = await instructor.extract<User>(
  'John Doe, 28 years old, active user with email john@example.com',
  { id: 'string', username: 'string', email: 'string', age: 'number', isActive: 'boolean' },
  'Extract user information'
);

Few-Shot Examples for Complex Schemas

Provide examples of desired outputs to improve extraction quality.

interface FewShotExtractor {
  extract(text: string): Promise<Record<string, unknown>>;
}

function createFewShotExtractor(examples: Array<{ input: string; output: unknown }>): FewShotExtractor {
  return {
    async extract(text: string): Promise<Record<string, unknown>> {
      const exampleStr = examples
        .map((ex) => `Input: "${ex.input}"\nOutput: ${JSON.stringify(ex.output)}`)
        .join('\n\n');

      const prompt = `You are extracting structured data. Here are examples:\n\n${exampleStr}\n\nNow extract from: "${text}"`;

      const response = await fetch('https://api.openai.com/v1/chat/completions', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
        },
        body: JSON.stringify({
          model: 'gpt-4-turbo',
          messages: [{ role: 'user', content: prompt }],
          response_format: { type: 'json_object' },
          temperature: 0,
        }),
      });

      const data = (await response.json()) as { choices: Array<{ message: { content: string } }> };
      return JSON.parse(data.choices[0].message.content);
    },
  };
}

const examples = [
  {
    input: 'Meeting with Sarah on Tuesday at 2pm',
    output: { attendee: 'Sarah', date: '2026-03-18', time: '14:00' },
  },
  {
    input: 'Call John tomorrow 10am',
    output: { attendee: 'John', date: '2026-03-18', time: '10:00' },
  },
];

const extractor = createFewShotExtractor(examples);
const result = await extractor.extract('Sync with team Thursday 9am');

Validation and Error Correction Loop

Validate outputs and automatically correct invalid data.

class ValidatingExtractor<T extends z.ZodSchema> {
  constructor(private schema: T) {}

  async extractWithValidation(text: string, maxAttempts: number = 3): Promise<z.infer<T>> {
    for (let attempt = 0; attempt < maxAttempts; attempt++) {
      try {
        const response = await this.callLLM(text);
        const parsed = JSON.parse(response);
        return this.schema.parse(parsed);
      } catch (error) {
        if (attempt === maxAttempts - 1) throw error;

        // Generate corrective prompt from validation error
        const errorMsg = error instanceof z.ZodError ? error.issues[0]?.message : String(error);
        text = `${text}\n\nPrevious attempt failed: ${errorMsg}. Please fix and retry.`;
      }
    }

    throw new Error('Validation failed after max attempts');
  }

  private async callLLM(text: string): Promise<string> {
    const response = await fetch('https://api.openai.com/v1/chat/completions', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      },
      body: JSON.stringify({
        model: 'gpt-4-turbo',
        messages: [{ role: 'user', content: text }],
        response_format: { type: 'json_object' },
        temperature: 0,
      }),
    });

    const data = (await response.json()) as { choices: Array<{ message: { content: string } }> };
    return data.choices[0].message.content;
  }
}

const productSchema = z.object({
  sku: z.string().min(3),
  name: z.string(),
  price: z.number().positive(),
  inStock: z.boolean(),
});

const extractor = new ValidatingExtractor(productSchema);
const product = await extractor.extractWithValidation('Product ABC123 costs $29.99 and is in stock');

Testing LLM Extraction With Fixtures

Create comprehensive test suites using fixture data.

interface ExtractionTestCase {
  name: string;
  input: string;
  expected: unknown;
}

class ExtractionTestSuite {
  private cases: ExtractionTestCase[] = [];
  private passCount = 0;
  private failCount = 0;

  addCase(testCase: ExtractionTestCase): void {
    this.cases.push(testCase);
  }

  async run(extractor: (text: string) => Promise<unknown>): Promise<void> {
    console.log(`Running ${this.cases.length} extraction tests...\n`);

    for (const testCase of this.cases) {
      try {
        const result = await extractor(testCase.input);
        const matches = JSON.stringify(result) === JSON.stringify(testCase.expected);

        if (matches) {
          this.passCount++;
          console.log(`✓ ${testCase.name}`);
        } else {
          this.failCount++;
          console.log(`✗ ${testCase.name}`);
          console.log(`  Expected: ${JSON.stringify(testCase.expected)}`);
          console.log(`  Got: ${JSON.stringify(result)}`);
        }
      } catch (error) {
        this.failCount++;
        console.log(`✗ ${testCase.name}: ${error instanceof Error ? error.message : String(error)}`);
      }
    }

    console.log(`\nResults: ${this.passCount} passed, ${this.failCount} failed`);
  }
}

const suite = new ExtractionTestSuite();
suite.addCase({
  name: 'Extract valid email',
  input: 'Contact us at support@example.com',
  expected: { email: 'support@example.com' },
});
suite.addCase({
  name: 'Extract invoice data',
  input: 'Invoice #INV-2026-001 for $150.00',
  expected: { invoiceId: 'INV-2026-001', amount: 150.0 },
});

Checklist

Use OpenAI's response_format: json_schema for guaranteed JSON structure
Implement Anthropic tool_use as a native alternative
Add automatic retry logic with exponential backoff
Validate outputs immediately with Zod or similar validators
Provide few-shot examples for complex extraction tasks
Build test suites with fixture data for regression testing
Log all extraction failures for analysis and prompt improvement
Monitor extraction success rates as a quality metric

Conclusion

Structured output extraction is critical for AI-powered automation. Combine response_format validation, tool use, automatic retries, and comprehensive testing to ensure bulletproof extraction at scale. Start with JSON Schema validation, add few-shot examples when accuracy drops below 95%, and maintain a fixture library for continuous regression testing.