JSON Prompting: A Complete Guide for Structured LLM Outputs
2026-05-04 8 min read
Modern LLMs like GPT-4 can generate structured JSON output. By providing a schema and clear examples, you can reliably extract structured data from text. This guide covers proven patterns for production-grade JSON prompting.
Core Pattern: Schema + Examples + Instructions
Prompt structure
{
"system": "You are a data extraction assistant. Return ONLY valid JSON.",
"examples": [
{
"input": "Extract entities from: Alice works at Acme Corp in New York.",
"output": {
"person": "Alice",
"company": "Acme Corp",
"location": "New York"
}
}
],
"user_input": "Extract entities from: Bob is a developer at TechCorp in San Francisco.",
"response_schema": {
"type": "object",
"properties": {
"person": {"type": "string"},
"company": {"type": "string"},
"location": {"type": "string"}
},
"required": ["person", "company", "location"]
}
}Best Practices
- Always show examples: LLMs learn from patterns. 2-3 well-chosen examples often make the difference between success and failure.
- Include a schema: Provide the exact JSON structure you expect.
- Validate the response: LLMs can hallucinate. Always parse and validate output against your schema before using it.
- Use function calling when available: OpenAI's function calling API is more reliable than raw JSON prompting.
- Handle failures gracefully: Implement retry logic with backoff for malformed responses.
Safety & Validation
Validating LLM output
import json
from pydantic import BaseModel, ValidationError
class EntityExtraction(BaseModel):
person: str
company: str
location: str
# Get response from LLM
llm_response = '{"person": "Bob", "company": "TechCorp", "location": "San Francisco"}'
try:
parsed = EntityExtraction.model_validate_json(llm_response)
# parsed is now validated and typed
print(f"Extracted: {parsed.person}")
except ValidationError as e:
print(f"Invalid response from LLM: {e}")Design LLM prompts with JSON
Structure your LLM workflows with validated JSON templates and schemas.