Structured Output Prompts: How to Get Reliable JSON and Structured Data from LLMs
If you've ever tried to get an LLM to output data that another piece of software can actually consume, you know the pain. You ask for JSON, and you get JSON wrapped in a markdown code block. You ask for a list of objects, and you get prose that kinda looks like a list. You ask for a specific schema, and the model adds fields you didn't want, omits fields you required, or wraps everything in a helpful sentence that breaks your parser.
Structured output prompting is the discipline of getting language models to produce machine-readable, predictable, schema-conformant output — consistently, every time. It's the bridge between "AI generated some text" and "AI generated data my application can use." And as LLMs get embedded into real software pipelines, it's becoming one of the most valuable prompt engineering skills you can have.
In this guide, we'll cover everything from the basics of asking for JSON to advanced schema-driven prompting, function calling, and the patterns that production systems use to get reliable structured output from models like GPT-4, Claude, and Gemini.
Why Structured Output Matters
When you're using an LLM as a chatbot, free-form text is fine. But the moment you want the model to:
- Extract data from documents (names, dates, amounts, entities)
- Classify text into categories
- Generate configuration (settings, rules, parameters)
- Power a pipeline where its output feeds into another system
- Build agentic workflows where the model's output triggers actions
...you need structure. You need the model to output data in a format your code can parse, validate, and act on. Unstructured text — no matter how good the content — is useless if your application can't reliably extract what it needs.
Structured output is what makes LLMs useful as components in software systems, not just chat interfaces. And the quality of your prompt is the single biggest factor in whether you get clean, parsable, schema-conformant output or a mess you have to post-process with regex and prayer.
The Three Approaches to Structured Output
There are three main ways to get structured output from an LLM, and they're not mutually exclusive — most production systems combine them:
- Prompt-based structuring: You use prompt engineering to instruct the model to output a specific format. Works with any model, any API. Most flexible, least guaranteed.
- Native structured output / JSON mode: Many model APIs now offer a "JSON mode" or structured output feature that enforces valid JSON at the decoding level. More reliable than prompts alone.
- Function calling / tool use: You define a function schema and the model outputs arguments that conform to it. The most reliable for complex schemas, supported by GPT-4, Claude, Gemini, and others.
Let's dive into each.
Approach 1: Prompt-Based Structuring
This is the foundation. Even if you use JSON mode or function calling, you'll still need prompt engineering to get good results. Here's how to do it right.
The Basic JSON Request
The simplest version:
Extract the following information from the text below and return it as a JSON object with these fields:
- name (string): the person's full name
- email (string): their email address
- company (string): the company they work for
- role (string): their job title
Text: "Hi, I'm Sarah Chen, Senior Product Manager at Techflow Inc. You can reach me at [email protected]."
Return ONLY the JSON object, no other text.
This will work most of the time with modern models. But "most of the time" isn't good enough for production. Here's how to make it reliable.
Rule 1: Show, Don't Just Tell — Use Examples
Models follow examples better than instructions. Always include at least one example of the exact output you want:
Extract contact information and return as JSON.
Example:
Input: "I'm John Smith, CTO at Acme Corp, [email protected]"
Output:
{"name": "John Smith", "email": "[email protected]", "company": "Acme Corp", "role": "CTO"}
Now process this:
Input: "Hi, I'm Sarah Chen, Senior Product Manager at Techflow Inc. You can reach me at [email protected]."
Output:
Notice: end the prompt right after Output: so the model continues directly with the JSON. This is a simple but powerful trick — you're priming it to complete the pattern.
Rule 2: Provide a Schema Explicitly
For anything beyond trivial output, give the model an explicit schema. You can describe it in natural language or use a JSON-like pseudoschema:
Return a JSON object matching this schema:
{
"contacts": [
{
"name": "string — full name",
"email": "string — email address, or null if not found",
"phone": "string — phone number, or null if not found",
"company": "string — company name, or null if not found",
"role": "string — job title, or null if not found",
"confidence": "number — 0.0 to 1.0, your confidence this contact info is correct"
}
],
"source": "string — the document or text this was extracted from",
"total_contacts": "number — count of contacts found"
}
Rules:
- Always include all fields, even if the value is null.
- Do not add any fields not in the schema.
- Output ONLY valid JSON, no markdown, no explanation.
Rule 3: Forbid Surrounding Text
The #1 parsing failure is the model wrapping JSON in markdown or adding commentary:
{"name": "Sarah Chen", ...}
or
Here's the extracted contact info:
{"name": "Sarah Chen", ...}
Hope this helps!
Both break your parser. Prevent this with explicit instructions:
CRITICAL FORMATTING RULES:
- Output ONLY the JSON object.
- Do NOT wrap it in markdown code blocks (no ```json ... ```).
- Do NOT add any text before or after the JSON.
- Do NOT add explanatory comments inside the JSON.
- The very first character of your output must be { and the very last must be }.
Rule 4: Handle Edge Cases in the Prompt
What should the model do when a field is missing? When the input is ambiguous? When there are multiple entities? Specify it:
Edge case handling:
- If a field cannot be found in the input, use null (not "N/A", not "unknown", not "").
- If multiple contacts are found, include all of them in the contacts array.
- If no contacts are found, return {"contacts": [], "source": "...", "total_contacts": 0}.
- If a field is ambiguous (e.g., two possible emails), pick the most likely one and set confidence below 0.7.
Rule 5: Use XML Tags to Separate Instructions from Output
When your prompt gets long, the model can lose track of where instructions end and output should begin. XML-style tags help:
<instructions>
Extract contact info as JSON.
[all your rules here]
</instructions>
<input>
Sarah Chen, Senior PM at Techflow. email: [email protected]
</input>
<output_schema>
{contacts: [{name, email, company, role, confidence}], source, total_contacts}
</output_schema>
Now output the JSON:
Approach 2: Native JSON Mode and Structured Output APIs
Most major model APIs now offer native structured output features that enforce valid JSON at the decoding level. This is dramatically more reliable than prompt-only approaches.
OpenAI JSON Mode and Structured Outputs
OpenAI offers two levels:
JSON Mode (response_format: { type: "json_object" }): Forces the model to output valid JSON. But it doesn't enforce a specific schema — the model could output any valid JSON object.
Structured Outputs (response_format: { type: "json_schema", json_schema: {...} }): You provide a full JSON schema, and the API enforces it. The model's output is guaranteed to conform to the schema. This is the gold standard for structured output.
Example with structured outputs:
response = client.chat.completions.create(
model="gpt-4o",
response_format={
"type": "json_schema",
"json_schema": {
"name": "contact_extraction",
"schema": {
"type": "object",
"properties": {
"contacts": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": ["string", "null"]},
"company": {"type": ["string", "null"]},
"role": {"type": ["string", "null"]}
},
"required": ["name", "email", "company", "role"]
}
},
"total_contacts": {"type": "integer"}
},
"required": ["contacts", "total_contacts"]
}
}
},
messages=[...]
)
With structured outputs, you don't need to spend prompt tokens telling the model not to wrap things in markdown — the API handles it. But you still need good prompts to get the right content into the right fields.
Anthropic Claude Tool Use for Structured Output
Claude doesn't have a "JSON mode" per se, but its tool use feature achieves the same thing. You define a tool with an input schema, and Claude outputs arguments that conform to it:
response = client.messages.create(
model="claude-sonnet-4-20250514",
tools=[{
"name": "extract_contacts",
"description": "Extract contact information from text",
"input_schema": {
"type": "object",
"properties": {
"contacts": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": ["string", "null"]}
},
"required": ["name", "email"]
}
}
},
"required": ["contacts"]
}
}],
messages=[...]
)
Claude will return a tool_use block with structured arguments that match your schema. This is extremely reliable for structured extraction.
Google Gemini Structured Output
Gemini supports responseMimeType: "application/json" plus responseSchema for schema enforcement:
response = model.generate_content(
prompt,
generation_config={
"response_mime_type": "application/json",
"response_schema": {
"type": "object",
"properties": {
"contacts": {"type": "array", "items": {...}}
}
}
}
)
When to Use Native Structured Output vs Prompt-Only
| Situation | Recommended Approach | |-----------|---------------------| | Production pipeline, strict schema required | Native structured output (schema enforced) | | Need guaranteed valid JSON, flexible schema | JSON mode | | Complex nested schema, multiple objects | Function calling / structured outputs | | Quick prototype, any model | Prompt-only with examples | | Model doesn't support JSON mode | Prompt-only with strong formatting rules | | Need to extract into existing code types | Function calling with typed schemas |
Always use native structured output when it's available. It eliminates an entire class of parsing failures. But always pair it with a good prompt — the API ensures the format is correct, but your prompt ensures the content is correct.
Approach 3: Function Calling / Tool Use
Function calling is the most structured approach because the model is explicitly outputting arguments for a predefined function. It was originally designed for letting models invoke external tools, but it's excellent for any structured extraction task.
Why Function Calling Is Great for Structured Output
- Schema enforcement: The model's output must conform to the function's parameter schema.
- Type safety: Parameters have explicit types (string, number, boolean, array, object).
- Required vs optional fields: You can mark which fields are mandatory.
- Enum constraints: You can limit values to specific options (e.g.,
"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]}). - Native parsing: The API returns the structured arguments as a parsed object, not a string you have to parse yourself.
A Practical Function Calling Example: Sentiment Analysis
Define a function:
{
"name": "analyze_sentiment",
"description": "Analyze the sentiment of a text and return structured results",
"parameters": {
"type": "object",
"properties": {
"overall_sentiment": {
"type": "string",
"enum": ["very_positive", "positive", "neutral", "negative", "very_negative"]
},
"confidence_score": {
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"key_phrases": {
"type": "array",
"items": {"type": "string"},
"description": "Phrases that most influenced the sentiment"
},
"aspects": {
"type": "array",
"items": {
"type": "object",
"properties": {
"aspect": {"type": "string", "description": "e.g., 'delivery', 'quality', 'price'"},
"sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
"text": {"type": "string", "description": "the text snippet about this aspect"}
},
"required": ["aspect", "sentiment", "text"]
}
}
},
"required": ["overall_sentiment", "confidence_score", "key_phrases", "aspects"]
}
}
The model will return structured, typed, schema-conformant data. No markdown wrappers, no missing fields, no invalid enum values.
Common Structured Output Patterns
Pattern 1: Entity Extraction
Extracting entities (people, companies, dates, amounts) from unstructured text:
Extract all entities from the text and return as JSON:
Schema:
{
"people": [{"name": "string", "role": "string|null"}],
"organizations": [{"name": "string", "type": "string|null"}],
"dates": [{"date": "string (ISO format)", "context": "string"}],
"monetary_amounts": [{"amount": "number", "currency": "string", "context": "string"}],
"locations": [{"name": "string", "type": "city|country|region|address|null"}]
}
Rules:
- Use ISO 8601 format for dates (YYYY-MM-DD).
- Use null for fields that cannot be determined.
- Do not duplicate entities. If "John Smith" appears twice, list once.
Pattern 2: Classification with Confidence
Classify the following text and return as JSON:
{
"category": "string — one of: billing, technical, complaint, sales, general",
"subcategory": "string — more specific category, or null",
"urgency": "string — one of: low, medium, high, critical",
"confidence": "number — 0.0 to 1.0",
"reasoning": "string — one sentence explaining your classification",
"suggested_action": "string — recommended next step, or null"
}
Rules:
- You MUST pick a category from the listed options. Do not invent new categories.
- Confidence should reflect how certain you are. Below 0.6 means uncertain.
Pattern 3: Data Transformation
Converting unstructured input into a structured record:
Convert the following natural language description into a structured event record:
Schema:
{
"event_name": "string",
"date": "string (ISO 8601)",
"start_time": "string (HH:MM, 24-hour)",
"end_time": "string (HH:MM, 24-hour) or null",
"timezone": "string (IANA timezone)",
"location": {
"venue": "string or null",
"address": "string or null",
"city": "string or null",
"virtual": "boolean"
},
"attendees_expected": "number or null",
"description": "string"
}
Input: "We're hosting a product launch on July 15th at 2pm EST, probably running about 90 minutes. It'll be at the downtown Marriott in Seattle. Expecting around 200 people."
Pattern 4: Multi-Item Extraction with Nested Objects
Extract all action items from the meeting notes and return as JSON:
{
"meeting_title": "string",
"date": "string (ISO 8601)",
"participants": ["string"],
"action_items": [
{
"id": "string — unique identifier like 'AI-001'",
"description": "string — what needs to be done",
"assignee": "string or null — who is responsible",
"due_date": "string (ISO 8601) or null",
"priority": "string — one of: high, medium, low",
"status": "string — one of: not_started, in_progress, completed",
"dependencies": ["string"] — IDs of other action items this depends on
}
],
"decisions": ["string — key decisions made in the meeting"],
"next_meeting": "string (ISO 8601) or null"
}
Pattern 5: Structured Comparison
Compare the products mentioned in the text and return as JSON:
{
"products": [
{
"name": "string",
"price": "number or null",
"currency": "string or null",
"features": ["string"],
"pros": ["string"],
"cons": ["string"],
"rating": "number (0-5) or null",
"recommended_for": ["string"] — use cases this product is good for
}
],
"best_overall": "string — name of the recommended product",
"best_value": "string — name of the best value product",
"comparison_summary": "string — 1-2 sentence summary"
}
Debugging Structured Output Failures
Even with the best prompts and native structured output, things go wrong. Here's how to diagnose and fix common failures:
Failure: Model Adds Extra Fields
Symptom: The model includes fields you didn't ask for.
Cause: The model is "being helpful" by adding data it thinks you'll want.
Fix: Add "Do NOT add any fields not in the schema. Only include the exact fields specified." If using structured outputs with a schema, set additionalProperties: false.
Failure: Model Wraps JSON in Markdown
Symptom: Output starts with ````jsoninstead of{`.
Cause: The model defaults to markdown formatting.
Fix: Use native JSON mode. If prompt-only, add "Do NOT use markdown code blocks. First character must be {."
Failure: Model Returns Empty or Null Values
Symptom: All fields are null or empty strings. Cause: The model didn't understand the input or couldn't find the information. Fix: Add instructions for what to do when information isn't present. Add an example where some fields are populated and some are null. Check if your input text actually contains the information.
Failure: Inconsistent Field Types
Symptom: Sometimes price is a number, sometimes a string like "$29.99".
Cause: The model isn't being strict about types.
Fix: Specify types explicitly: "price must be a NUMBER, not a string. Remove currency symbols." Or use function calling with strict type schemas.
Failure: Model Outputs Prose Instead of JSON
Symptom: The model writes a paragraph instead of structured data. Cause: Your output instructions aren't emphatic enough, or the model's helpfulness instinct overrides them. Fix: Move the output format instruction to the very end of the prompt (recency bias). Add an example that ends right before the output. Consider switching to function calling.
Failure: Truncated JSON
Symptom: The JSON is cut off mid-object.
Cause: You hit the model's max output token limit.
Fix: Increase max_tokens. If the output is naturally large (many entities), consider processing in batches or using a model with longer output limits.
Best Practices Checklist for Structured Output Prompts
- [ ] Provide at least one complete input→output example
- [ ] Specify the schema explicitly (field names, types, required/optional)
- [ ] Include edge case instructions (missing data → null, ambiguous → low confidence)
- [ ] Forbid markdown wrapping and extra text
- [ ] Use native structured output / JSON mode / function calling when available
- [ ] Set
additionalProperties: falsein schemas to prevent extra fields - [ ] Use enums for fields with a fixed set of valid values
- [ ] Specify date formats (ISO 8601), number formats (no currency symbols), and string constraints
- [ ] For complex schemas, break into nested objects with clear descriptions
- [ ] Test with edge cases: empty input, ambiguous input, input with no matching data
- [ ] Validate output with a JSON schema validator before using it in your application
- [ ] Have a fallback / retry strategy for when the model fails to produce valid output
The Golden Rule of Structured Output
Always validate the output. No prompt, no API feature, no function calling guarantee is 100% reliable. Your application should always validate the model's output against your expected schema before using it. If validation fails:
- Retry: Send the same request again (often works — the failure was stochastic).
- Repair: Send the broken output back to the model with "Fix this JSON: [broken output]. Return only valid JSON matching this schema: [schema]."
- Fallback: Use a default value or escalate to a human if retries and repair fail.
This defense-in-depth approach is what separates production structured output systems from demos.
Conclusion
Structured output is the skill that turns LLMs from chat toys into reliable software components. The techniques in this guide — explicit schemas, worked examples, native JSON mode, function calling, edge case handling, output validation — are what every production AI system uses to get machine-readable data out of language models. Master them, and you can build pipelines, agents, and automations that actually work in the real world.
The landscape is evolving fast — structured output features are getting more powerful and more native to the APIs themselves. But the fundamentals won't change: be explicit about what you want, show examples, handle edge cases, enforce with native features when possible, and always validate the output.
Start Building with Structured Prompts
PromptWright helps you design, test, and deploy structured output prompts that produce reliable, schema-conformant results. Build prompts with built-in validation, test against edge cases, and ship with confidence. Sign up free and start engineering prompts that power real software.
Enjoyed This Article?
Get more prompt engineering tips delivered weekly. Free, no spam.
Ready to build better prompts?
Try PromptWright free — structured prompt editor with multi-model testing.
Get Started Free →