Structured Outputs for Reliable AI APIs

Why the schema boundary matters

Most teams start with a prompt and a parser. That works until the model drops a field, invents an enum value, or quietly changes shape after a model upgrade. The real problem is not JSON syntax. The real problem is that the application has no contract boundary between probabilistic text generation and deterministic software.

OpenAI’s current guidance is clear: use Structured Outputs when you want schema adherence, and use strict function calling when the model is selecting a tool or action. That is a much stronger design than “please return valid JSON.” It turns the model into a producer of typed data instead of a clever but unreliable string generator.

If you already ship agents or tool-using systems, this layer sits right next to the boundary work in my MCP security threat modeling guide and my AI agent stacks guide. If you care about production diagnostics, it also complements the telemetry discipline in OpenTelemetry GenAI observability.

What Structured Outputs actually buys you

Structured Outputs gives you three concrete properties that plain prompting does not:

1. The model must produce data that conforms to your schema.
2. Safety refusals become programmatically detectable instead of being buried in free-form prose.
3. You stop relying on “strongly worded” prompts to force formatting.

That matters because most downstream failures are not dramatic. They are boring. A missing field becomes a bad UI state. A string where you expected a number becomes a crashed job. An unvalidated enum becomes a write to the wrong row or the wrong workflow branch.

This visual proves the core claim from the docs: JSON mode only guarantees valid JSON, while Structured Outputs adds schema adherence. It also shows the docs’ preference order, which is the right default for new systems.

The diagram is the point: the model never gets to skip the contract. In the response path, the schema constrains text generation before your app sees the payload. In the tool path, the schema constrains arguments before your code executes the side effect.

JSON mode, Structured Outputs, and strict tools

These are related, but they are not interchangeable.

That last row is important. Post-parse validation is still worth doing. It is just not enough on its own. If your schema is the front door, validation after the fact is the alarm system.

OpenAI’s docs also make the division of labor explicit: use function calling when the model bridges to tools or data in your system; use structured text formatting when the model is responding to the user in a structured way. That distinction is easy to miss and expensive to learn the hard way.

A TypeScript pattern that survives real teams

In a JavaScript or TypeScript codebase, the cleanest ergonomics come from Zod. OpenAI’s Node SDK helper converts the schema into JSON Schema and parses the model response back into a typed object. The important detail is not the helper itself. It is the discipline it enforces: schema first, parse second, business logic last.

1import OpenAI from 'openai';
2import { zodResponseFormat, zodFunction } from 'openai/helpers/zod';
3import { z } from 'zod/v3';
4
5const SupportReply = z.object({
6  sentiment: z.enum(['calm', 'frustrated', 'urgent']),
7  summary: z.string().describe('Short, user-facing summary'),
8  next_action: z.string().describe('What the system should do next'),
9  confidence: z.number(),
10  handoff_reason: z.string().nullable(),
11});
12
13const client = new OpenAI();
14
15const completion = await client.chat.completions.parse({
16  model: 'gpt-5.6',
17  messages: [
18    { role: 'system', content: 'Extract the support triage fields.' },
19    { role: 'user', content: 'My invoice is wrong and I need this fixed today.' },
20  ],
21  response_format: zodResponseFormat(SupportReply, 'support_reply'),
22});
23
24const parsed = completion.choices[0]?.message.parsed;
25
26if (!parsed) {
27  throw new Error('Model refused or returned no parseable output');
28}
29
30if (parsed.sentiment === 'urgent') {
31  // Route to a higher-priority queue.
32}

Two implementation details are easy to get wrong:

1. The root schema should be an object, not a free-form union.
2. If a field can be absent, model it as nullable instead of assuming a plain optional field will behave the way you expect.

That second point comes straight out of the SDK helper docs. The helper is strict about what the API can actually represent. That is a feature, not a nuisance. It forces you to model the contract the way the model can reliably emit it.

Strict function tools are not “just another schema”

Function calling looks similar on the surface, but the design goal is different. You are not asking for a structured answer. You are asking the model to choose a capability. That means the schema is attached to an action surface, not only to a response surface.

Here is the pattern I prefer for a read-only or side-effectful tool:

1const LookupOrder = z.object({
2  order_id: z.string().describe('Internal order identifier'),
3  include_refund_status: z.boolean(),
4});
5
6const run = await client.chat.completions.parse({
7  model: 'gpt-5.6',
8  messages: [
9    { role: 'system', content: 'Use the lookup tool when the user asks about an order.' },
10    { role: 'user', content: 'What is the status of order 18422?' },
11  ],
12  tools: [
13    zodFunction({
14      name: 'lookup_order',
15      parameters: LookupOrder,
16    }),
17  ],
18});
19
20const toolCall = run.choices[0]?.message.tool_calls?.[0];
21if (toolCall?.function.parsed_arguments) {
22  const args = toolCall.function.parsed_arguments;
23  // Execute a read-only lookup with args.order_id.
24}

The strict tool boundary is where many agent systems either become safe or become embarrassing. A tool schema with vague descriptions, too many free-form fields, or mixed read/write intent gives the model too much room to improvise. If the action mutates state, the tool should be narrow, explicit, and auditable.

This is why tool design belongs in the same room as your threat model. If the tool can send money, delete data, or write to a customer-facing system, then the schema is not just developer convenience. It is a security control.

The contract design rules I actually follow

When I review an AI endpoint or tool surface, I look for six things:

1. Keep the schema small and obvious.
2. Make invalid states unrepresentable with enums, nested objects, and required fields.
3. Use additionalProperties: false or the equivalent strict mode whenever the contract is supposed to be closed.
4. Make each field description specific enough that a human intern could use the tool correctly.
5. Split read-only extraction from state-changing actions.
6. Treat a refusal as a real program state, not an exception to hand-wave away.

The OpenAI function-calling guide is blunt about keeping the number of initially available tools small, using namespaces for related tools, and leveraging deferred loading only when the surface is large enough to justify it. That advice is not just about token cost. It is about model accuracy and operational clarity.

I also like a simple naming rule: if the tool name sounds like a verb, it is probably a mutation surface and should be treated as high risk. If it sounds like a query, it should still be read-only unless you have a very good reason otherwise.

Where teams get burned

Most failures are not exotic. They are schema hygiene failures.

If you are already doing observability well, instrument the parse path separately from the tool execution path. That makes it easy to see whether failures are coming from generation, validation, or your own application logic. The same request tracing pattern I described in OpenTelemetry GenAI observability works well here.

A production rollout checklist

Before you let this pattern into production, make sure the following are true:

• The schema lives in code, not in a one-off prompt string.
• The parser returns a typed object or a refusal state, never a raw blob that gets passed around unchecked.
• The tool surface is small at first and expands only with review.
• Mutation tools require explicit policy checks outside the model.
• The validation path is covered by tests for missing fields, wrong enums, refusals, and schema drift.
• Model upgrades run against a fixture set before they touch production.
• Tool execution is logged with enough detail to explain who asked for what, but not so much that you create a second data leak.
• You can point to the exact source of every contract claim in a source file or docs URL.

That last point is the one people skip. If the contract only exists in tribal memory, it will drift the moment the team changes. Put the schema next to the code, and keep the code next to the tests.

Frequently asked questions

Is JSON mode obsolete?

No. It still has a place when you need valid JSON on older model paths. But it is weaker than Structured Outputs because it does not guarantee schema adherence. If the application depends on the shape, prefer the stronger contract.

Do Structured Outputs remove the need for validation?

No. They remove one class of failure: malformed or off-schema model output. They do not remove business validation, authorization, rate limits, permission checks, or downstream API constraints. Think of Structured Outputs as the input contract, not the whole safety story.

Should I use Structured Outputs or function calling?

Use Structured Outputs when the model is producing a structured answer for the user or for downstream parsing. Use function calling when the model is choosing an action or tool. In practice, many production systems use both.

Why do the SDK helpers care so much about strict schemas?

Because the model can only emit what the API can faithfully represent. The helper enforces the strict subset that the API understands, which is much safer than silently pretending a richer schema will work.

Can I keep optional fields?

Yes, but model them carefully. In the Zod helper flow, nullable values are usually the safer choice because they map cleanly to the contract the model can actually satisfy. Treat “optional” as a data modeling decision, not a convenience default.

What is the biggest mistake people make with tool calling?

They mix “the model may suggest an action” with “the model is allowed to perform the action.” Those are different decisions. The model can propose the call, but your app still owns authorization, confirmation, and execution policy.

How does this relate to MCP or agents?

It is the lower-level contract layer underneath them. MCP and agent frameworks decide how tools are discovered, approved, and orchestrated. Structured Outputs and strict function tools decide what shape the model is allowed to emit. The boundaries stack, they do not replace each other.

Methodology and scope

This guide is a source-backed engineering analysis, not a benchmark or product announcement. I relied on OpenAI’s Structured Outputs guide, OpenAI’s function-calling guide, the official OpenAI Node and Python helper documentation in the openai-node and openai-python repositories, and the JSON Schema project’s overview page. Claims about schema adherence, strict tools, parser helpers, nullable-vs-optional behavior, and supported schema shape are grounded in those sources as of 2026-07-25 UTC.

Where I recommend a pattern beyond the docs, I label it as design guidance. Where the docs are specific, I keep the wording tight to avoid implying guarantees the API does not make. If your deployment has its own model policy, safety requirements, or tool-approval flow, those rules win.

Conclusion

The useful mental model is simple: the model should propose, the schema should constrain, and the application should decide. When you apply that sequence consistently, AI endpoints stop feeling like stringly-typed demos and start behaving like real software boundaries.

If you are building typed AI workflows, start by shrinking the contract until it is almost boring. Then add tests for refusals, schema drift, and tool-policy failures. Once those are stable, the rest of the system gets dramatically easier to trust.

For teams that want help hardening the full stack around this boundary, I’m available through essamamdani.com/hire.

Author context

I write about AI systems and full-stack engineering from the point where product behavior meets production reality. My bias is toward explicit contracts, narrow tool surfaces, and enough telemetry to debug failures without exporting private data everywhere.

Sources

• OpenAI Structured model outputs guide
• OpenAI Function calling guide
• OpenAI Node SDK helpers
• OpenAI Python SDK helpers
• JSON Schema overview
• OpenAI announcement: Introducing Structured Outputs in the API