Structured Outputs

Making LLM Responses Reliable for Applications

So far in this series, we’ve spent a lot of time improving how models respond. Check out the LLM Fundamentals here.

We learned how prompts influence behavior.

We saw how examples can reduce ambiguity.

We explored how temperature controls creativity and response diversity.

Together, these form an important foundation for working with LLMs.

But there is a practical problem that appears the moment we move beyond chat applications.

Humans can comfortably read paragraphs.

Applications cannot.

Imagine asking an LLM to extract customer information from a support ticket.

One response might be:

The customer's name is Elsa and she is 28 years old.

Another response might be:

Elsa (Age: 28)

And another might be:

{
  "name": "Elsa",
  "age": 28
}

For a human reader, all three responses mean exactly the same thing.

For software, they are completely different.

A backend service, database, or API integration often expects data in a predictable format.

If the structure changes every time, the application becomes difficult to automate.

This is where structured outputs become important.

Because in real-world systems, generating text is only part of the problem.

The bigger challenge is generating responses that other systems can reliably understand and process.

The Problem With Free-Form Text

At first glance, this may not seem like a difficult problem.

After all, can’t we simply ask the model to return JSON?

For example:

Extract the user's name and age.
Return the answer as JSON.

Most of the time, the model will do exactly that.

{
  "name": "Elsa",
  "age": 25
}

Problem solved.

Or at least, that’s what it looks like.

Because if you’ve worked with LLMs long enough, you’ll discover something interesting.

The model doesn’t always respond in exactly the same way.

One response might look like:

{
  "name": "Elsa",
  "age": 25
}

Another might look like:

{
  "name": "Elsa",
  "age": 25
}

Hope this helps!

And another might decide to format the output differently altogether.

A human can easily understand all of these responses.

A parser cannot.

Suddenly, a response that looks perfectly reasonable to us becomes a source of bugs for the application.

So the question becomes:

How do we move from asking nicely to enforcing structure?

Defining the Structure

The problem here is not JSON itself.

The problem is that we are still relying on the model to voluntarily follow instructions.

What we really want is a way to define exactly what the response should look like.

Instead of describing the structure in a prompt, modern LLM applications often define the structure explicitly.

For example:

from pydantic import BaseModel

class User(BaseModel):
    name: str
    age: int

This schema tells the application exactly what fields are expected and what type of data each field should contain.

At this point, we are no longer saying:

Please return JSON like this...

Instead, we are defining a contract.

Get Kusuma Pindi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The model is expected to generate a response that matches the structure we have specified.

Many modern LLM frameworks and APIs support this approach directly, making responses significantly more predictable than relying on prompt instructions alone.

But defining a structure introduces another important question.

If the response follows the correct structure, does that automatically mean the data is valid?

Not necessarily.

Structure Is Not Validation

Consider the following response:

{
  "name": "Elsa",
  "age": "twenty five"
}

At first glance, everything looks fine.

The required fields are present.

The JSON is valid.

The structure matches our schema.

But there is still a problem.

Our application expects age to be a number, not a string.

This is where validation becomes important.

Tools like Pydantic can verify that the generated response actually matches the schema we defined.

If a required field is missing, a data type is incorrect, or the structure does not match expectations, validation fails.

In other words:

• Structured Outputs define the shape.
• Validation verifies the contents.

Together, they help transform LLM responses from free-form text into data that applications can reliably work with.

Structured Outputs in Real Systems

At this point, structured outputs may seem like a formatting feature.

In reality, they are what make LLMs useful inside larger software systems.

Consider an AI assistant processing customer support emails.

The goal is not simply to generate a response.

The system may need to:

• Extracting customer information from support tickets
• Pulling invoice details from documents
• Generating database records
• Calling external APIs
• Routing requests inside AI agents

In all of these situations, the model is not producing an answer for a person to read.

It is producing data that another system will consume.

The response becomes part of a larger workflow.

And workflows depend on consistency.

Once we start looking at LLM applications this way, the model begins to feel less like a chatbot and more like a component inside a software pipeline.

A request comes in.

The model processes information.

A structured response is generated.

The response is validated.

Then another system takes over.

The flow often looks something like this:

Prompt
   ↓
LLM Generation
   ↓
Structured Output
   ↓
Validation
   ↓
Application

The goal is no longer just generating text.

The goal is generating data that software can reliably use.

There is, however, one important limitation that structured outputs cannot solve.

A response can be perfectly structured and still be incorrect.

For example:

{
  "capital": "Berlin"
}

The JSON is valid.

The structure is correct.

The schema passes validation.

The answer is still wrong.

And that raises an interesting question.

If prompts can influence behavior, temperature can influence creativity, and structured outputs can enforce format, why do models still generate incorrect information?

That’s exactly what we’ll explore in the next article on Hallucinations.

See you there 👋