Structured Outputs
Last updated
Last updated
Dify’s structured outputs ensures LLMs return data in predictable JSON formats, making the output easier to process and integrate into applications.
Consistent Data Formatting: All LLM-generated content follows predefined formats, eliminating data inconsistencies.
Seamless Integration: : Databases, APIs, or frontend applications can directly parse JSON Schema without additional data cleaning.
Simplified Development: Developers can skip writing complex validation code by letting JSON Schema handle output constraints.
Dify offers two ways to create structured outputs:
Define your output structure directly in tool parameters. See > Data Structures > Return Variable Definitions.
Use the built-in editor in LLM nodes. See > Advanced Features > Structured Outputs and > Use Cases > Structured Outputs.
What Can Go Wrong
When you are working with the JSON Schema Editor, you might run into a few common challenges:
Limited Model Performance: Smaller LLMs (like GPT-3.5 Turbo) sometimes have trouble following JSON Schema instructions correctly.
Format Issues: Some LLMs only understand basic JSON, not full JSON Schema.
Error Messages: You might see Failed to parse structured output: output is not a valid json str when the model fails to generate proper JSON.
How to Fix These Issues
Choose the Right Model. These models work best with JSON Schema:
Gemini 2.0 Flash/Flash-Lite
Gemini 1.5 Flash 8B (versions 0827/0924)
Gemini-1.5 pro
GPT-4o and GPT-4o-mini
o1-mini/o3-mini series
Write Clear Instructions. Make sure your prompts match what you are trying to do. For example, if you need mathematical formulas, don’t ask for legal analysis—this helps the AI understand exactly what you need.
Have a Backup Plan.
Enable Retry on Failure with custom retry attempts and intervals.
Set up Failure Branch to handle any errors.
See for more details.