Part 13 — AI Frameworks: How Developers Build Smarter AI Applications
Understanding what AI frameworks are, how they differ from SDKs, and how developers use them to build AI-powered workflows, RAG systems, and intelligent agents.
Introduction
You have built your first AI-powered feature.
Your application sends a question to an AI model through an API and receives an answer.
Everything works.
Now imagine your requirements start growing.
You want your AI application to:
- Read information from multiple documents.
- Search a database before answering.
- Remember previous conversations.
- Call external APIs when necessary.
- Perform multiple tasks in a specific order.
- Validate results before displaying them.
Suddenly, a simple API call is no longer enough to organize everything.
You could write all this logic yourself.
But as your application grows, managing these different components becomes more complicated.
This is where AI Frameworks come into the picture.
In our previous articles, we learned about AI APIs and AI SDKs.
Now let’s understand the next layer: AI Frameworks.
What exactly are they? How do they work? Which frameworks are commonly used? And does every AI application really need one?
Let’s explore everything step by step.
1. What Is an AI Framework?
An AI framework is a collection of reusable components, tools, and programming structures that helps developers build and organize AI-powered applications.
Instead of implementing every integration, workflow, and coordination mechanism from scratch, developers can use a framework.
Think about Android development.
You may use Jetpack Compose to build UI components and navigation libraries to manage navigation.
These tools provide structure and reusable functionality.
You still write the application logic, but you don’t have to create every building block yourself.
AI application frameworks follow a similar concept.
For example, imagine you are building an AI assistant that answers questions using company documentation.
The application needs to:
- Receive a question.
- Search relevant documents.
- Retrieve useful information.
- Send that information to an AI model.
- Generate an answer.
- Return the result to the user.
An AI framework can help connect and coordinate these steps.
In simple words: An AI framework helps developers organize multiple AI components into a working application.
One important distinction:
The term AI framework is broad. It can also refer to machine learning frameworks such as TensorFlow or PyTorch.
In this article, we are focusing on frameworks for building applications using existing generative AI models, such as LangChain, LangGraph, and LlamaIndex.
2. Why Do We Need AI Frameworks?
Let’s start with a simple scenario.
Suppose you want to build a chatbot.
Your first version only needs to answer general questions.
The architecture is straightforward.
User
|
v
Application
|
v
AI SDK
|
v
AI Model
|
v
AnswerAn AI SDK is usually enough for this.
Now your product team asks for additional features.
The chatbot should answer questions using internal company documents.
It should also retrieve customer details from a database and check order status through an external API.
Now the architecture becomes more complex.
User
|
v
Application
|
v
AI Workflow Layer
|
+------+------+
| | |
v v v
Docs Database API
| | |
+------+------+
|
v
AI Model
|
v
Process Result
|
v
UserNow developers must coordinate different components.
They need to decide which information to retrieve, when to call the model, and how to handle failures.
An AI framework can help manage this coordination.
However, adding a framework does not automatically make an application better.
For a simple feature, manually writing a few functions may be easier and more maintainable.
Frameworks become useful when the complexity they remove is greater than the complexity they introduce.
3. AI API vs AI SDK vs AI Framework
These three terms are closely related, but each solves a different problem.
Let’s understand them together.
AI API
An AI API provides an interface for communicating with an AI service.
Example:
POST /v1/responses
The application sends a request and receives a result.
AI SDK
An AI SDK provides ready-made libraries and functions to simplify that communication.
Example:
response = client.responses.create(
model="gpt-5-mini",
input="Explain artificial intelligence."
)AI Framework
An AI application framework helps organize multiple operations, such as model calls, document retrieval, workflow steps, and tool execution.
It provides building blocks for more complex applications.
Quick Comparison
ComponentMain purposeAI APICommunicate with an AI serviceAI SDKSimplify AI service integrationAI FrameworkOrganize complex AI application logicAI ModelPerform the underlying AI processing
Simple Architecture
Your Application
|
v
AI Framework
(Workflow / Logic)
|
v
AI SDK
(Integration Layer)
|
v
AI API
(Communication Layer)
|
v
AI Model
(AI Processing)This diagram represents one common setup, not a mandatory architecture.
Some frameworks communicate directly with model providers. Others integrate with provider SDKs or additional libraries.
The key point is that these components have different responsibilities.
4. How Does an AI Framework Work?
An AI framework provides reusable building blocks that developers combine into an application.
Let’s use a news application as an example.
Suppose a user asks:
“Summarize this article and tell me whether it is related to technology or business.”
The application needs to perform multiple steps.
Step 1 — Receive the article
The application receives the text or retrieves the article from its backend.
Step 2 — Generate a summary
The AI model summarizes the content.
Step 3 — Classify the article
The application determines whether the article belongs to technology, business, or another category.
Step 4 — Prepare the result
The backend validates the generated output and prepares a response suitable for the application.
Step 5 — Return the response
The application displays the result.
Workflow
News Article
|
v
Summarize
|
v
Classify
|
v
Validate Output
|
v
Return ResultThis is an example of a multi-step workflow.
A framework can help developers define these steps, pass data between them, and control the execution flow.
The same workflow can also be implemented using ordinary Python functions.
The framework becomes more valuable when the application needs additional branching, persistent state, human approval, or complicated execution paths.
5. Important Components of AI Frameworks
Before exploring popular frameworks, let’s understand some common concepts.
1. Models
The AI model performs tasks such as generating text, answering questions, or extracting information.
A framework typically connects to existing models instead of requiring you to train one.
2. Prompts
Prompts contain instructions provided to the model.
For example:
Summarize the following article in three short bullet points. Do not add information that is not present in the article.
Frameworks may offer reusable prompt templates.
3. Tools
Tools allow AI applications to interact with external functionality.
Examples include:
- Searching a database
- Calling a weather API
- Looking up customer orders
- Performing calculations
The application must enforce permissions and validate tool inputs.
4. State
State represents information maintained during a workflow.
For example, an article-processing workflow might store:
- Article text
- Generated summary
- Category
- Validation status
Some frameworks support state management and persistence.
5. Workflows
Workflows define how different tasks connect.
For example:
Retrieve → Summarize → Classify → Validate
They may include conditions, retries, loops, and approval steps.
6. Memory
Memory allows certain AI applications to retain useful information between interactions or workflow steps.
This can involve conversation history, stored user preferences, or other relevant context.
Memory is not automatic simply because a framework is installed.
7. Retrievers
Retrievers find relevant information from documents, search indexes, or other data sources.
They are commonly used in Retrieval-Augmented Generation (RAG) applications.
Not every framework supports all these features in the same way.
6. Popular AI Frameworks Developers Should Know
Several frameworks help developers build AI-powered applications.
Here are some important ones.
1. LangChain
LangChain is an open-source framework for building applications using language models.
It provides abstractions for working with models, tools, agents, and other AI application components.
Common use cases include:
- AI assistants
- Tool-enabled applications
- Language model integrations
- Agent-based applications
Website: https://www.langchain.com/
2. LangGraph
LangGraph is a framework for building stateful AI workflows.
It allows developers to organize operations as nodes connected by edges.
This is especially useful when an application requires branching, loops, or complex workflow coordination.
Common use cases include:
- Multi-step AI workflows
- Stateful agents
- Human approval flows
- Workflows mixing AI decisions with fixed business logic
Website: https://www.langchain.com/langgraph
3. LlamaIndex
LlamaIndex focuses on connecting AI applications with external and private data.
It provides components for working with documents, retrieval systems, indexes, agents, and workflows.
Common use cases include:
- Question answering over company documents
- Retrieval-Augmented Generation
- Document analysis
- Knowledge assistants
Website: https://www.llamaindex.ai/
4. Haystack
Haystack is an open-source framework for building AI applications using pipelines, components, document stores, and agents.
Common use cases include:
- Semantic search
- Document retrieval
- RAG applications
- AI processing pipelines
Website: https://haystack.deepset.ai/
5. Microsoft Agent Framework
Microsoft Agent Framework supports AI agents and workflow orchestration, including scenarios involving tools and multiple agents.
It brings together capabilities from Microsoft’s earlier Semantic Kernel and AutoGen projects.
Common use cases include:
- Enterprise AI applications
- Tool-enabled AI agents
- Stateful agent workflows
- Multi-agent orchestration
Documentation: https://learn.microsoft.com/en-us/agent-framework/
Quick Comparison
FrameworkUseful starting pointLangChainAI assistants and tool integrationLangGraphStateful and multi-step workflowsLlamaIndexAI applications using external dataHaystackRetrieval and AI pipelinesMicrosoft Agent FrameworkEnterprise agents and orchestration
These are not strict boundaries.
Several frameworks support overlapping capabilities.
The right choice depends on your actual requirements.
7. Practical Example — Building a Simple AI Workflow
Let’s build a small application using an AI framework.
Our requirement is straightforward.
We want to:
- Receive an article.
- Generate an AI summary.
- Format the result.
- Return it.
We’ll use Python, LangGraph, and the OpenAI SDK.
This is intentionally a small example to understand how frameworks organize application logic.
For only two steps, ordinary Python functions would also be perfectly reasonable.
Step 1 — Install Dependencies
pip install langgraph openaiConfigure your API key as an environment variable.
For macOS or Linux:
export OPENAI_API_KEY="your_api_key"Step 2 — Define the Workflow
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
from openai import OpenAIclient = OpenAI()# Define workflow data
class ArticleState(TypedDict, total=False):
article: str
summary: str
result: str
# Step 1: Generate summary
def summarize_article(state: ArticleState):
response = client.responses.create(
model="gpt-5-mini",
instructions=(
"Summarize the article in "
"three simple sentences. "
"Use only the provided content."
),
input=state["article"]
) return {
"summary": response.output_text
}
# Step 2: Format result
def format_result(state: ArticleState):
return {
"result": (
"Article Summary:\n"
+ state["summary"].strip()
)
}
# Create workflow
workflow = StateGraph(ArticleState)workflow.add_node(
"summarize",
summarize_article
)workflow.add_node(
"format",
format_result
)workflow.add_edge(START, "summarize")
workflow.add_edge("summarize", "format")
workflow.add_edge("format", END)app = workflow.compile()# Execute workflow
output = app.invoke({
"article": (
"A technology company introduced "
"a new expense tracking application. "
"The application helps users monitor "
"spending and categorize transactions."
)
})print(output["result"])
Step 3 — Understand the Code
Let’s break down the main components.
ArticleState
Defines the information shared across the workflow.
summarize_article()
Calls the AI model and generates a summary.
format_result()
Formats the generated content for display.
StateGraph()
Creates the workflow structure.
add_node()
Registers the functions that will run.
add_edge()
Defines the order in which the steps execute.
compile()
Builds the executable workflow.
invoke()
Runs the workflow using the supplied input.
Example Output
The output might look like:
Article Summary:A technology company introduced
a new expense tracking application.The application helps users monitor
their spending.Users can also categorize transactions.
The exact wording may vary.
What Did the Framework Actually Do?
The AI SDK communicated with the model.
LangGraph managed the workflow and passed information from one step to the next.
In other words:
Get Yogendra Pratap Singh (Tech Architect)’s stories in your inbox
Join Medium for free to get updates from this writer.
SDK = AI communication
Framework = Workflow coordination
This is a useful distinction when designing AI applications.
8. What Are Nodes, Edges, and State?
These terms appear frequently when working with workflow frameworks.
Let’s simplify them.
Node
A node represents an operation.
For example:
- Retrieve article
- Summarize content
- Classify result
- Validate output
In LangGraph, a node can be a Python function.
Edge
An edge connects one node to another.
It tells the workflow where to go next.
For example:
Summarize → Classify
Some edges can be conditional.
State
State stores information that different nodes need during execution.
For example:
{
"article": "Example article content",
"summary": "Generated summary",
"category": "Technology"
}The workflow updates this state as execution progresses.
Example With Conditional Logic
User Input
|
v
Detect Request
|
v
What Is Required?
/ \
/ \
v v
Summarize Translate
| |
+------+------+
|
v
Validate
|
v
ResponseThis is where graph-based frameworks become useful.
Developers can explicitly define possible execution paths instead of managing everything inside one large function.
9. AI Frameworks and RAG Applications
One important use case for AI frameworks is Retrieval-Augmented Generation, commonly called RAG.
Let’s understand this with an example.
Suppose your company has 500 internal documents.
These documents contain information about employee policies, technical architecture, processes, and product requirements.
You want to build an AI assistant that answers questions using those documents.
For example:
“What is our company’s leave policy?”
A general language model might not know your organization’s internal policies.
Even if it produces an answer, that answer may not be correct.
A RAG system retrieves relevant information from the company’s approved data sources and provides it to the model.
Basic RAG Architecture
User Question
|
v
Search / Retriever
|
v
Relevant Documents
|
v
Prepare AI Context
|
v
AI Model
|
v
Generated Answer
|
v
UserAI frameworks such as LlamaIndex, LangChain, and Haystack provide components that can help build this kind of application.
Why Is RAG Useful?
It allows an AI application to work with information that was not included in the model’s original training.
Common examples include:
- Internal knowledge assistants
- Customer support systems
- Document search
- Enterprise knowledge bases
- Technical documentation assistants
But remember:
RAG does not guarantee factual accuracy.
The application must still check retrieval quality, access permissions, source relevance, and generated responses.
We’ll explore RAG in greater detail in a future article.
10. AI Workflows vs AI Agents
Another important distinction is between workflows and agents.
They are related, but not identical.
AI Workflow
In a workflow, developers generally define the sequence of operations.
For example:
Receive Document
|
v
Extract Text
|
v
Summarize Text
|
v
Generate OutputThe order is explicitly controlled by application logic.
AI Agent
An AI agent may use a model to decide which permitted tools or actions to use while completing a task.
For example:
A user asks:
“Find my latest order and tell me its delivery status.”
An agent could decide to:
- Look up the user’s order.
- Call the delivery-status tool.
- Interpret the result.
- Generate a response.
Main Difference
AI WorkflowAI AgentUsually follows predefined execution rulesCan dynamically choose actions within defined boundariesEasier to make execution predictableMore flexible for variable tasksSuitable for well-defined processesUseful when the next action depends on the taskCan include AI model callsOften uses models for action selection
An application can combine both approaches.
For example, a workflow might use an AI agent for one specific step.
Not every AI application needs an agent.
If the required sequence is known, a normal workflow is often simpler, easier to test, and more predictable.
11. Real-World Example — AI Framework in a News Application
Let’s take a practical architecture example.
Suppose you are building an Android and iOS news application.
Your product team wants the following features:
- Summarize articles.
- Categorize content.
- Generate short headlines.
- Check whether generated output follows editorial rules.
- Return structured results to the applications.
Instead of building each feature as unrelated AI calls, your backend can use an organized workflow.
Architecture
Android / iOS App
|
v
Backend API
|
v
AI Framework Layer
|
v
Retrieve Article
|
v
Generate Summary
|
v
Classify Category
|
v
Generate Headline
|
v
Validate AI Output
|
v
Structured Result
|
v
Android / iOS AppExample Response
{
"articleId": "12345",
"summary": "A short article summary.",
"category": "Technology",
"headline": "New Technology Service Launched",
"validationStatus": "passed"
}Here, the framework can coordinate different tasks.
The backend is still responsible for validation, authentication, authorization, caching, and error handling.
Do We Really Need a Framework Here?
Not necessarily.
If there are only three or four simple sequential operations, normal backend functions may work perfectly well.
But if the workflow grows to include branching, parallel processing, retries, human review, or complex state management, a framework may become more useful.
The decision should depend on complexity, not popularity.
12. AI Frameworks vs Traditional Backend Architecture
Developers sometimes assume AI frameworks replace normal backend architecture.
They don’t.
An AI framework does not replace:
- Authentication
- Authorization
- Database management
- Business logic
- API design
- Monitoring
- Infrastructure
- Security controls
Think of an AI framework as one layer within your application.
Example Production Architecture
Mobile / Web App
|
v
API Gateway
|
v
Backend Service
|
+------+------+
| |
v v
Database AI Workflow
|
v
AI Framework
|
+-----+-----+
| |
v v
AI Model External ToolsThe backend remains responsible for enforcing the application’s rules.
The framework helps coordinate AI-specific processing.
This separation matters when building applications that must be secure, scalable, and maintainable.
13. Do AI Frameworks Improve Performance?
Not automatically.
A framework can simplify development, but it does not necessarily make AI models respond faster.
In fact, frameworks may introduce additional overhead.
For example, suppose your application originally makes one AI API request.
After introducing a complex framework, it performs:
- An AI call to understand the request.
- Another call to select a tool.
- A retrieval operation.
- Another AI call to generate the answer.
- A final AI call to review the output.
Your application now performs more operations.
This can increase latency and cost.
What Should Developers Measure?
Some important metrics include:
MetricWhat it measuresEnd-to-end latencyTotal time required to complete a requestModel callsNumber of AI model requestsToken consumptionInput and output token usageError ratePercentage of requests that failTool execution timeTime spent calling external servicesOutput qualityWhether the result meets requirementsCost per requestAverage expense per completed task
Frameworks may support more efficient workflow coordination, but they cannot eliminate the cost of unnecessary operations.
A more complicated AI architecture is not automatically a better one.
14. How Much Do AI Frameworks Cost?
Many popular AI application frameworks are open source.
For example, you can install LangGraph or LangChain without purchasing a commercial license for the core open-source packages.
pip install langgraphBut this does not mean your application is free to operate.
Your total costs may include:
- AI model API usage
- Cloud infrastructure
- Database hosting
- Vector databases
- Storage
- Monitoring
- Managed framework or orchestration services
Simple Example
Suppose a single AI model request costs a small amount.
If your workflow requires five model calls instead of one, the total inference cost can increase.
The actual increase depends on model choice, token usage, caching, and the type of operations.
Before deploying a complex AI workflow, estimate the cost per successful user request.
Not merely the cost per model call.
This gives you a more realistic understanding of production expenses.
15. Common Mistakes Developers Make With AI Frameworks
Let’s look at some common mistakes.
Mistake 1 — Using a Framework for Every AI Feature
A simple text summarizer probably does not require a complex framework.
An AI SDK and a backend function may be sufficient.
Mistake 2 — Choosing Frameworks Based on Popularity
A framework may be widely discussed but still be unsuitable for your project.
Always evaluate actual requirements.
Mistake 3 — Adding Too Many Dependencies
Combining multiple frameworks without clear responsibilities can make debugging and maintenance difficult.
Start with one framework only if you genuinely need it.
Mistake 4 — Ignoring Model Costs
Complex workflows may call multiple models.
Monitor usage at the workflow level, not only at the individual API level.
Mistake 5 — Assuming Frameworks Handle Security
Frameworks may provide useful security-related features.
But they do not automatically protect API keys, prevent prompt injection, or enforce user permissions.
Your application must still implement those controls.
Mistake 6 — Confusing Frameworks With Models
LangChain and LangGraph are not language models.
They are software frameworks.
They typically coordinate interactions with existing models.
Mistake 7 — Following Outdated Tutorials
AI frameworks change quickly.
Classes, package names, and recommended integration patterns may evolve.
For example, older LangChain tutorials may use APIs that are no longer recommended in current versions.
Use official documentation, verify dependencies, and test your code against the specific framework version you install.
16. When Should You Use an AI Framework?
Here is a simple decision guide.
RequirementRecommended starting pointGenerate a short summaryAI SDKTranslate textAI SDKClassify an articleAI SDKBuild a chatbot with basic conversation historyAI SDK plus backend storageAnswer questions using many company documentsEvaluate a RAG frameworkCoordinate complex, stateful AI tasksEvaluate a workflow frameworkBuild a tool-using AI assistantEvaluate an agent frameworkProcess several tasks with branching and approvalEvaluate a graph or workflow framework
There is no universal rule that every project above a certain size requires a framework.
For example, a basic RAG system can also be built using ordinary backend code and a vector search service.
Frameworks are optional tools.
Choose them when they make your architecture easier to build, understand, test, and maintain.
17. A Practical AI Framework Development Workflow
If you decide that your application needs a framework, follow a structured approach.
Step 1 — Define the user problem
Clearly identify what the AI feature needs to achieve.
Step 2 — Start with the simplest solution
Check whether one AI SDK call and a few backend functions are enough.
Step 3 — Identify workflow complexity
Determine whether your application requires multiple tools, state, branching, retrieval, or human approval.
Step 4 — Evaluate frameworks
Compare the frameworks based on your programming language, use case, documentation, and maintenance requirements.
Step 5 — Build a small prototype
Test one representative use case before introducing the framework throughout your application.
Step 6 — Measure performance
Compare latency, costs, output quality, and reliability.
Step 7 — Add security and validation
Protect credentials, validate tool inputs, and enforce user permissions.
Step 8 — Test failure scenarios
Check what happens if a model times out, a tool fails, or a workflow produces invalid data.
Step 9 — Monitor production behavior
Track execution paths, model calls, errors, and spending.
Step 10 — Improve gradually
Add components only when they solve a specific requirement.
Complete Workflow
Define Problem
|
v
Check Simple Solution
|
v
Need Complex Workflow?
/ \
No Yes
| |
v v
AI SDK Evaluate Framework
| |
| v
| Build Prototype
| |
+------+-----+
|
v
Test & Validate
|
v
Security Review
|
v
Production Release
|
v
Monitor & ImproveThe goal is not to adopt the most advanced framework.
The goal is to build a solution that meets your requirements without unnecessary complexity.
18. Do Developers Really Need to Learn AI Frameworks?
If you are a frontend, backend, mobile, or full-stack developer, understanding AI frameworks can be useful.
But you don’t need to master every framework.
A good learning sequence is:
First: Understand how AI APIs work.
Second: Learn to integrate models using AI SDKs.
Third: Build a simple AI-powered feature.
Fourth: Understand prompts, response validation, latency, and cost.
Fifth: Learn how RAG, tools, and AI workflows work.
Finally: Explore a framework when you understand the problem it helps solve.
For mobile developers, the most important skill is often understanding how AI fits into the broader application architecture.
For backend developers, it is designing reliable services that coordinate models, data sources, and external tools.
For full-stack developers, it is connecting all these capabilities into a useful product.
You do not need five different frameworks to build a good AI application.
You need a clear understanding of the problem and the right tools to solve it.
Summary
AI frameworks help developers build applications that require more than a single AI model request.
They provide reusable components for organizing workflows, connecting external data, coordinating tools, and managing application state.
Here are the main takeaways:
- AI frameworks organize application logic. They help developers connect different AI components and operations.
- AI APIs, SDKs, and frameworks solve different problems. APIs provide access, SDKs simplify integration, and frameworks coordinate application behavior.
- Not all AI applications need frameworks. Simple features often work well with an AI SDK and normal backend code.
- LangChain, LangGraph, LlamaIndex, Haystack, and Microsoft Agent Framework are useful options. Each offers different strengths and overlapping capabilities.
- RAG is a common framework use case. It helps applications retrieve relevant external information before generating responses.
- Workflows and agents are not the same. Workflows generally follow predefined logic, while agents can dynamically select permitted actions.
- Frameworks do not replace software engineering fundamentals. Security, validation, monitoring, performance, and maintenance remain essential.
- More abstraction can introduce more complexity. Choose frameworks based on measured requirements, not trends.
AI frameworks can make sophisticated AI applications easier to develop.
But they are not magic solutions.
A framework helps you organize complexity. It should not become the reason your application is complex.
The best architecture is not necessarily the one using the most advanced AI tools.
It is the one that solves the user’s problem reliably, securely, and efficiently.
Previous Article: Part 12 — AI SDKs: How Developers Integrate AI Without Building Everything from Scratch
Next Article: Part 14 — AI Agents: How AI Can Use Tools and Complete Tasks
Explore the Series: The AI Terminology Handbook for Beginners (2026)
Additional Learning Resources
- LangChain Documentation
- LangGraph Documentation
- LlamaIndex Documentation
- Haystack Documentation
- Microsoft Agent Framework Documentation
Artificial Intelligence · AI Frameworks · Generative AI · LangChain · LangGraph · RAG · AI Engineering · Software Architecture
