<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Abhilash Puli on Medium]]></title>
        <description><![CDATA[Stories by Abhilash Puli on Medium]]></description>
        <link>https://medium.com/@abhilashpuli?source=rss-d3969283167e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*N-6P_qER1gxalYhoc_inoQ.jpeg</url>
            <title>Stories by Abhilash Puli on Medium</title>
            <link>https://medium.com/@abhilashpuli?source=rss-d3969283167e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 22:13:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abhilashpuli/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Agent Graph Engineering : A Complete Blog to Production AI Agent Architecture — Part-3]]></title>
            <link>https://ai.plainenglish.io/agent-graph-engineering-a-complete-blog-to-production-ai-agent-architecture-part-3-7676098c04e6?source=rss-d3969283167e------2</link>
            <guid isPermaLink="false">https://medium.com/p/7676098c04e6</guid>
            <category><![CDATA[workflow-automation]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm-agent]]></category>
            <dc:creator><![CDATA[Abhilash Puli]]></dc:creator>
            <pubDate>Thu, 24 Sep 2026 19:10:58 GMT</pubDate>
            <atom:updated>2026-09-24T20:06:58.854Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong><em>AI Agents Aren’t Just Models Anymore!</em></strong></blockquote><p>Written by Abhilash Puli, exploring the future of AI through, engineering, and in-depth technical writing on machine learning, LLMs, and AI agents.</p><p>Welcome back to Part 3. If you’re new to the series, I’d recommend reading Part 1 and Part 2 first, where we covered <a href="https://medium.com/ai-in-plain-english/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-882e0c2e083c"><strong>Agent Harness Engineering</strong></a> and <a href="https://ai.plainenglish.io/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-6fbd080ab85f"><strong>Loop Engineering</strong></a>.</p><h3>Part 3: Graph Engineering</h3><h3>From Making Agents Improve to Giving Them Structure</h3><p>In Part 1, we looked at <strong>Agent Harness Engineering</strong>, the environment that gives an AI agent context, tools, memory, execution controls, safety, and observability.</p><p>In Part 2, we moved one step further with <strong>Loop Engineering</strong>, where the agent can evaluate its results, learn from failures, and try again.</p><blockquote><strong>But now we have another problem but we also have solution don’t worry!!!.</strong></blockquote><h3>What happens when the task itself becomes more complex?</h3><p>Real-world tasks rarely follow one simple path.</p><blockquote>An agent might need to research something, analyze the results, ask for approval, perform an action, handle a failure, and then continue from a different step we can observe during use of IDE’s Like Antigravity,Cursor,Vscode and so on.</blockquote><p>A simple loop starts becoming difficult to manage.That is where <strong>Graph Engineering</strong> comes in.</p><blockquote><strong>The harness gives the agent an environment. The loop gives it feedback. The graph gives it structure.</strong></blockquote><p>Instead of thinking about an agent as one continuous loop, Graph Engineering lets us define <strong>different steps, decisions, branches, and recovery paths</strong> that the agent can move through.</p><p>In this part, we’ll look at how these graphs are designed, why they are useful, and when adding a graph actually makes sense instead of just making the architecture more complicated.</p><h3>I. What Is Graph Engineering?</h3><p>Firstly What Is Graph Engineering in sinple it’s <strong>A loop works well when an agent needs to repeatedly improve the same task</strong>. But real-world tasks are often more complicated than that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XCaAjAxyGFqxdpL1YVFcbw.png" /><figcaption>An agent may need to <strong>research, evaluate, make a decision, ask for approval, perform an action, or recover from a failure</strong>.</figcaption></figure><p>Graph Engineering gives structure to these workflows by representing them as connected steps.</p><blockquote><strong><em>Nodes define what the agent does. Edges define where it goes next.</em></strong></blockquote><blockquote>Instead of forcing everything into one large loop, a graph lets the agent follow different paths based on what happens at each step.</blockquote><p>[ Think of it as a <strong>map for the agent’s workflow</strong>. ]</p><h3>II. Why Graphs Exist actually?</h3><p>A simple loop works when the agent keeps repeating the same kind of action.</p><p>But real tasks rarely stay that simple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0qcVXCfMRvCGNZjXyUTFJg.png" /><figcaption>Workflow — Simple vs Graph</figcaption></figure><p>An agent might have to follow routes based on what occurs. For instance after looking at research findings it could keep going if the sources are strong look if they are not good or get a person to approve before continuing.</p><p><strong>Trying to manage all these situations, within a loop can get complicated fast.</strong></p><p>Graphs show the process clearly. Of making the agent work out the whole path each time we set the possible routes ahead of time.</p><blockquote><strong><em>Loops help agents repeat. Graphs help agents navigate.</em></strong></blockquote><h3>[ A Simple Graph Example ]</h3><p>Let’s Take a <strong>customer support agent</strong>.</p><p>Instead of one big loop, the workflow can look like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rUUb3oQtTZjHTn-QEO3jLw.png" /><figcaption><strong>Receive Request =&gt; Understand Issue =&gt;Search Knowledge Base =&gt;Decide</strong></figcaption></figure><p>From the decision, the agent can take different paths:</p><ul><li><strong>Answer found</strong> <strong>=&gt;</strong>Generate Response <strong>=&gt;</strong>Send</li><li><strong>More information needed</strong> <strong>=&gt;</strong>Ask User <strong>=&gt;</strong>Continue</li><li><strong>Issue is sensitive</strong> →<strong>=&gt;</strong>Human Review</li><li><strong>Search failed</strong> <strong>=&gt;</strong>Retry Search <strong>=&gt;</strong>Evaluate Again</li></ul><p>Here, each step is a <strong>node</strong>, while the arrows between them are <strong>edges</strong>.</p><p>The important part is that the agent doesn’t have to improvise the entire workflow every time. The graph defines the possible paths, while the agent handles the work inside those paths.</p><p>That is where graphs become useful. They give complex agent workflows a structure that is easier to reason about and control.</p><h3>III. [ Core Building Blocks ]</h3><p>A graph is basically made up of a few simple pieces. Once you understand these, most agent workflows become much easier to visualize.</p><h3>1. Nodes</h3><p>A <strong>node</strong> is one step in an agent’s workflow. It represents a specific task the agent needs to perform before moving forward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pXwa_ROXvT0OjxLUx_Uq0w.png" /><figcaption>Nodes</figcaption></figure><p>For example, a research workflow might have nodes like:</p><p><strong>Steps be like :</strong></p><ol><li><strong>Search</strong></li><li><strong>Read Sources</strong></li><li><strong>Evaluate</strong></li><li><strong>Summarize</strong></li></ol><p>Each node has a clear responsibility. Keeping these steps separate makes the workflow easier to understand, debug, and modify.</p><blockquote><strong>Example:</strong> A Search node finds information, while an Evaluate node checks whether that information is actually useful.</blockquote><h3>2. Edges</h3><p>An <strong>edge</strong> defines how the workflow moves from one node to another.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lQh0R0iYG9fMW0IaNtcTeg.png" /><figcaption>Edges — OverView</figcaption></figure><p>For example:</p><ol><li><strong>Search</strong></li><li><strong>Evaluate</strong></li><li><strong>Generate</strong></li></ol><p>Here, the edges tell the agent what step comes next.</p><p>Edges can also represent decisions:</p><p><strong>Evaluate =&gt; Good?</strong></p><p>Code Sample be like:</p><pre>def Evaluator(**kwargs):<br>  if Yes:<br>    Generate()<br>  else:<br>    Search(**kwargs)</pre><p>So, while <strong>nodes define the work</strong>, <strong>edges define the flow between that work</strong>.</p><h3>3. Routing</h3><p>Routing decides <strong>which path the agent should take next</strong>. Instead of always moving from one node to the next in a fixed order, the graph can look at the current result or state and choose a different path.</p><p>For example, after an agent evaluates its research:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Y02sffOYFP5-oS_M972vg.png" /><figcaption>Routing — Flow</figcaption></figure><p><strong>Evaluate defines Is the result good enough?</strong></p><p><strong>Sample code snippet:</strong></p><pre>def route(result):<br>    if result == &quot;good&quot;:<br>        return generate()<br>    elif result == &quot;uncertain&quot;:<br>        return human_review()<br>    return search_again()</pre><p>This is useful because real tasks rarely have only one possible outcome. Routing lets the workflow adapt to what happens at each step while keeping the overall process structured.</p><blockquote><strong>In simple terms, routing answers one question: “Where should the agent go next?”</strong></blockquote><h3>4. Parallel Execution</h3><p>Some tasks in an agent workflow are independent, so there is no need to run them one after another.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GpALoQAh4kn1vOg69sn8ZA.png" /><figcaption>Parallel Execution WorkFlow</figcaption></figure><h4><strong>For example</strong>,</h4><p>a research agent can <strong>search the web and check its internal knowledge base at the same time</strong>. Once both finish, their results can be combined and passed to the next node.</p><p><strong>Search Web + Check Documents → Combine Results → Evaluate</strong></p><p>The important part is the <strong>merge step</strong>. The workflow waits until both parallel tasks finish, combines their results into one shared state, and then passes that combined information to the next node.</p><p>The result is a workflow that can do more work without making every step wait for the previous one.</p><h3>5. Human Approval Gates</h3><p>Some actions are too important to let an agent execute without a human checking them first.</p><blockquote>A graph can include a <strong>human approval gate</strong> before sensitive or irreversible actions.</blockquote><p>For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q9IQHHde1tur4cqPLafYmg.png" /><figcaption>Human Approval Gates</figcaption></figure><p>The agent prepares the action, but the workflow pauses until a person approves it.</p><p>This is useful for things like <strong>sending emails, approving transactions, deleting data, or making important decisions</strong>.</p><blockquote><strong>The idea is simple: let the agent do the work, but keep a human in control when it matters.</strong></blockquote><h3><strong>IV. Recovery Paths</strong></h3><p>Agents will fail sometimes. A good graph should define <strong>what happens when a node fails</strong> instead of simply stopping the entire workflow.</p><p>For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cBDjOYqBRi2_HCByS2BZEA.png" /><figcaption>Recovery Paths WorkFlow</figcaption></figure><p>If retries keep failing, the workflow can take another path:</p><p><strong>Retry to Human Review</strong></p><p>Recovery paths can also use a different tool, return to an earlier node, or safely stop the workflow.</p><blockquote><strong>The idea is simple:</strong> <strong>failure should be another path in the graph, not the end of the graph.</strong></blockquote><h3><strong>V. Designing Stateful Workflows</strong></h3><p>In a real agent workflow, the agent needs to <strong>remember what has already happened</strong>.</p><h4><strong>For example,</strong></h4><p>A research agent might store the user’s question, sources it found, evaluation results, retry count, and decisions made along the way.</p><p><strong>This shared state moves between nodes and gets updated as the workflow progresses.</strong></p><p><strong>Workflows:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8qJU1CYrXPMwWrfz5WGy_Q.png" /><figcaption>Stateful Workflows</figcaption></figure><p>The second research step should know what was already searched. Otherwise, the agent might keep repeating the same work like it just discovered the internet.</p><p>Stateful workflows make this possible by keeping important information available across the graph.</p><blockquote><strong>The graph defines the path. The state carries the context.</strong></blockquote><p>Example code snippet ( [Code snippets are for Understanding Purpose Only]):</p><pre>state = {<br>    &quot;question&quot;: &quot;What are the benefits of renewable energy?&quot;,<br>    &quot;sources&quot;: [],<br>    &quot;evaluation&quot;: None,<br>    &quot;retry_count&quot;: 0,<br>    &quot;max_retries&quot;: 3,<br>    &quot;final_answer&quot;: None<br>}</pre><h3>1. Research Node</h3><pre>def research(state):<br>    results = search_web(state[&quot;question&quot;])<br>    state[&quot;sources&quot;].extend(results)<br>    return state</pre><h3>2. Evaluation Node</h3><pre>def evaluate(state):<br>    state[&quot;evaluation&quot;] = check_sources(<br>        state[&quot;question&quot;],<br>        state[&quot;sources&quot;]<br>    )<br>    return state</pre><h3>3. Routing / Retry Condition</h3><pre>def route(state):<br>    if state[&quot;evaluation&quot;][&quot;sufficient&quot;]:<br>        return &quot;generate&quot;<br>    if state[&quot;retry_count&quot;] &lt; state[&quot;max_retries&quot;]:<br>            return &quot;retry&quot;<br>        return &quot;stop&quot;</pre><h3>4. Retry Node</h3><pre>def retry(state):<br>    state[&quot;retry_count&quot;] += 1<br>    state[&quot;question&quot;] += &quot; economic and environmental benefits&quot;<br>    return state</pre><h3>5. Generate Answer Node</h3><pre>def generate_answer(state):<br>    state[&quot;final_answer&quot;] = generate(<br>        state[&quot;question&quot;],<br>        state[&quot;sources&quot;]<br>    )<br>    return state</pre><h3>6. Putting the Workflow Together</h3><pre>while True:<br>    state = research(state)<br>    state = evaluate(state)<br>    next_step = route(state)<br>    if next_step == &quot;generate&quot;:<br>        state = generate_answer(state)<br>        break<br>    if next_step == &quot;retry&quot;:<br>        state = retry(state)<br>        continue<br>    print(&quot;Could not find enough information.&quot;)<br>    break</pre><blockquote>The important part here isn’t the code itself. <strong>The same </strong><strong>state moves through every node, gets updated along the way, and determines what the graph does next.</strong></blockquote><h3>VI . When Graphs Make Sense</h3><p>Till so far we Understood How graphs works and Reasons to Move from simple flows to Graphs now ‘ <strong>When Graphs Make Sense?</strong>’</p><p><strong>So, should every AI agent be built as a graph?</strong></p><p><strong>Definitely not Dude.</strong></p><blockquote>If your agent only needs to <strong>do something, check the result, and try again</strong>, a simple loop is probably enough.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HYuPQd4iecZW4mdCa7DxbQ.png" /><figcaption>Loop Vs Graph</figcaption></figure><p>For example, a coding agent can simply:</p><pre>while not tests_pass:<br>    code = generate_code()<br>    tests_pass = run_tests(code)</pre><blockquote>There is no need for a graph here. The workflow is straightforward:</blockquote><ol><li><strong>generate</strong></li><li><strong>test</strong></li><li><strong>retry</strong>.</li></ol><p>Graphs start making sense when the workflow has <strong>multiple stages, decisions, branches, parallel tasks, human approvals, or recovery paths</strong>.</p><blockquote><strong><em>Use a graph when the workflow itself becomes complicated, not just because graphs look cool.</em></strong></blockquote><h3>VII. When Graphs Are Overkill</h3><p>Ofcourse Graphs are powerful, but that doesn’t mean every agent needs one.</p><p>If the workflow is just:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XnDs9l8zwX42J9ZF_taX-g.png" /><figcaption>Overkill Overview</figcaption></figure><p>adding multiple nodes, routing rules, and state management can make a simple problem harder than it needs to be.</p><p>For example, a basic summarization agent doesn’t need a graph just to summarize a document and check the output.More structure also means <strong>more code, more state to manage, and more things to debug</strong>.</p><blockquote><strong><em>If a loop explains the workflow clearly, stick with the loop. Don’t build a highway for a bicycle.</em></strong></blockquote><p>The goal of Graph Engineering isn’t to make every workflow complicated. It’s to add structure <strong>only when the complexity actually demands it</strong>.</p><h3>VIII. Signs You Actually Need a Graph</h3><p>A graph usually becomes useful when you start seeing things like:</p><ul><li><strong>Multiple branches:</strong> the next step depends on the result.</li><li><strong>Parallel work:</strong> several tasks need to run independently before merging.</li><li><strong>Human approval:</strong> the workflow needs to pause for a person.</li><li><strong>Recovery paths:</strong> failures need different retry or fallback routes.</li><li><strong>Long workflows:</strong> the process has too many steps to comfortably manage in one loop.</li><li><strong>Shared state:</strong> different steps need to read and update the same information.</li></ul><blockquote><strong><em>If you’re starting to draw more </em></strong><strong><em>if/else conditions than actual agent logic, that&#39;s probably a pretty good sign that your loop wants to become a graph.</em></strong></blockquote><h3>IX. Loop vs Graph</h3><p>Now, The easiest way to separate them is to a<strong>sk your self</strong>:</p><p><strong>“Am I repeating a step, or am I navigating between different steps?”</strong></p><p>A <strong>loop</strong> is mainly about repetition:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rpKbynh2Gah6W-_uM8vLXQ.png" /><figcaption>Loop vs Graphs — Clear View</figcaption></figure><p>So, a loop helps an agent <strong>keep trying</strong>. A graph helps an agent <strong>decide where to go next</strong>.</p><p>And in real systems, they can work together. A graph node might contain a loop for retries or self-correction. They are not competing architectures. <strong>The graph provides the workflow structure, while loops handle iteration inside that workflow.</strong></p><h3>Addon : [ Best Practices ]</h3><p>A good graph should make a workflow <strong>easier to understand, not harder</strong>.</p><p>Here are few things help:</p><ul><li><strong>Keep nodes focused.</strong> One node should ideally handle one clear responsibility.</li><li><strong>Make routing explicit.</strong> It should be obvious why the workflow moves from one node to another.</li><li><strong>Keep state clean.</strong> Store only the information that the workflow actually needs.</li><li><strong>Design for failure.</strong> Retries, fallbacks, and human review should have defined paths.</li><li><strong>Don’t overbuild.</strong> If a simple loop can solve the problem, use the loop.</li></ul><p>The goal is not to create the most complicated graph. It’s to create the <strong>simplest graph that can reliably handle the workflow</strong>.</p><h3>Final Thoughts :</h3><p>Graphs give AI agents something loops alone cannot: <strong>structure when the workflow gets complicated.</strong></p><p>They help agents navigate decisions, branches, parallel tasks, approvals, and failures without turning everything into one giant loop.</p><blockquote><strong>A great harness enables an agent to work. A great loop enables it to improve. A great graph enables it to navigate complexity.</strong></blockquote><p><em>Enjoyed this article? Give it a few 👏 claps, leave your thoughts in the comments, and follow for more in-depth guides on AI engineering, LLMs, and modern agent architectures.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7676098c04e6" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/agent-graph-engineering-a-complete-blog-to-production-ai-agent-architecture-part-3-7676098c04e6">Agent Graph Engineering : A Complete Blog to Production AI Agent Architecture — Part-3</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agent Loop Engineering : A Complete Blog to Production AI Agent Architecture — Part-2]]></title>
            <link>https://ai.plainenglish.io/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-6fbd080ab85f?source=rss-d3969283167e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6fbd080ab85f</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Abhilash Puli]]></dc:creator>
            <pubDate>Sun, 09 Aug 2026 14:33:33 GMT</pubDate>
            <atom:updated>2026-08-09T14:50:00.312Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong><em>AI Agents Aren’t Just Models Anymore!</em></strong></blockquote><p>Written by Abhilash Puli, exploring the future of AI through, engineering, and in-depth technical writing on machine learning, LLMs, and AI agents.</p><p>Welcome back to Part 2. If you’re new to the series, I’d recommend reading Part 1 first, where we covered Agent Harness Engineering.</p><h3>Part 2: Loop Engineering</h3><h3>From Giving Agents Tools to Making Them Improve</h3><p>In Part 1, we looked at <strong>Agent Harness Engineering (If you missed Check out now </strong><a href="https://abhilashpuli.medium.com/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-882e0c2e083c"><strong>Click me</strong></a><strong>.) </strong>the environment that gives an AI agent access to context, tools, memory, execution controls, safety, and observability.</p><h4>But there is a problem.</h4><blockquote><strong>Having the ability to act doesn’t mean the agent knows when it has done the job correctly.</strong></blockquote><p>An agent can write code, run a command, call an API, or generate a report. What happens when the result is wrong?</p><p>It needs to <strong>check, learn, adjust, and try again and That is where Loop Engineering comes in</strong>.</p><blockquote><strong><em>The harness gives an agent the ability to work. The loop gives it the ability to improve.</em></strong></blockquote><h3>I. What Is Loop Engineering?</h3><p>Loop Engineering is the practice of designing the <strong>feedback and iteration cycle</strong> that determines how an AI agent performs a task, evaluates its result, responds to failure, and decides what to do next.</p><p>A basic agent might work like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VAfLrkwpVJ88dVQq9iCW5g.png" /><figcaption>Loop Eng- Flow</figcaption></figure><p><strong>Flow: Prompt =&gt; Response =&gt; Done</strong></p><p>A production agent is often closer to:</p><p><strong>Flow : Task =&gt;Act → Observe =&gt; Evaluate =&gt; Improve =&gt; Act Again</strong></p><p>The important part isn’t simply repeating the model call. It’s having a <strong>reason to continue</strong> and <strong>evidence that tells the agent what to change</strong>.</p><h3>II. Why Every AI Agent Needs Loops</h3><p>Let’s be honest for a while even very capable models don’t magically get everything right on the first attempt <strong>if you have used any AI IDE’s you can observe models Just dump code that even won’t work in most of cases</strong>.</p><p>1. Code can fail.<br> 2. APIs can return unexpected results.<br> 3. Research can have missing evidence.<br> 4. Generated content can violate requirements.</p><p>so, to over come these demerits we need loop’s it can be be defined as “A loop gives the agent a chance to deal with reality instead of confidently declaring victory after attempt one.”</p><h4>1.Basic Agent Loop</h4><pre>while not goal_reached:<br>    result = agent.act()<br>    evidence = evaluate(result)<br>  if evidence.success:<br>     break<br>  agent.learn(evidence.feedback)</pre><p>Confused with code snippet?</p><p>Here’s Explanation Buddy:</p><p><em>The important part isn’t the </em><em>while loop itself. The agent acts, receives evidence, updates its state, and decides whether another iteration is necessary.</em></p><h3>Example: Coding Agent</h3><p>Imagine asking an agent:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SwySp1peTsxxOIRCt1qdDQ.png" /><figcaption>Coding Loop</figcaption></figure><blockquote><em>“</em><strong><em>Build a REST API for user authentication.</em></strong><em>”</em></blockquote><p>A useful agent doesn’t just generate the code and say <em>“Done.”</em></p><p>It can:</p><ol><li>Write the implementation.</li><li>Run the tests.</li><li>Observe the failure.</li><li>Identify the problem.</li><li>Modify the code.</li><li>Run the tests again.</li><li>Stop when the required checks pass.</li></ol><p>That’s the difference between <strong>generation</strong> and <strong>iterative execution</strong>.</p><h3>III. Anatomy of a Well-Designed Loop</h3><p>A reliable loop isn’t just a <strong>while True:</strong> with an LLM inside it. That&#39;s how you accidentally create a very expensive screensaver.</p><h4>A good loop has a few clearly defined pieces:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jrLTR7j-QFNzrRraD6eYTA.png" /><figcaption>Well-Designed Loop Anatomy!</figcaption></figure><h3>1. Trigger</h3><p>What starts the loop?</p><p>Examples include a user request, failed test, new data, scheduled task, or evaluator feedback.</p><h3>2. Goal</h3><p>What exactly is the agent trying to achieve?</p><p>For example:</p><pre>&quot;All unit tests must pass.&quot;</pre><p>A measurable goal is much better than:</p><pre>&quot;Make it better.&quot;</pre><h3>3. State</h3><p>What does the agent know about previous attempts?</p><p>State can include previous outputs, errors, tool results, progress, and intermediate artifacts.</p><blockquote><strong><em>Without state, the agent may happily make the same mistake five times with increasing confidence.</em></strong></blockquote><h3>4. Action Policy</h3><h4>What is the agent allowed to do?</h4><p>This could include calling tools, modifying files, querying APIs, running code, or asking for human approval.</p><h3>5. Evidence</h3><p>How does the system know whether the result is actually good?</p><p><strong>Examples:</strong></p><ul><li>Tests passing</li><li>Schema validation</li><li>Evaluation scores</li><li>API responses</li><li>Citation checks</li><li>Human approval</li></ul><p>This is one of the most important ideas in loop engineering:</p><blockquote><strong><em>Confidence is not evidence.</em></strong></blockquote><h3>6. Feedback</h3><p><strong>If something went wrong, what tells the agent <em>why</em>?</strong></p><p>Instead of:</p><pre>&quot;Try again.&quot;</pre><p>Give it:</p><pre>&quot;Three tests failed because the API returns a string where an integer is expected.&quot;</pre><p>Now the next iteration has something useful to work with.</p><h3>7.Stop Rule</h3><p>When does the loop end?</p><p>Possible conditions include:</p><ul><li>Goal achieved</li><li>Maximum retries reached</li><li>Time limit exceeded</li><li>Budget exhausted</li><li>Human approval required</li><li>Unrecoverable failure</li></ul><p>A loop without a stop condition isn’t autonomous.</p><p>It’s just <strong>an invoice waiting to happen</strong>.</p><h3>IV. Evidence-Driven vs Confidence-Driven Loops</h3><p>This distinction is one of the most important ideas in Loop Engineering.</p><p>A <strong>confidence-driven loop</strong> stops because the model believes its answer is correct.</p><p>An <strong>evidence-driven loop</strong> stops because something outside the model provides evidence that the answer is correct.</p><p>For example:</p><p><strong>Confidence-driven</strong></p><pre>&quot;The code looks correct.&quot;</pre><p><strong>Evidence-driven</strong></p><pre>&quot;The tests pass, the type checker passes, and the API returns the expected response.&quot;</pre><p>The second approach is much more reliable because the agent isn’t grading its own homework and giving itself an A.</p><blockquote><strong><em>Confidence can guide a decision. Evidence should validate it.</em></strong></blockquote><h3>V. Common Loop Patterns</h3><p>Different tasks need different kinds of loops. Four patterns appear frequently in agent systems.</p><h3>1. Retry Loops</h3><p>The simplest pattern.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gjuUDYMD_ipu4PAEvndbLw.png" /><figcaption>Retry Loops</figcaption></figure><p>Useful for temporary failures such as network errors, rate limits, or unavailable services.</p><p>The important part is having a retry limit and backoff strategy. Otherwise, your agent may spend forever knocking on a server’s door.</p><h3>2. Reflection Loops</h3><p>Here, the agent reviews its own output and identifies possible weaknesses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dTBc4IVgOcTtgDxUoPs6wQ.png" /><figcaption>Reflection Loops</figcaption></figure><p><strong>Flow: Generate → Reflect → Improve → Repeat</strong></p><ul><li>Useful for writing, reasoning, planning, and other tasks where quality can improve through critique.</li></ul><p>Reflection can help, but self-criticism alone isn’t proof of correctness. Pair it with external validation when possible.</p><h3>3. Evaluation Loops</h3><p>An evaluator checks the agent’s output against defined criteria.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wjWxZ8H19jofKaO8cQgzYQ.png" /><figcaption>Evaluation Loops</figcaption></figure><p>The evaluator could be:</p><ul><li>A rule-based validator</li><li>Another model</li><li>A benchmark</li><li>A test suite</li><li>A human reviewer</li></ul><p>This is especially useful when you can define what “good” actually means.</p><h3>4. Self-Correction Loops</h3><p>The agent acts, observes what happened, and changes its approach based on the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0-n9viLHHnybo7N61DFZLA.png" /><figcaption>Self-Correction Loops</figcaption></figure><ul><li>Coding agents are a great example.</li><li>The agent writes code, runs it, encounters an error, analyzes the error, modifies the code, and runs it again.</li></ul><p>This is more powerful than simply retrying because the <strong>next action is informed by the previous failure</strong>.</p><pre>def solve_problem(problem):<br>    for _ in range(5):<br>        solution = agent.generate(problem)<br><br>        result = run_tests(solution)<br><br>        if result.passed:<br>            return solution<br><br>        problem = agent.analyze_failure(<br>            problem,<br>            result.errors<br>        )<br><br>    raise RuntimeError(&quot;Unable to solve the problem&quot;)</pre><h3>VI. Real-World Examples</h3><h3>1. Coding Agents</h3><p>A coding agent can:</p><p><strong>Flow be like : Write → Test → Observe Failure → Fix → Test Again</strong></p><h3>Example: Coding Agent</h3><p>Imagine asking an agent:</p><p>The test results provide objective feedback, making this a strong example of evidence-driven iteration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SwySp1peTsxxOIRCt1qdDQ.png" /><figcaption>Coding Agent</figcaption></figure><h4>Code Snippet: Evidence-Driven Coding Loop</h4><pre>for attempt in range(MAX_RETRIES):<br>    code = agent.generate()<br>    result = run_tests(code)<br>    if result.passed:<br>        return code<br>    agent.update(result.errors)<br>raise RuntimeError(&quot;Could not produce a valid solution&quot;)</pre><p>This directly demonstrates:</p><ul><li><strong>Generate =&gt; Test =&gt; Failure =&gt; Fix =&gt; Test</strong></li></ul><h3>2. Research Agents</h3><p>A research agent can:</p><p>An evidence-driven research loop: the agent continues gathering information until the important knowledge gaps are resolved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-c395vR6IjIXX7UMfBjZOg.png" /><figcaption>Research Agents</figcaption></figure><h4>Code Snippet:</h4><pre>MAX_ITERATIONS=6<br><br>def research_agent(topic):<br>    state = {<br>        &quot;sources&quot;: [],<br>        &quot;gaps&quot;: [topic]<br>    }<br><br>    for _ in range(MAX_ITERATIONS):<br><br>        # 1.Search<br>        results=search_web(state[&quot;gaps&quot;])<br><br>        # 2.Read<br>        documents=read_sources(results)<br><br>        # 3.Evaluate<br>        useful_sources=evaluate_sources(documents)<br>        state[&quot;sources&quot;].extend(useful_sources)<br><br>        # 4.Identify gaps<br>        state[&quot;gaps&quot;]=find_knowledge_gaps(<br>            topic,<br>            state[&quot;sources&quot;]<br>        )<br><br>        # 5.Stop when enough evidence exists<br>        if not state[&quot;gaps&quot;]:<br>            break<br><br>    # 6.Synthesize<br>    return synthesize_report(<br>        topic,<br>        state[&quot;sources&quot;]<br>    )</pre><p><strong>Flow: Search → Read → Evaluate Sources → Identify Gaps → Search Again → Synthesize</strong></p><ul><li>If the available evidence is weak, the loop continues rather than immediately producing a confident-looking report.</li></ul><h3>3. Content Generation</h3><p>A content agent might:</p><p>The agent doesn’t treat its first draft as the final answer. it uses evaluation feedback to progressively improve it until the requirements are satisfied.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_p0Eo4YX5F-EqtKP_tNL1A.png" /><figcaption>Content Generation</figcaption></figure><h4>Code Snippet</h4><pre>def content_agent(topic, requirements):<br>    draft = generate_draft(topic)<br>    for _ in range(3):<br>        evaluation = evaluate(<br>            draft,<br>            requirements<br>        )<br>        if evaluation.passes:<br>            return draft<br><br>        draft = revise(<br>            draft,<br>            evaluation.feedback<br>        )<br>    raise RuntimeError(&quot;Could not satisfy requirements!&quot;)</pre><ul><li><strong>Flow: Draft → Evaluate → Revise → Check Requirements → Finalize</strong></li></ul><p>The evaluation could check structure, factual requirements, tone, or formatting.</p><p>Across all three examples, the pattern is the same: <strong>do something, inspect the result, learn from it, and decide what happens next.</strong></p><h3>VII. Common Mistakes</h3><h3>1. Infinite Loops</h3><p>No maximum iterations, timeout, or budget.</p><p><strong>You can fix out by:</strong> Always define termination conditions.</p><h3>2. Blind Retries</h3><p>Repeating the exact same action after failure.</p><p><strong>You can fix out by :</strong> Feed the failure back into the next decision.</p><h3>3. No External Evidence</h3><p>Stopping because the model says everything looks good.</p><p><strong>You can fix out by :</strong> Use tests, validators, metrics, or human approval.</p><h3>4. Weak Feedback</h3><p>Giving the agent vague feedback such as “try again.”</p><p><strong>You can fix out by :</strong> Provide specific, actionable failure information.</p><h3>5. Losing State</h3><p>Each iteration forgets what happened previously.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6fbd080ab85f" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-6fbd080ab85f">Agent Loop Engineering : A Complete Blog to Production AI Agent Architecture — Part-2</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agent Harness Engineering : A Complete Blog to Production AI Agent Architecture — Part-1]]></title>
            <link>https://ai.plainenglish.io/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-882e0c2e083c?source=rss-d3969283167e------2</link>
            <guid isPermaLink="false">https://medium.com/p/882e0c2e083c</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[graph-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <dc:creator><![CDATA[Abhilash Puli]]></dc:creator>
            <pubDate>Sun, 02 Aug 2026 03:18:33 GMT</pubDate>
            <atom:updated>2026-08-09T14:46:01.058Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>AI Agents Aren’t Just Models Anymore!</blockquote><p>Written by Abhilash Puli, exploring the future of AI through, engineering, and in-depth technical writing on machine learning, LLMs, and AI agents.</p><p>Large Language Models (LLMs) are the brain of an AI agent but not the complete package anymore. The modern AI agent needs to use tools, <em>API calls, databases, browser interfaces, and file manipulation</em>. They need to remember information between sessions and take actions in several steps. To make a robust AI agent in production, engineering surrounding the model becomes the key. This is where harness engineering, loop engineering, and graph engineering help out.</p><h4>Why Everyone Keeps Mixing These Three Concepts?</h4><p>With the increase in capabilities of AI agents, the phrases “<strong><em>harness engineering</em></strong>,” “<strong><em>loop engineering</em></strong>,” and “<strong><em>graph engineering</em></strong>” have become synonymous. However, although these three have a role to play in constructing robust agent systems, they each refer to a different element in the system’s architecture. It would therefore be wrong to confuse the three.</p><blockquote>Guess What You’ll Learn in This Blog…?<br>Don’t try hallucinate!<br>Leme clear things first.</blockquote><p><strong><em>By the end of this Blog, you’ll understand:</em></strong></p><blockquote>I. What agent harness engineering, loop engineering, and graph engineering actually mean.<br>II. The role each layer plays in a production AI agent.<br>III. How these layers work together to build reliable, scalable agent systems.<br>IV. When to use each approach and how to identify which layer needs attention when something goes wrong.</blockquote><h3>The 30-Second Explanation</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4UtEVyT_SjScyPlicCqCkg.png" /><figcaption>The Three Layers at a Glance</figcaption></figure><h3>The Three Layers at a Glance</h3><h3>I. Harness Engineering: The Environment</h3><p>Harness engineering builds the operating environment around the model. It provides the tools, memory, permissions, context, and execution controls an agent needs to perform real-world tasks safely and reliably.</p><h3>II. Loop Engineering: The Feedback Cycle</h3><p>Loop engineering defines how an agent improves its work through repeated cycles of execution, evaluation, and refinement. It focuses on evidence-driven retries, validation, and clear stop conditions rather than repeated prompting.</p><h3>III. Graph Engineering: The Workflow</h3><p>Graph engineering makes the execution path explicit. It defines how tasks move between different steps, how decisions are made, where branching occurs, when work can run in parallel, and where human approvals or recovery paths fit into the workflow.</p><h4>Why These Layers Matter?</h4><p>As these AI systems progress beyond simple chatbots into more sophisticated autonomous agents, developing such applications needs significantly more than selecting a robust language model. The agent needs to be able to interface with other software systems, reason step-by-step, handle failures, keep contextual information, and perform actions in the real world.</p><p>All this is not the job of the language model but rather of the engineering system that surrounds it. This is why it is critical to understand these layers to develop robust AI agents.</p><h3>From [Chatbots] to [Production AI Systems]</h3><p>Early AI applications followed a simple pattern: a user asked a question, the model generated a response, and the interaction ended. While this worked well for conversational assistants, today’s AI systems are expected to do far more.</p><p><strong>A production AI agent might:</strong></p><ul><li>Search the web for relevant information.</li><li>Read and update files.</li><li>Query databases.</li><li>Call external APIs.</li><li>Write and execute code.</li><li>Coordinate with specialized agents.</li><li>Wait for human approval before performing critical actions.</li></ul><p>For instance, an <strong>AI coding assistant such as Claude Code, OpenAI Codex, and Cursor does not just generate code</strong>. <strong>The tool moves through the entire process of browsing your project, editing files, executing terminal commands, conducting tests, analyzing failed processes,</strong> and going back to the beginning of the cycle until the process ends.</p><p>At this point, you’re no longer building a chatbot you’re engineering an intelligent system.</p><p>Some of us would say that “Prompt Engineering” Enough and if we some how feed a perfect Structured Instructed Prompt we can end up with good result. Okay it’s a good solution to guide so but we have to think of project state and some of failures too. Let’s dive in then…</p><h3>Why Prompt Engineering Isn’t Enough</h3><p>Prompt engineering is an important part of working with language models, but it solves only one piece of the problem: <strong>guiding the model during a single interaction</strong>.</p><p>Production AI systems face challenges that prompts alone cannot solve:</p><ul><li>How should the agent access external tools?</li><li>How is conversation or project state preserved across sessions?</li><li>What happens if an API call fails?</li><li>How does the system verify whether the output is actually correct?</li><li>When should the agent retry, escalate, or stop?</li><li>How are permissions and security enforced?</li></ul><blockquote>Take for example a scenario where you task an AI agent to prepare a financial report. While the prompt might enable the creation of a very credible financial report, it will not ensure the following that the information used is up-to-date, the calculations made are correct, and that corporate procedures have been adhered to.</blockquote><p>Prompt engineering improves responses But it isnt improves Systems so we shifted anf focused on <strong>Agent engineering which improves systems.</strong></p><h3>The Rise of Agent Engineering</h3><p>The industry is moving rapidly from model-based development to system-based development. With organizations utilizing AI in software development, customer support, research, health care, financial institutions, and enterprise automation, it is not only about the efficiency of the model anymore, but the development of the system around the model.</p><p>This shift has led to the emergence of frameworks and standards such as:</p><ul><li><strong>LangGraph</strong> for stateful workflow orchestration.</li><li><strong>Model Context Protocol (MCP)</strong> for standardized tool integration.</li><li><strong>OpenAI Agents SDK</strong> for building production-ready agents.</li><li><strong>CrewAI</strong> for multi-agent collaboration.</li><li><strong>Google Agent Development Kit (ADK)</strong> for enterprise agent workflows.</li><li><strong>AutoGen</strong> for agent-to-agent communication and orchestration.</li></ul><p>Although these frameworks differ in implementation, they all address the same fundamental challenge: <strong>how to transform a language model into a dependable production system</strong>.</p><p>This is where the three layers explored in this Blog become essential:</p><ul><li><strong>Harness Engineering</strong> provides the capabilities and operating environment.</li><li><strong>Loop Engineering</strong> ensures iterative improvement through evidence-driven feedback.</li><li><strong>Graph Engineering</strong> organizes complex workflows with explicit control over execution.</li></ul><p>Together, these layers form the architectural foundation of modern AI agents.</p><blockquote>A helpful mind-set: Stop viewing the AI agent as a model with prompts. Start viewing it as a software system based on a model. The model produces the intelligence; the architecture around the model defines whether the intelligence is robust, secure, and ready for production.</blockquote><h3>Layer 1: Agent Harness Engineering</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o2_il14r9GXm1A1NgcUjPA.png" /><figcaption>Layer — 1 [ AHE ]</figcaption></figure><h3>What Is Agent Harness Engineering?</h3><p>Agent Harness Engineering is the process of building the <strong>operating environment</strong> around an AI model. While the language model provides reasoning and decision-making capabilities, the harness supplies everything the model needs to interact with the real world.</p><p>Think of the harness as the software infrastructure that transforms a language model into an AI agent. It manages the agent’s context, tools, memory, permissions, execution, and monitoring, allowing it to perform tasks beyond simple text generation.</p><p>Without a harness, an LLM is simply answering prompts. With a well-designed harness, it can search the web, manipulate files, execute code, interact with APIs, collaborate with other agents, and safely operate in production.</p><h3>Why the Harness Exists</h3><p>A language model has impressive reasoning capabilities, but it also has important limitations. By itself, it can’t:</p><ul><li>Access external tools or APIs.</li><li>Remember previous work across sessions.</li><li>Execute shell commands.</li><li>Read or modify project files.</li><li>Enforce security policies.</li><li>Recover after interruptions.</li><li>Track execution history.</li><li>Control resource usage or budgets.</li></ul><p>The harness fills these gaps by providing a controlled execution environment. It acts as the bridge between the model’s intelligence and the external systems it needs to interact with.</p><p><strong>Don’t Get messy up your Neural Systems To be short it mean:</strong></p><blockquote><strong><em>The model decides what to do. The harness makes it possible to do it.</em></strong></blockquote><h3>Core Components of a Production Harness</h3><p>A production-grade harness typically consists of several interconnected components that work together to support reliable agent execution.</p><h3>1. Context Injection</h3><p>Every decision an agent makes depends on the information it receives. Context injection determines what <strong>information is presented to the model before every reasoning step</strong>.</p><p>Typical context includes:</p><ul><li>System instructions</li><li>User requests</li><li>Retrieved knowledge (RAG)</li><li>Conversation history</li><li>Project state</li><li>Organization policies</li><li>Long-term memory</li><li>Runtime variables</li></ul><p>Providing relevant context improves reasoning, while excessive or noisy context often reduces performance.</p><p><strong>Example</strong></p><p>A coding agent may receive below things instead of only the user’s prompt.:</p><ul><li>Repository structure</li><li>Coding standards</li><li>Previous edits</li><li>Current bug report</li><li>Relevant documentation as we can observe in many code repos Claude.md file documentation exists.</li></ul><h3>2. Action Surfaces</h3><p>Reasoning alone cannot complete real-world tasks. The agent requires mechanisms to interact with external systems.</p><p>Common action surfaces include:</p><ul><li>REST APIs</li><li>Browser automation</li><li>Shell commands</li><li>File systems</li><li>Databases</li><li>Python execution</li><li>MCP tools</li></ul><p><strong>Example</strong></p><p>Instead of describing how to retrieve weather data, the harness allows the agent to directly call a weather API and return live results.</p><h3>3. Persistence</h3><p>Production agents often execute workflows that span minutes, hours, or even days. Persistence ensures important information survives beyond a single model invocation.</p><p>Persistent data may include:</p><ul><li>Files</li><li>Checkpoints</li><li>Session state</li><li>Memory</li><li>Progress logs</li><li>Vector databases</li><li>Git history</li></ul><p>Without persistence, every interaction effectively starts from scratch.</p><p><strong>Example</strong></p><p>An autonomous research agent collecting papers over several hours should resume exactly where it stopped after a system restart rather than repeating the entire process.</p><h3>4. Execution Control</h3><p>Execution control governs how the agent performs work and manages available resources.</p><p>Typical controls include:</p><ul><li>Retry policies</li><li>Timeouts</li><li>Cost budgets</li><li>Token limits</li><li>Model selection</li><li>Task scheduling</li><li>Parallel execution</li><li>Human approval gates ( <strong>If you have used tools like Antigravity by Google, MINMAX-code,Cursor typical type of Dev tool IDE’s You know this gates well</strong>).</li><li>Sub-agent spawning</li></ul><p>These mechanisms improve reliability while preventing runaway execution and unnecessary costs.</p><h3>5. Safety &amp; Governance</h3><p>Production AI systems often interact with sensitive information and perform high-impact actions. Safety mechanisms ensure the agent operates within defined boundaries.</p><p>A secure harness commonly implements:</p><ul><li>Least-privilege access</li><li>Permission management</li><li>Secret handling</li><li>Tool allowlists</li><li>Sandboxed execution</li><li>Audit logs</li><li>Human approvals</li><li>Policy enforcement</li></ul><p>Security should be designed into the harness rather than added after deployment.</p><h3>6. Observability</h3><p>Reliable systems are observable systems. Observability provides visibility into every stage of the agent’s execution, making debugging and evaluation significantly easier.</p><p>Typical telemetry includes:</p><ul><li>Execution traces</li><li>Tool calls</li><li>Model responses</li><li>State transitions</li><li>Latency</li><li>Token usage</li><li>Costs</li><li>Evaluation results</li></ul><p>Without observability, identifying failures often becomes guesswork.</p><h3>So Far You Learnt Just The Components but now the Main Question to be get into our Neural sys is How a Harness Works?</h3><p>A harness coordinates the entire lifecycle of an agent execution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c1Nn5lMdMDzD5dVInePBGg.png" /><figcaption>How it Works Sketch</figcaption></figure><p><strong>A typical flow looks like this:</strong></p><ol><li>Receive the user’s request.</li><li>Gather relevant context and memory.</li><li>Select the appropriate model.</li><li>Provide access to approved tools.</li><li>Execute tool calls as needed.</li><li>Store progress and intermediate state.</li><li>Monitor execution and enforce limits.</li><li>Return the final response while recording logs and traces.</li></ol><p>Throughout this process, the harness continuously manages resources, permissions, and execution state without requiring the language model to handle these responsibilities directly.Real-World Harness Examples</p><h3>Real-World Harness Examples</h3><p>Many modern AI platforms implement sophisticated harnesses behind the scenes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*9XeAU-wNEy9FAsdozdGirw.png" /><figcaption>Harness Examples</figcaption></figure><p>Although these platforms expose different interfaces, they all rely on a harness to bridge the gap between model reasoning and real-world execution.</p><p>Eventhough we have many Advantages there are still some Disadvantages that have to be faced still:</p><h3>Cases Where Harness Engineering Fails</h3><p>Many agent failures are not caused by the language model — they are caused by weaknesses in the harness.</p><p>Common symptoms include:</p><ul><li>Missing tool access.</li><li>Lost conversation or project state.</li><li>Incorrect permissions.</li><li>Excessive context leading to poor reasoning.</li><li>API failures without recovery.</li><li>No execution logging.</li><li>Uncontrolled costs.</li><li>Inconsistent behavior across environments.</li></ul><p>Before replacing the model or rewriting prompts, it is often worth examining whether the harness itself is responsible for the failure.</p><blockquote><strong>But How to Overcome those Failures?</strong></blockquote><h3>Here are some of the Best Practices That Dev’s follow :</h3><p>A well-designed harness should follow a few core principles:</p><ul><li>Provide only the tools the agent actually needs.</li><li>Keep context concise, relevant, and structured.</li><li>Persist important state between executions.</li><li>Design for interruption and recovery.</li><li>Monitor every significant action through logs and traces.</li><li>Implement clear execution limits for time, cost, and retries.</li></ul><h3>Key Takeaways</h3><p>Before wrapping up, here’s what to remember:</p><ul><li>A language model alone is not an AI agent.</li><li>The harness provides the environment that enables an agent to operate.</li><li>Context, tools, memory, execution control, safety, and observability are the foundation of reliable AI systems.</li><li>Many production failures originate from weaknesses in the harness rather than the model itself.</li><li>A well-designed harness makes agents reliable, secure, and scalable.</li></ul><h3>Final Thoughts</h3><p>The effectiveness of the AI agent does not only depend on the intelligence of its language model. The two groups can employ the same language model but still get very different results since what truly makes the difference is the environment around the model.<br>The effective harness enables the transformation of the LLM into an agent able to interact with the real world.</p><p>However, even the best harness cannot guarantee high-quality outcomes on its own. An agent also needs a mechanism to evaluate its work, recover from mistakes, and improve through evidence-driven iteration.</p><p>That’s where <strong>Loop Engineering</strong> comes in.</p><h3>What’s Next in This Series?</h3><h3>Part 2: Loop Engineering “Designing Reliable Feedback Cycles for AI Agents”</h3><p>In the next article, we’ll explore:</p><ul><li>Why one-shot agents often fail.</li><li>How production AI systems improve through iterative feedback.</li><li>Evidence-driven loops vs. confidence-driven loops.</li><li>Common loop patterns used by coding agents, research agents, and autonomous systems.</li><li>How leading AI frameworks implement reliable execution cycles.</li></ul><blockquote><strong><em>A great harness enables an agent to work. A great loop enables it to improve.</em></strong></blockquote><p>See you in <strong>Part 2.</strong></p><p><em>Enjoyed this article? Give it a few 👏 claps, leave your thoughts in the comments, and follow for more in-depth guides on AI engineering, LLMs, and modern agent architectures.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=882e0c2e083c" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/agent-harness-engineering-vs-loop-engineering-vs-graph-engineering-a-complete-blog-to-production-882e0c2e083c">Agent Harness Engineering : A Complete Blog to Production AI Agent Architecture — Part-1</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Gradient Descent: The Math Behind How Modern AI Systems Learn]]></title>
            <link>https://ai.plainenglish.io/demystifying-gradient-descent-the-math-behind-how-modern-ai-systems-learn-d0d844cde6ac?source=rss-d3969283167e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0d844cde6ac</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Abhilash Puli]]></dc:creator>
            <pubDate>Fri, 29 May 2026 15:05:24 GMT</pubDate>
            <atom:updated>2026-05-29T15:10:58.377Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Math Behind Gradient Descent: A Step by Step Numeric Guide</h3><p><em>Written by Abhilash, an AIML student passionate about machine learning, deep learning, and making complex AI concepts easy to understand.</em></p><p><strong>Demystifying the core engine of machine learning and deep learning optimization.</strong></p><p>In this blog, we will learn about the math behind gradient descent with a step by step numeric example.</p><p>Gradient descent is the most fundamental optimization algorithm used to train machine learning and deep learning models. Understanding the math behind it gives us a clear picture of how models actually learn. Whether you are adjusting weights for a simple linear regression or training complex architectures like Convolutional Neural Networks and attention based models in PyTorch, gradient descent is the optimization engine powering the entire process under the hood.</p><p>Do not worry if math is not your strongest domain. We will go through each concept step by step, using clear logic and straightforward numbers so that everything is highly accessible and easy to understand.</p><h3>What We Will Cover</h3><p>To provide a complete understanding from foundational theory to practical application, we will explore the following topics in sequence:</p><ol><li><strong>What is a Loss Function?</strong></li><li><strong>Understanding how we measure the error of our model predictions.</strong></li><li><strong>What is Gradient Descent?</strong></li><li><strong>A high level conceptual overview of the optimization algorithm.</strong></li><li><strong>The Intuition Behind Gradient Descent</strong></li><li><strong>Visualizing the optimization process like walking down a mountain to find the lowest valley.</strong></li><li><strong>The Math Behind Gradient Descent</strong></li><li><strong>Breaking down the core formulas, derivatives, and partial derivatives.</strong></li><li><strong>Step by Step Numeric Example</strong></li><li><strong>Plugging real numbers into our mathematical formulas to see the algorithm in action.</strong></li><li><strong>Gradient Descent with Multiple Parameters</strong></li><li><strong>Scaling the mathematics for models dealing with more than one variable.</strong></li><li><strong>The Role of Learning Rate</strong></li><li><strong>Analyzing how step size impacts the speed, convergence, and stability of model training.</strong></li><li><strong>Types of Gradient Descent</strong></li><li><strong>Exploring the architectural differences between Batch, Stochastic, and Mini Batch gradient descent.</strong></li><li><strong>Gradient Descent in Python</strong></li><li><strong>Translating our mathematical formulas into clean, functional code.</strong></li><li><strong>Visualizing the Descent</strong></li></ol><blockquote>Using an interactive Gradient Descent Visualizer to observe how the loss decreases and the model converges during training.<strong> click on this link to Visualizer [ </strong><a href="https://gradient-descent-visualizer.pages.dev/"><strong>Gradient Descent Visualizer</strong></a><strong> ]</strong></blockquote><h3>I. The Big Picture</h3><p>Before diving into the mathematics, let’s first understand the bigger picture.</p><p>Machine learning models learn by adjusting their parameters, often called weights, so that their predictions become closer to the actual values. The goal is to minimize the difference between what the model predicts and what the data says is correct.</p><blockquote>This is where gradient descent comes in.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DPYRpf1frRUT56iIp9ecng.png" /><figcaption>Analogy- Reference 1 — CC Included</figcaption></figure><p>Gradient descent is an optimization algorithm that guides the model toward better parameter values. It repeatedly updates the weights in a way that reduces the prediction error. With each update, the model takes a small step toward a better solution until the error becomes as small as possible.</p><p>Think of it as a traveler trying to reach the lowest point in a valley while standing in thick fog. Since the traveler cannot see the entire landscape, they look at the slope around them and take a small step downhill. By continuously moving in the downhill direction, they eventually reach the bottom of the valley.</p><p><strong>In simple terms:</strong></p><p><strong>Gradient Descent = = A method for gradually moving down the error curve until the minimum error is reached.</strong></p><p>This simple idea forms the foundation of how most machine learning and deep learning models learn from data.</p><h3>II. What is a Loss Function?</h3><p>Before exploring gradient descent, we must define the <strong>loss function</strong>. Simply put, a loss function measures how wrong a model is by calculating the distance between its predictions and the actual real world values.</p><p>Whether you are building a model to predict real estate prices or calculating facial embedding distances for a face recognition attendance management system, you need a reliable mathematical way to evaluate accuracy.</p><p>Let us look at a numerical example using house prices:</p><ul><li><strong>Actual Price:</strong> 60</li><li><strong>Predicted Price:</strong> 50</li><li><strong>Raw Error:</strong> The difference is 10</li><li><strong>Error(a.k.a Raw Loss) </strong>= Actual Value − Predicted Value = 60 − 50 = 10</li><li>A common way to measure this error is to square it:</li><li><strong>Loss = (60 − 50)² = 100</strong></li></ul><p>A standard method to evaluate this mistake is to square the error, making the individual loss 100. We square the error for two critical reasons:</p><blockquote><strong>Positivity:</strong> It ensures all errors become positive, preventing negative and positive deviations from canceling each other out when added together.</blockquote><blockquote><strong>Penalization:</strong> It heavily penalizes larger errors, forcing the model to prioritize fixing its biggest mistakes.</blockquote><p>When evaluating multiple examples, we average these squared values across the entire dataset. This metric is called <strong>Mean Squared Error (MSE)</strong> and serves as one of the most widely used loss functions in machine learning.</p><p><strong>The ultimate objective during training is to minimize this loss.</strong> Gradient descent is the precise mathematical engine we use to drive that error down to its lowest possible point.</p><p>Here’s The simple Pythonic Implementation below:</p><pre># 1. Mean Squared Error Formula<br># Calculates how wrong the model is<br>def calculate_loss(actual_price, predicted_price):<br>    error = actual_price - predicted_price<br>    squared_error = error ** 2<br>    return squared_error</pre><pre># 2. The Gradient Descent Update Rule Formula<br># Adjusts the weight to reduce the error (as out obj is to minimize the error)<br>def update_weight(old_weight, learning_rate, gradient):<br>    new_weight = old_weight - (learning_rate * gradient)<br>    return new_weight</pre><h3>III. What is Gradient Descent</h3><p>Let’s break the term into two parts:</p><blockquote><strong>Gradient</strong> refers to the slope of a function. It tells us both the direction and the rate at which the function is changing.</blockquote><blockquote><strong>Descent</strong> means moving downward toward a lower value.</blockquote><p>Combining these ideas, <strong>gradient descent</strong> is the process of moving in the direction that decreases a function’s value most rapidly.</p><p>So, Gradient Descent means going downward in the direction of the steepest slope.</p><p>In simple terms, <strong>gradient descent is an optimization algorithm that continuously adjusts model parameters to reduce prediction error and minimize the loss function.</strong></p><h3>IV. The Intuition Behind Gradient Descent</h3><p>The easiest way to understand gradient descent is through a simple analogy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h9c4u0Qt8QSFIdh_Bo3lUA.png" /><figcaption>Intuition Reference 2 — CC Included</figcaption></figure><p>Imagine you are standing on a hill in dense fog. You cannot see the surrounding landscape or the bottom of the valley. Your goal is to reach the lowest point, but the only information available to you is the slope of the ground beneath your feet.</p><p>What would you do?</p><p>You would feel the slope and take a step in the downhill direction. If the ground is steep, you would naturally take a larger step. If the ground is only slightly sloped, you would take a smaller step. By continuously repeating this process, you would eventually reach a point where the ground feels flat. At that moment, you have reached the bottom of the valley.</p><p>Gradient descent follows the same idea when training a machine learning model:</p><ul><li>The <strong>hill</strong> represents the loss function.</li><li>The <strong>bottom of the valley</strong> represents the minimum loss.</li><li>The <strong>slope of the ground</strong> represents the gradient.</li><li>Each <strong>step downhill</strong> represents an update to the model’s weights.</li></ul><pre>#Algorithm 1 Gradient Descent<br>Initialize w1<br>  for k = 1 to K do<br>      Compute ∇F(wk) = Pn<br>      i=1 ∇fi(wk)<br>      Update wk+1 ← wk − α∇F(wk)<br>  end for<br>Return wK</pre><p>By repeatedly moving in the direction that reduces the loss, gradient descent gradually guides the model toward the optimal set of weights.</p><h3>V. What is a Derivative?</h3><p>To understand gradient descent, we first need to understand the concept of a derivative.</p><p>A derivative measures the rate at which a function changes with respect to its input. In simpler terms, it tells us the slope of a function at a specific point.</p><p>Consider a car moving along a road. The car’s position changes over time, and the speedometer tells us how quickly that position is changing. In calculus, this speed is the derivative of position with respect to time.</p><p>The same idea applies to machine learning.</p><p>In gradient descent, we need to know how the loss changes when we slightly change a weight. The derivative provides this information by measuring the slope of the loss function at the current weight value.</p><p>For a function (f(w)), the derivative is written as:</p><pre>f&#39;(w)=df(w)/dw</pre><p>Here, (f’(w)) represents the rate of change of the function with respect to (w). It tells us how much the function’s output changes when (w) changes by a very small amount.</p><p>The sign of the derivative gives us important information about the direction of the slope:</p><blockquote><strong>Positive derivative ((f’(w) &gt; 0))</strong>: The function is increasing.</blockquote><blockquote><strong>Negative derivative ((f’(w) &lt; 0))</strong>: The function is decreasing.</blockquote><blockquote><strong>Zero derivative ((f’(w) = 0))</strong>: The slope is flat, indicating a potential minimum or maximum point.</blockquote><p>This information is exactly what gradient descent needs. By looking at the derivative, the algorithm can determine which direction will reduce the loss and update the weights accordingly.</p><h3>The Update Rule</h3><p>Now that we understand what a derivative tells us, we can define the rule that gradient descent uses to update a weight.</p><p>The gradient descent update rule is:</p><pre>w_new = w_old − α × f&#39;(w_old)</pre><p>where:</p><ul><li><strong>w_old</strong> = current value of the weight</li><li><strong>w_new</strong> = updated value of the weight</li><li><strong>α (alpha)</strong> = learning rate, which controls how large each step is</li><li><strong>f’(w_old)</strong> = derivative (gradient) of the loss function at the current weight</li></ul><p>This equation is the heart of gradient descent. At every iteration, the algorithm computes the gradient and updates the weight using this rule.</p><h3>VI. Why Do We Subtract the Gradient?</h3><p>The goal of gradient descent is to minimize the loss function, which means moving downhill toward the minimum point.</p><p>The derivative tells us the direction of the slope at the current position:</p><ul><li>If the gradient is <strong>positive</strong>, the function is increasing. To move downhill, we need to move in the opposite direction, so we decrease the weight.</li><li>If the gradient is <strong>negative</strong>, the function is decreasing. Subtracting a negative value effectively adds to the weight, causing us to move in the opposite direction of the slope.</li><li>If the gradient is <strong>zero</strong>, the slope is flat, indicating that we may have reached a minimum point.</li></ul><p>This is why the update rule uses a minus sign. Regardless of where we are on the loss curve, subtracting the gradient always moves us toward lower loss values.</p><p>In short:</p><ul><li>If Positive gradient :move left</li><li>If Negative gradient : move right</li><li>If Zero gradient : stop updating</li></ul><p>This simple update rule is what allows machine learning models to learn from data and gradually reduce prediction error.</p><h3>Step by Step Numerical Example</h3><p>Now let’s see gradient descent in action with a simple example.</p><p>Suppose our loss function is:</p><pre>f(w) = (w − 3)²</pre><p>This function measures the error for different values of <strong>w</strong>. The goal is to find the value of <strong>w</strong> that produces the smallest loss.</p><p>If we plot this function, we get a U-shaped curve. The lowest point of the curve occurs at:</p><pre>w = 3<br>At this point:<br>f(3) = (3 − 3)² = 0</pre><p>which is the minimum possible loss.</p><p>Of course, during training, the model does not know that the optimal value is 3. It starts with a random guess and gradually moves toward the minimum using gradient descent.</p><h3>Step 1: Calculate the Gradient</h3><p>Our loss function is:</p><p><strong>f(w) = (w − 3)²</strong></p><p>Taking the derivative gives:</p><p><strong>f’(w) = 2(w − 3)</strong></p><p>This derivative tells us the slope of the loss function at any value of <strong>w</strong>.</p><h3>Step 2: Choose Initial Values</h3><p>Let’s start with:</p><ul><li>Initial weight: <strong>w = 0</strong></li><li>Learning rate: <strong>α = 0.1</strong></li></ul><pre>The update rule is:<br>w_new = w_old − α × f&#39;(w_old)<br>Iteration 1<br>Current weight:<br>w = 0<br>Gradient:<br>f&#39;(0) = 2(0 − 3) = −6<br>The gradient is negative, which means we are on the left side of the minimum.<br>Update:<br>w = 0 − 0.1 × (−6)<br>w = 0.6</pre><p>Notice that the weight increased from 0 to 0.6, moving closer to 3.</p><pre>Iteration 2<br>Current weight:<br>w = 0.6<br>Gradient:<br>f&#39;(0.6) = 2(0.6 − 3) = −4.8<br>Update:<br>w = 0.6 − 0.1 × (−4.8)<br>w = 1.08<br>Again, the weight moves closer to 3.</pre><h3>Iteration 3</h3><pre>Current weight:<br>w = 1.08<br>Gradient:<br>f&#39;(1.08) = 2(1.08 − 3) = −3.84<br>Update:<br>w = 1.08 − 0.1 × (−3.84)<br>w = 1.464</pre><p>The weight continues moving toward the minimum.</p><h3>VII. What Do We Observe?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*6ufjbE9NqaeUksfawn1yEA.png" /></figure><p>Two important things are happening:</p><ol><li>The weight is getting closer to <strong>3</strong> with every update.</li><li>The gradient is becoming smaller in magnitude.</li></ol><p>As we approach the minimum, the slope becomes flatter. Since the gradient becomes smaller, the updates also become smaller. Eventually, the weight settles very close to <strong>w = 3</strong>, where the gradient is nearly zero.</p><p>This process is called <strong>convergence</strong>. Gradient descent keeps taking steps toward the minimum until there is almost no slope left to follow.</p><h3>VIII. Gradient Descent with Multiple Parameters</h3><p>In our previous example, we optimized a single weight, <strong>w</strong>. However, real machine learning models and neural networks contain many parameters. Modern deep learning models can have millions or even billions of weights.</p><p>Suppose our loss function depends on two weights:</p><pre>L(w₁, w₂)</pre><p>To minimize this loss, we need to know how the loss changes with respect to each weight individually. This is where <strong>partial derivatives</strong> come into play.</p><p>A partial derivative measures the effect of changing one parameter while keeping all other parameters fixed.</p><p>For example:</p><pre>∂L/∂w₁</pre><p>represents how the loss changes when <strong>w₁</strong> changes, while <strong>w₂</strong> remains constant.</p><p>Similarly,</p><pre>∂L/∂w₂</pre><p>measures how the loss changes with respect to <strong>w₂</strong>.</p><h3>IX . The Gradient Vector</h3><p>For functions with multiple parameters, the gradient is a collection of all partial derivatives:</p><pre>∇L = [∂L/∂w₁, ∂L/∂w₂]</pre><p>This vector points in the direction of the steepest increase in the loss function.</p><p>To minimize the loss, gradient descent moves in the opposite direction of the gradient.</p><h3>Update Rules</h3><p>Each parameter is updated using its own partial derivative:</p><pre>w₁(new) = w₁(old) − α × (∂L/∂w₁)<br>w₂(new) = w₂(old) − α × (∂L/∂w₂)</pre><p>The same idea extends to any number of parameters. Every weight receives its own gradient, and gradient descent updates all weights simultaneously during each iteration.</p><p>This simple principle allows gradient descent to optimize models containing millions of parameters, making modern machine learning and deep learning possible.</p><h3>X. The Role of Learning Rate</h3><p>The learning rate (α) determines how large each update step will be during gradient descent. It controls how quickly the model moves toward the minimum of the loss function.</p><p>Choosing the right learning rate is crucial. A value that is too small can make training painfully slow, while a value that is too large can prevent the model from converging altogether.</p><p>As mentioned earlier, there are several learning rate strategies such as Constant Learning Rate, Learning Rate Schedules (Decay), Adaptive Learning Rates (ALR), and Cyclical Learning Rates (CLR). These are important topics on their own and will be discussed in a future blog. For now, let’s focus on understanding why the learning rate matters.</p><p>Using the same loss function:</p><p><strong>f(w) = (w − 3)²</strong></p><p>and starting with:</p><p><strong>w = 0</strong></p><p>let’s see what happens with different learning rates.</p><h3>Learning Rate Too Small (α = 0.01)</h3><pre>Step 1:<br>Gradient = 2(0 − 3) = -6<br>Using the gradient descent update formula:<br>w = w − α × ∇f(w)<br>where α is the learning rate and ∇f(w) is the gradient of the loss function.<br>New w = 0 − 0.01 × (-6) = 0.06<br>Step 2:<br>Gradient = 2(0.06 − 3) = -5.88<br>New w = 0.06 − 0.01 × (-5.88) = 0.1188</pre><p>Notice that the weight is moving very slowly. Even after many updates, it remains far from the optimal value of 3.</p><p>A very small learning rate makes gradient descent stable, but training can take a long time because the model makes only tiny improvements at each step.</p><h3>Learning Rate Just Right (α = 0.1)</h3><p><strong>Using α = 0.1, the weight moves steadily toward the minimum.</strong></p><p>As we observed in the numerical example, the updates are large enough to make good progress while remaining stable. The loss decreases smoothly, and the algorithm converges toward the optimal value.</p><p>This is the ideal behavior we want from gradient descent.</p><h3>Learning Rate Too Large (α = 1.5)</h3><pre>Step 1:<br>Gradient = 2(0 − 3) = -6<br>New w = 0 − 1.5 × (-6) = 9<br>Step 2:<br>Gradient = 2(9 − 3) = 12<br>New w = 9 − 1.5 × 12 = -9</pre><p>Instead of moving toward the minimum at w = 3, the weight jumps past it. On the next update, it jumps back to the other side by an even larger amount.</p><p>Rather than approaching the minimum, the weight oscillates and moves farther away. This behavior is known as <strong>divergence</strong>.</p><p>When the learning rate is too large, gradient descent becomes unstable and may never find the minimum.</p><h3>Key Takeaway</h3><ol><li>The learning rate controls the balance between speed and stability.</li></ol><p>2. A very small learning rate leads to slow training.</p><p>3.A very large learning rate causes unstable updates and divergence.</p><blockquote>A well chosen learning rate allows gradient descent to converge efficiently toward the minimum loss.</blockquote><p>In practice, values such as (<strong>0.001–0.01)</strong> are common starting points. For very large deep learning models, learning rates such as <strong>0.0001 (1e-4)</strong> are also widely used.</p><h3>XI. Types of Gradient Descent</h3><p>So far, we have worked with a simple function to understand the mathematics of gradient descent. In real machine learning problems, however, the loss is computed over an entire training dataset.</p><p>For a dataset containing <strong>N</strong> training examples, the loss function can be written as:</p><pre>L = (1/N) Σ loss(xᵢ, yᵢ)</pre><p>where:</p><ul><li><strong>xᵢ</strong> is the input</li><li><strong>yᵢ</strong> is the actual output</li><li><strong>N</strong> is the total number of training examples</li></ul><p>The way we compute the gradient from this dataset gives rise to different types of gradient descent.</p><h3>1. Batch Gradient Descent</h3><p>Batch Gradient Descent uses the entire training dataset to compute the gradient before updating the weights.</p><p>Gradient:</p><pre>∇L = (1/N) Σ ∇loss(xᵢ, yᵢ)</pre><p>Update Rule:</p><pre>w = w − α × ∇L</pre><p>Since all training examples are used, the gradient is very accurate. However, when the dataset is large, computing the gradient can be slow and memory intensive.</p><h4>TensorFlow Example</h4><pre>with tf.GradientTape() as tape:<br>    predictions = model(X_train)<br>    loss = loss_fn(y_train, predictions)</pre><pre>gradients = tape.gradient(loss, model.trainable_variables)<br>optimizer.apply_gradients(zip(gradients, model.trainable_variables))</pre><h4>PyTorch Example</h4><pre>optimizer.zero_grad()<br>predictions = model(X_train)<br>loss = criterion(predictions, y_train)<br>loss.backward()<br>optimizer.step()</pre><h3>2. Stochastic Gradient Descent (SGD)</h3><p>Stochastic Gradient Descent updates the weights using only one randomly selected training example at a time.</p><pre>Algorithm 2 Stochastic Gradient Descent<br>  Initialize w1<br>    for k = 1 to K do<br>    Sample an observation i uniformly at random<br>    Update wk+1 ← wk − α∇fi(wk)<br>   end for<br>Return wK.</pre><p>Gradient:</p><pre>∇L = ∇loss(xᵢ, yᵢ)</pre><p>Update Rule:</p><pre>w = w − α × ∇loss(xᵢ, yᵢ)</pre><p>Since only one example is used, each update is very fast. However, the gradient can be noisy, causing the loss to fluctuate during training.</p><h4>TensorFlow Example</h4><pre>for x, y in dataset.batch(1):<br>    with tf.GradientTape() as tape:<br>        prediction = model(x)<br>        loss = loss_fn(y, prediction)<br>gradients = tape.gradient(loss, model.trainable_variables)<br>    optimizer.apply_gradients(zip(gradients, model.trainable_variables))</pre><h4>PyTorch Example</h4><pre>for x, y in dataloader:<br>    optimizer.zero_grad()<br>prediction = model(x)<br>    loss = criterion(prediction, y)<br>    loss.backward()<br>    optimizer.step()</pre><blockquote>Here, the DataLoader batch size is set to 1.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*pMXuIqvQOAzxmYOW_tKZZg.png" /><figcaption>Table 1: Comparison of complexity of GD and SDG</figcaption></figure><h3>3. Mini Batch Gradient Descent</h3><p>Mini Batch Gradient Descent is a compromise between Batch Gradient Descent and SGD.</p><p>Instead of using the entire dataset or a single example, it uses a small batch of examples.</p><p>If the batch size is <strong>B</strong>, the gradient becomes:</p><pre>∇L = (1/B) Σ ∇loss(xᵢ, yᵢ)</pre><p>Update Rule:</p><pre>w = w − α × ∇L</pre><blockquote>Common batch sizes include 32, 64, 128, 256,512 and 1024. Modern large language models often use much larger batch sizes distributed across multiple GPUs.</blockquote><p>Mini Batch Gradient Descent is the most widely used approach because it provides a good balance between computational efficiency and gradient stability.</p><h4>TensorFlow Example</h4><pre>batch_size = 64<br>for x_batch, y_batch in dataset.batch(batch_size):<br>    with tf.GradientTape() as tape:<br>        predictions = model(x_batch)<br>        loss = loss_fn(y_batch, predictions)<br>    gradients = tape.gradient(loss, model.trainable_variables)<br>    optimizer.apply_gradients(zip(gradients, model.trainable_variables))</pre><h4>PyTorch Example</h4><pre>train_loader = DataLoader(<br>    dataset,<br>    batch_size=32,<br>    shuffle=True<br>)<br>for x_batch, y_batch in train_loader:<br>    optimizer.zero_grad()<br>    predictions = model(x_batch)<br>    loss = criterion(predictions, y_batch)<br>    loss.backward()<br>    optimizer.step()</pre><h3>Comparison</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*8Q1PPqBFcIIVafVIJxvyUw.png" /><figcaption>Comparison</figcaption></figure><p>In practice, Mini Batch Gradient Descent is the preferred choice because it combines the stability of Batch Gradient Descent with the efficiency of Stochastic Gradient Descent.</p><blockquote><strong>Fun Fact:</strong> When people say “I trained my model using SGD,” they are usually using Mini Batch Gradient Descent under the hood with a batch size such as 32 or 64. The name SGD has remained popular even though pure SGD with batch size 1 is rarely used in modern deep learning.</blockquote><h3>4. Parallel SGD</h3><p>So far, we have discussed Batch Gradient Descent, Stochastic Gradient Descent (SGD), and Mini Batch Gradient Descent. However, when working with very large datasets, training can still be slow even with SGD. To address this, researchers proposed a technique called <strong>Parallel SGD</strong>.</p><p>The main idea behind Parallel SGD is simple: instead of training a single model on the entire dataset, we divide the dataset into multiple random partitions and train several SGD models simultaneously.</p><pre>Algorithm 3 Parallel SGD<br>Shuffle the data . this is an expensive operation<br>  for k = 1 to K do . in parallel<br>  Perform SGD on the kth random partition of the data, producing model wk.<br>  end for<br>  Compute the average of the models, w =1/KΣk wk.<br>Return w.</pre><p>The final model is computed as:</p><p><strong>w = (1/K) Σ wₖ</strong></p><p>where:</p><ul><li><strong>K</strong> is the number of partitions.</li><li><strong>wₖ</strong> is the model trained on the k-th partition.</li></ul><p>For example, if we have four partitions, we train four SGD models independently:</p><p><strong>w₁, w₂, w₃, w₄</strong></p><p>The final model becomes:</p><pre>w = (w₁ + w₂ + w₃ + w₄) / 4</pre><p>A crucial step is shuffling the data before partitioning. Without shuffling, different partitions may contain very different data distributions, causing each model to learn different patterns. Random shuffling helps ensure that every partition is representative of the overall dataset.</p><p>Parallel SGD is particularly effective for <strong>strongly convex optimization problems</strong>, where all models tend to converge toward a similar optimum. By training multiple models simultaneously, it can significantly reduce training time while maintaining good accuracy.</p><p>However, for deep neural networks with highly non-convex loss functions, simple model averaging may not always work as effectively because different models can converge to different regions of the loss landscape.</p><h3>5. Streaming SGD</h3><p>So far, we have assumed that the entire dataset is available before training begins. In many real-world applications, however, data arrives continuously as a stream and may even be effectively infinite.</p><p>In <strong>Streaming SGD</strong>, the model is updated whenever a new batch of data arrives. Instead of storing the entire dataset, we collect a small batch of <strong>B</strong> samples, compute the gradient on that batch, and immediately update the model.</p><pre>The gradient for the current batch is:<br>gₖ = Σ ∇fᵢ(wₖ)<br>The model is then updated as:<br>wₖ₊₁ = wₖ - ηgₖ</pre><p>where:</p><ul><li><strong>η</strong> is the learning rate.</li><li><strong>gₖ</strong> is the gradient computed from the latest batch of incoming data.</li></ul><p>This approach is widely used in real-time systems such as recommendation engines, online advertising, fraud detection, and sensor data processing, where new data is continuously generated and the model must adapt on the fly.</p><h3>6. K-Means Clustering</h3><p>K-Means is an unsupervised learning algorithm used to group similar data points into <strong>k clusters</strong>. Given a set of observations:</p><pre>{x₁, x₂, …, xₙ} ∈ ℝᵈ<br>the goal is to find k cluster centers:<br>{c₁, c₂, …, cₖ} ∈ ℝᵈ</pre><p>that minimize the total squared distance between each data point and its nearest cluster center.</p><h3>Objective Function</h3><p>The objective is to minimize:</p><pre>φ = min{c₁,…,cₖ} Σᵢ₌₁ⁿ ||xᵢ − c(xᵢ)||²₂*</pre><p>where <em>c(xᵢ)</em>* denotes the closest center to point <strong>xᵢ</strong>.</p><h3>Lloyd’s Iteration (a.k.a K-Means Algorithm)</h3><ol><li>Start with an arbitrary set of <strong>k</strong> cluster centers:</li><li><strong>c₁⁽¹⁾, c₂⁽¹⁾, …, cₖ⁽¹⁾</strong></li><li>Repeat until convergence:</li><li><strong>Assignment Step</strong></li><li>Assign each point to its nearest center:</li><li><strong>Cᵢ⁽ᵗ⁾ = {x : ||x − cᵢ⁽ᵗ⁾||² ≤ ||x − cˡ⁽ᵗ⁾||², ∀ l = 1,…,k}</strong></li><li><strong>Update Step</strong></li><li>Update each cluster center by taking the average of all points assigned to it:</li><li><strong>cᵢ⁽ᵗ⁺¹⁾ = (1 / |Cᵢ⁽ᵗ⁾|) Σₓ∈Cᵢ⁽ᵗ⁾ x</strong></li><li>Continue until the cluster centers stop changing significantly.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jbvh60fB7QkjdySvhW1TBg.png" /><figcaption>Algo-4</figcaption></figure><h3>Key Idea</h3><p>K-Means alternates between assigning points to the nearest cluster and updating cluster centers to the mean of their assigned points. This process gradually improves the clustering and reduces the objective function.</p><h3>XI. Local Minimum vs Global Minimum</h3><p>When discussing optimization, it is important to understand the difference between a local minimum and a global minimum.</p><blockquote>A <strong>global minimum</strong> is the lowest point of the entire loss function. This is the best possible solution because no other point has a lower loss.</blockquote><blockquote>A <strong>local minimum</strong> is a point where the loss is lower than the surrounding points, but it is not necessarily the lowest point overall.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G__z4C3reop1L4zdrfG4qg.png" /><figcaption>Reference — 3.1 CC Included</figcaption></figure><p>Imagine a mountain range with multiple valleys. If you start walking downhill, you may end up in the nearest valley. That valley is a local minimum. However, there might be another valley somewhere else that is even lower, which would be the global minimum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6zPrnLp-ieG6Tk-zXznhag.png" /><figcaption>Reference — 3.2 CC Included</figcaption></figure><p>In simple loss landscapes:</p><ul><li>Local Minimum = Lowest point in a nearby region</li><li>Global Minimum = Lowest point in the entire function</li></ul><p>For convex functions such as:</p><p><strong>f(w) = (w − 3)²</strong></p><p>there is only one minimum, so the local minimum and global minimum are the same.</p><p>However, deep neural networks often have highly complex, non-convex loss functions containing many local minima and saddle points. Even so, gradient descent and its variants are usually able to find solutions that perform very well in practice.</p><h3>XII. Convex vs Non Convex Loss Functions</h3><p>The shape of the loss function plays an important role in optimization.</p><p>A <strong>convex loss function</strong> has only one minimum point. No matter where gradient descent starts, it will eventually reach the same minimum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jOyNSrBRI4NCUIJ2V__xqg.png" /><figcaption>Reference 4CC Included</figcaption></figure><p>Our example:</p><p><strong>f(w) = (w − 3)²</strong></p><p>is a convex function because it has a single global minimum at <strong>w = 3</strong>.</p><p>A <strong>non convex loss function</strong> can contain multiple local minima, saddle points, and flat regions. The final solution found by gradient descent may depend on the starting point.</p><p>Most traditional machine learning algorithms, such as Linear Regression, often use convex loss functions.</p><p>Deep neural networks, on the other hand, typically have highly non convex loss landscapes because they contain millions of parameters interacting with one another.</p><p>Despite this complexity, gradient descent and its variants are still able to find solutions that perform extremely well in practice.</p><h3>XIII. Gradient Descent in Python</h3><p>Now let’s implement gradient descent in Python using the same loss function:</p><p><strong>f(w) = (w — 3)²</strong></p><pre>w = 0.0<br>learning_rate = 0.1<br>for step in range(50):<br>    gradient = 2 * (w - 3)<br>    w = w - learning_rate * gradient<br>    loss = (w - 3) ** 2<br>    print(<br>        f&quot;Step {step + 1}: &quot;<br>        f&quot;w = {w:.4f}, &quot;<br>        f&quot;loss = {loss:.6f}&quot;<br>    )</pre><p>Let’s break down what happens in each iteration:</p><ol><li>Compute the gradient of the loss function.</li><li>Update the weight using the gradient descent update rule.</li><li>Calculate the new loss.</li><li>Print the current weight and loss.</li></ol><p>After several iterations, the output approaches:</p><pre>Step 50: w = 3.0000, loss = 0.000000</pre><p>Notice that the weight gradually converges to <strong>w = 3</strong>, which is the minimum point of the loss function. At the same time, the loss approaches <strong>0</strong>, indicating that gradient descent has successfully found the optimal solution.</p><p>Although this example uses a simple mathematical function, the exact same idea is used when training machine learning and deep learning models. The main difference is that modern models contain millions of parameters, and the gradients are computed automatically using <strong>backpropagation</strong>.</p><p>Frameworks such as <strong>TensorFlow</strong> and <strong>PyTorch</strong> handle these gradient calculations behind the scenes, allowing us to focus on building and training models rather than manually computing derivatives.</p><h3>Putting It All Together</h3><p>Let’s quickly recap the key concepts we covered:</p><ul><li>A <strong>loss function</strong> measures how far a model’s predictions are from the actual values.</li><li><strong>Gradient descent</strong> is an optimization algorithm that minimizes the loss function.</li><li>The <strong>gradient (derivative)</strong> tells us the direction and steepness of the slope.</li><li>The update rule <strong>w = w − α × gradient</strong> moves the weights toward lower loss values.</li><li>The <strong>learning rate</strong> controls the size of each update step.</li><li>Choosing an appropriate learning rate is essential for stable and efficient training.</li><li>In practice, <strong>Mini Batch Gradient Descent</strong> is the most commonly used variant.</li><li>Modern deep learning frameworks compute gradients automatically using <strong>backpropagation</strong>.</li></ul><p>Gradient descent is one of the most fundamental ideas in machine learning. Whether you are training a simple linear regression model or a billion-parameter neural network, the core principle remains the same: repeatedly adjust the parameters in the direction that reduces the loss.</p><p>Understanding this mathematical foundation makes it much easier to learn advanced topics such as backpropagation, optimization algorithms like Adam and RMSProp, and the training of modern deep learning models.</p><blockquote>If this made gradient descent finally click for you, give it a clap and follow for more practical breakdowns of the concepts that power modern AI.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0d844cde6ac" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/demystifying-gradient-descent-the-math-behind-how-modern-ai-systems-learn-d0d844cde6ac">Demystifying Gradient Descent: The Math Behind How Modern AI Systems Learn</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Engineering Behind Emojis: How  Transforms from Code to Emotion]]></title>
            <link>https://medium.com/data-science-collective/the-hidden-engineering-behind-emojis-how-transforms-from-code-to-emotion-d7e317eeb234?source=rss-d3969283167e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7e317eeb234</guid>
            <category><![CDATA[design]]></category>
            <category><![CDATA[unicode]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[user-experience-design]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Abhilash Puli]]></dc:creator>
            <pubDate>Thu, 20 Nov 2025 17:50:07 GMT</pubDate>
            <atom:updated>2025-11-20T17:50:07.268Z</atom:updated>
            <content:encoded><![CDATA[<p>This Blog is an Journey Through the Invisible Architecture that turn’s a Block of code into an emotion.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FBf_uKbrKbjQ%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DBf_uKbrKbjQ&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FBf_uKbrKbjQ%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/72c0c143af2080139db2c00e901fa0b2/href">https://medium.com/media/72c0c143af2080139db2c00e901fa0b2/href</a></iframe><p><em>We talk to our friends and fam every day, and honestly, we throw in the 🫠 emoji constantly while chatting.</em></p><p>Three seconds. May be less. A thought turns into a feeling,the feeling turns into pixels and suddenly they get you,perfectly: that overwhelming sense of meltingunder life’s absurd chaos.Now you might me Think of what this “overwhelming sense of meltingunder life’s absurd chaos”? isn’t it?</p><p>Leme clear what do that exactly mean too!</p><blockquote>Feeling like everything in life is too muchtoo crazy. In simple it’s ust Happiness out of Control.</blockquote><p><strong>But have you ever just stopped to wonder what just happend in this millisecond time interval? </strong>Between your tap and their screen, a whole unseen world of engineering come a live.</p><p><strong>Unicodes gives meaning , bytes carry the message, fonts give it’s personality, and rendeers paint it into light</strong>.in just milliseconds , raw code transforms into a real Human Emotion.</p><p>This is the Hidden Story of ‘Emoji’ , a digital symbol crafed from a tap to language , design and tech. Suprisingly a Beautiful Engineering.</p><h3><strong><em>ACT I: The Movement Before Emoji</em></strong></h3><p>For Instance: You’re typing a message. Your brain decides “I Need to send that melting Face. Your finger hovers over the keyboard.</p><p>what happens next isn’t a magic , but it feels like it.</p><p>When you tap on 🫠, you’re not choosing a picture, you’re triggering a code point in the Unicode(a.k.s UTF-8) the global system that assigns a meaning to every character used on every device.</p><blockquote>It say’s “Hey it’s just a symbol represents a melting face,nothing more,nothing less”.</blockquote><p>🫠 lives at <strong>U+1FAE0</strong> ,its universal identity. <em>Whether you’re on an iPhone in Guntur or a laptop in Chennai, </em><em>U+1FAE0 always means melting face</em>.Unicode doesn’t decides what it looks like,only what it means.</p><p><em>Just think of it’s like a Word dictionary in which Unicode is the Key and value is the Specific meaning.</em></p><h3><strong><em>ACT II: From Thoughts to Bytes (a.k.s The Encoding Journey)</em></strong></h3><p>So you triggred 🫠. Your Device now holds the code point U+1FAE0.But we Know computers don’t Understand and Speak Unicode diretly right.They speak binary(1|0),the only language machines Understand.</p><p>This is where UTF-8 encoding came into picture.Just imagine UTF-8 is just like a translator.it takes the Unicode code point and turns into a sequence of Bytes That travel Through the Network and Memory.it’s Basically the Postal service of digitial world.</p><p><strong>Transformation Example:</strong></p><pre>Code Point: U+1FAE0<br>Binary:     0001 1111 1010 1110 0000<br>UTF-8:      F0 9F AB A0 (four bytes)</pre><blockquote><em>The melting face 🫠 becomes four Bytes: ‘F0 9F AB A0’</em></blockquote><p>The first Byte (F0)signals the start of first four-byte sequence, and carry rest of content.</p><blockquote><em>But why UTF-8 why not ASCII?</em></blockquote><p>UTF-8’s Advantage is it’s backward compatibility,Efficient storage and Facilitabe vast number of languages char’s,older ASCII syatems don’t crash when they encounter emoji’s; they Just skip simply.</p><blockquote>Just Think of UTF-8 as a <strong>smart College Bag</strong>, fitting emojis, Chinese characters, Arabic script, and more into standard byte-sized Stuff that every computer can handle.</blockquote><h3><strong><em>ACT III : When Code Meets Design</em></strong></h3><p>Now those four Bytes travel throughyour device.they Reach the operating system which asks a crutial question.</p><p>“what does U+1FAE0 look like?” This is where fonts enter into the Flow.</p><p>A font isn’t just a collection of letter shapes. Modern fonts are vast libraries of <strong>glyphs-</strong>visual representations of characters. And for emojis, these <a href="https://developer.mozilla.org/en-US/docs/Glossary/Glyph#:~:text=A%20glyph%20is%20a%20term,the%20characters%20of%20the%20font."><strong>glyphs </strong></a>are works of art.</p><ul><li><strong>Apple</strong> designs emojis with a glossy, three-dimensional style — friendly and slightly playful</li><li><strong>Google</strong> opts for flatter, more cartoonish designs with bold outlines</li><li><strong>Microsoft</strong> creates simpler, often two-dimensional interpretations</li><li><strong>Samsung</strong> has its own distinct personality (remember their infamous blob emojis 🫡?)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YKfr7unuxIuDjVuAYzZgdg.png" /><figcaption>Different Visual Representations With same UTF-8 code</figcaption></figure><p>Unicode only standardizes the Meaning , not appearance.It’s like how the concept of “love” is universal, but every culture expresses it differently.</p><p>Your device checks which font to use. On iOS,’it’s Apple Color Emoji’, On Android it’s “Noto Color Emoji”.</p><pre>🫠 Emoji tap<br>      ↓<br>UTF-8 bytes: F0 9F AB A0<br>      ↓<br>Operating System<br>      ↓<br>&quot;Which font handles U+1FAE0?&quot;<br>      ↓<br>Emoji Font selected<br> (Apple Color Emoji / Noto Color Emoji / Segoe UI Emoji ...)<br>      ↓<br>Glyph lookup for U+1FAE0</pre><h3>ACT IV: How Color Emojis Actually Work(The Technical Deep Dive)</h3><p>Traditional fonts are simple: they store outlines of letters in black. Your system colors them as needed.But emojis are <strong>full-color</strong>, often with gradients, shading, and multiple layers. How do you store that?</p><p>There are three main approaches:</p><blockquote><strong>1.Bitmap Fonts(PNG Images)</strong></blockquote><p>Some emoji fonts work by storing small <strong>PNG images</strong> inside the font file — basically tiny pictures already drawn at fixed sizes (like 72px or 144px). When your device shows the emoji, it just displays that image directly.</p><p>This method is <strong>easy and fast</strong>, but not very flexible.<br> If you zoom in or try to enlarge the emoji, the image starts to look <strong>blurry or pixelated</strong> because a PNG can’t scale smoothly like vectors do.</p><blockquote><strong><em>2.</em></strong><a href="https://www.w3schools.com/graphics/svg_intro.asp"><strong><em>SVG </em></strong></a><strong><em>Fonts(Vector Graphics)</em></strong></blockquote><p>Other fonts store emojis as a<strong> Scalable Vector Graphics.</strong> Instead of Pixels they use Mathematical descriptios of shapes.</p><pre>&lt;svg viewBox=&quot;0 0 36 36&quot;&gt;<br>  &lt;circle cx=&quot;18&quot; cy=&quot;18&quot; r=&quot;18&quot; fill=&quot;#FFCC4D&quot;/&gt;<br>  &lt;path d=&quot;M18,30c-2.2,0-4-1.8-4-4h8C22,28.2,20.2,30,18,30z&quot; fill=&quot;#664500&quot;/&gt;<br>&lt;/svg&gt;</pre><p>This snippet of codes describes (the face) and the path (melting bottom). SVGs Scale infinetely without losing Quality.<strong>Even zoom in 1000x still they’re sharp.</strong></p><blockquote><strong><em>3.COLR/CAPAL Tables(Layered Color)</em></strong></blockquote><p>Microsoft and Google often use a format called <strong>COLR</strong> (color),which stores emojis as multiple layers of shapes,each uses a specific color from a palette(CAPL).Think of it like a digital printing press “each layer adds a color until the full image emerges”.</p><pre>Layer 1: Face outline (yellow #FFCC4D)<br>Layer 2: Eyes (brown #664500)<br>Layer 3: Mouth curve (brown #664500)<br>Layer 4: Melting effect gradient (yellow to transparent)</pre><h3>ACT V: Rendering Pipeline From Data to Light:</h3><p>Now Your device Knows what 🫠 should look like. But it Still needs to actually draw it on your screen.</p><p>Enter the Text Rendering Pipeline , one of the most complex, least appreciated systems in computing.</p><p>Let’s Start Journey:</p><h4>Step 1: Layout Engine</h4><p>Your messaging app sends the text (including emoji bytes) to the operating system’s layout engine. This engine <em>calculates where each character should appear. It handles word wrapping, line height, spacing between characters.</em> Emojis are treated as special characters with specific dimensions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2qTXQ0235cD4M21EozKv-A.png" /><figcaption>Rendering Pipeline Flow</figcaption></figure><h4>Step 2: Font Engine(CoreText,DirectWrite,FreeType)</h4><p>The font engine opens the emoji font file, locates the glyph for U+1FAE0, and loads its data (<strong>whether PNG, SVG, or COLR layers</strong>).</p><h4>Step 3: Rasterization</h4><p>If the glyph is vector-based (SVG or COLR), the <strong><em>rasterizer</em></strong><em> </em>converts those mathematical shapes into actual pixels at the correct size for your screen. This is like turning a blueprint into a building.<br>Think of SVG as a OOP class and <strong><em>rasterizer </em></strong>Executes it as an Object.</p><h4>Step 4: Compositing</h4><p>The rasterized emoji is combined with the background and other text elements. If there are gradients or transparencies, they’re calculated here using <strong><em>alpha(α) blending-</em></strong>mixing colors pixel by pixel.(<strong><em>alpha ranges[0,1]</em></strong>).</p><h4>Step 5: GPU Rendering</h4><p>Finally, your device’s <strong>graphics processor</strong> (GPU) paints those pixels onto the screen. Millions of tiny lights (LEDs or OLEDs) illuminate in precise colors to form that melting face.</p><p>All of this happens in <strong>milliseconds</strong>.</p><p>The melting face emoji 🫠 has traveled from code point to bytes to vectors to pixels to light.</p><p>FlowChart:</p><pre>┌───────────────┐<br>│  You tap 🫠   │<br>└──────┬────────┘<br>       ▼<br>┌────────────────────┐<br>│ Unicode Mapping    │<br>│      U+1FAE0       │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ UTF-8 Encoding     │<br>│  F0 9F AB A0       │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Data Transmission  │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ OS Recognition     │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────────────────────┐<br>│ Font Selection                     │<br>│ Apple / Noto / Segoe / Samsung ... │<br>└────────┬───────────────────────────┘<br>         ▼<br>┌────────────────────┐<br>│ Glyph Retrieval    │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Rendering Format   │<br>│ PNG / SVG / COLR   │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Rasterization      │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Compositing        │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ GPU Rendering      │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Light Emission     │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Human Perception   │<br>└────────┬───────────┘<br>         ▼<br>┌────────────────────┐<br>│ Emotional Meaning  │<br>│   “Overwhelmed”🫠  │<br>└────────────────────┘</pre><h3>Act VI: Why Your Emoji Looks Different Than Mine</h3><p>You send 🫠 to a friend.</p><p>On your iPhone, it looks like a glossy yellow face with gentle melting at the bottom. On their Samsung phone, it might look flatter, with sharper edges. On their Windows laptop? Simpler still, almost like a cartoon drawing.</p><p><strong>This isn’t a bug. It’s a feature.</strong></p><p>Remember: Unicode standardizes <em>meaning</em>, not <em>appearance</em>. The Consortium’s proposal for “melting face” described the emotion it should convey “overwhelmed exhaustion, embarrassment, or sarcastic humor” but left the visual interpretation to designers.</p><p>This creates fascinating situations:</p><p>🔫 (pistol) was once a realistic gun on iOS. After Apple changed it to a toy water gun, other platforms followed. Same code point, radically different meaning.</p><p>This variation is intentional. Emojis are meant to be <em>culturally adaptable</em>. Just like how different languages have different words for snow, different platforms can express the same emoji differently.</p><h3>Interlude: The Unicode Consortium — Guardians of Digital Language</h3><p>Who decides which emojis exist?</p><p>The <strong>Unicode Consortium</strong> — a nonprofit organization in California that includes representatives from Google, Apple, Microsoft, Adobe, government agencies, and linguistics experts.</p><p>They meet regularly to review emoji proposals. Yes, <strong>anyone can propose an emoji</strong>. You need to write a detailed document explaining:</p><ul><li>What the emoji represents</li><li>Why it’s needed (expected usage frequency)</li><li>How it’s distinct from existing emojis</li><li>Visual design suggestions (though the final design is up to platforms)</li></ul><h3>Act VII: The Psychology of Pixel Emotion</h3><p>Studies show that emojis activate the <strong>same brain regions</strong> as real human faces. A 😊 can trigger emotional responses similar to seeing someone genuinely smile. That’s <strong>digital emotional contagion</strong> feelings transmitted through pixels.</p><blockquote>Every emoji is a tiny rebellion against that idea. It says:<br><em> </em><strong><em>Yes, I’m made of code, bytes, and rendering pipelines but, I’m also art, emotion, and human connection.</em></strong></blockquote><p>That’s empathy, engineered.</p><p><em>Next time you send 🫠, take a moment to appreciate the journey. From your fingertip to their heart, through Unicode and bytes and pixels and light, your feeling traveled across the architecture of the internet itself.</em></p><p><em>That’s not just technology.</em></p><p><strong>Enjoyed the read?</strong><br> Follow me for more deep-dives on AI, tech, and innovation — where code meets creativity.</p><p><a href="https://www.linkedin.com/in/abhilash-puli">LinkedIn</a></p><p><a href="https://abhilash.bearblog.dev/blog">B-Blog</a></p><p><a href="https://abhilashpuli98.github.io/">git-io</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7e317eeb234" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/the-hidden-engineering-behind-emojis-how-transforms-from-code-to-emotion-d7e317eeb234">The Hidden Engineering Behind Emojis: How 🫠 Transforms from Code to Emotion</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Art of Making the Internet Talk Back : Python × APIs]]></title>
            <link>https://medium.com/data-science-collective/python-and-rest-apis-interacting-with-web-services-using-python-and-rest-apis-3227ce4cc119?source=rss-d3969283167e------2</link>
            <guid isPermaLink="false">https://medium.com/p/3227ce4cc119</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[rest-api]]></category>
            <dc:creator><![CDATA[Abhilash Puli]]></dc:creator>
            <pubDate>Wed, 27 Aug 2025 04:02:20 GMT</pubDate>
            <atom:updated>2025-10-15T14:34:53.908Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong>Table of Contents</strong></blockquote><ol><li><strong>What is API,API Stands for,Working?</strong></li><li><strong>REST Architectural Constraints</strong></li><li><strong>What are the Benifits of REST APIs?</strong></li><li><strong>REST APIs and Web Services</strong></li><li><strong>HTTP Methods</strong></li><li><strong>Status Codes</strong></li><li><strong>API Endpoints</strong></li><li><strong>REST and Python: Consuming APIs</strong></li></ol><ul><li>GET</li><li>POST</li><li>PUT</li><li>PATCH</li><li>DELETE</li></ul><p><strong>9. REST and Python: Building APIs</strong></p><ul><li>Identify Resources</li><li>Define Endpoints</li><li>Choose Data Interchange Format</li><li>Design Success Responses</li><li>Design Error Responses</li></ul><p><strong>10. REST and Python: Tools of the Trade</strong></p><ul><li>Flask</li><li>Django REST Framework</li><li>FastAPI</li></ul><p><strong>11. Conclusion</strong></p><p><strong>Prerequisites</strong></p><ul><li>Basic knowledge of <a href="https://docs.python.org/3/tutorial/index.html"><strong>Python</strong> </a>(functions, installing packages, running scripts).</li><li>Understanding of <strong>HTTP basics</strong> (what a <strong>r</strong><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods"><strong>equest</strong> <strong>methods</strong></a>/response is, URLs).</li><li>Installed tools:</li><li>Python 3.x</li><li>pip (for installing libraries like requests, Flask, FastAPI, etc.)</li></ul><h3>What is an API?</h3><p><a href="https://www.youtube.com/watch?v=bxuYDT-BWaI"><strong>APIs </strong></a>are mechanisms that enable two software components to communicate with each other using a set of definitions and protocols. For example, the weather bureau’s software system contains daily weather data. The weather app on your phone “talks” to this system via APIs and shows you daily weather updates on your phone.</p><h3>What does API stand for?</h3><p>API stands for Application Programming Interface. In the context of APIs, the word Application refers to any software with a distinct function. Interface can be thought of as a contract of service between two applications. This contract defines how the two communicate with each other using requests and responses. Their API documentation contains information on how developers are to structure those requests and responses.</p><h3>Types of APIs</h3><p>APIs can be categorized in several ways, but one common approach is based on <strong>who has access</strong> to them. The main categories include:</p><ol><li><strong>Private APIs (Internal APIs)</strong></li></ol><ul><li>These APIs are used <strong>within a single organization</strong> to connect different software components.</li><li>They are <strong>not accessible to third-party developers</strong>.</li><li>Example: A social media platform might have separate private APIs for login, feed management, and messaging. Large applications may contain dozens or even hundreds of private APIs.</li></ul><p><strong>2. Public APIs (Open APIs)</strong></p><ul><li>These APIs are <strong>available for use by external developers</strong> and the general public.</li><li>They can be <strong>free or paid</strong>.</li><li>Example: An e-commerce platform might use the <strong>Stripe API</strong> to handle payments, allowing developers to integrate payment processing without building it from scratch.</li></ul><p><strong>3. Partner APIs</strong></p><ul><li>These APIs enable <strong>data or functionality sharing between specific business partners</strong>.</li><li>They are <strong>restricted to authorized partners</strong> and usually require authentication.</li><li>Example: A travel booking platform might allow partner airlines to update availability or pricing via a secure API.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3pXWISgAzEDrkL5wX7d9fw.png" /><figcaption><strong>Types of APIs</strong></figcaption></figure><h3>How do APIs work?</h3><p>API architecture is usually explained in terms of client and server. The application sending the request is called the client, and the application sending the response is called the server. So in the weather example, the bureau’s weather database is the server, and the mobile app is the client.</p><p>There are four different ways that APIs can work depending on when and why they were created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZE-UNnZE1oELmieK" /></figure><p>Consider a <strong>book-distributing company</strong>. Instead of building a full-fledged cloud app for bookstore clerks to check inventory — which could be <strong>costly, platform-dependent, and time-consuming to maintain</strong> — the company could expose a <strong>REST API</strong> for stock availability.</p><p>The advantages of this approach are significant:</p><ol><li><strong>Centralized data access</strong>: Customers can aggregate inventory information from multiple distributors in one place.</li><li><strong>Internal flexibility</strong>: The distributor can modify internal systems without breaking customer workflows, as long as the API contract remains stable.</li><li><strong>Ecosystem growth</strong>: Third-party developers can build apps that leverage the API, enhancing customer experience and potentially driving <strong>higher sales and new business opportunities</strong>.</li></ol><p>This example highlights how APIs not only streamline operations but also <strong>create value for both the provider and its customers</strong>.</p><h3>SOAP APIs</h3><p>These APIs use Simple Object Access Protocol. Client and server exchange messages using XML. This is a less flexible API that was more popular in the past.</p><h3>RPC APIs</h3><p>These APIs are called Remote Procedure Calls. The client completes a function (or procedure) on the server, and the server sends the output back to the client.</p><h3>Websocket APIs</h3><p><a href="https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api-overview.html">Websocket API</a> is another modern web API development that uses JSON objects to pass data. A WebSocket API supports two-way communication between client apps and the server. The server can send callback messages to connected clients, making it more efficient than REST API.</p><h3>REST APIs</h3><p>These are the most popular and flexible APIs found on the web today. The client sends requests to the server as data. The server uses this client input to start internal functions and returns output data back to the client. Let’s look at REST APIs in more detail below.</p><p>The web is overflowing with data — and much of it is up for grabs if you know how to ask. Platforms like <strong>YouTube, GitHub, and X — Formerly Twitter(</strong> expose their data to developers through <strong>APIs (Application Programming Interfaces)</strong>. Among the many ways to design APIs, the <strong>REST (Representational State Transfer) architecture style</strong> has become the go-to standard.</p><p>Python makes working with REST APIs especially painless. Whether you want to <strong>consume existing APIs</strong> (pulling data into your projects) or <strong>build your own REST API</strong> from scratch, Python has the right tools to make it happen.</p><p><strong>By the end of this tutorial, you’ll understand:</strong></p><ul><li><strong>REST architecture</strong> — its principles and why it’s the dominant style for APIs</li><li><strong>How REST APIs expose web data</strong> and why platforms like YouTube and GitHub rely on them</li><li><strong>How to consume API data with Python’s </strong><strong>requests library</strong>, including basic GET and POST examples</li><li><strong>The key steps in building your own REST API</strong>, from identifying resources to handling errors</li><li><strong>Popular Python frameworks for API development</strong> — when to use Flask, Django REST Framework, or FastAPI</li></ul><blockquote><strong>REST (</strong><a href="https://en.wikipedia.org/wiki/REST"><strong><em>Representational State Transfer</em></strong></a><strong>)</strong> is a set of architectural constraints designed to create <strong>efficient, reliable, and scalable distributed systems</strong>.</blockquote><p>At its core, REST treats everything as a <strong>resource</strong> (for example, a document, image, or user record) that can be accessed and manipulated through standardized, language-agnostic <a href="https://en.wikipedia.org/wiki/Client%E2%80%93server_model"><strong><em>client–server interactions</em></strong></a>.</p><p>An API is considered <strong>RESTful</strong> if it follows these constraints. In practice, though, many HTTP APIs are labeled “RESTful” even if they don’t strictly comply. For beginners, you can think of a REST API simply as an <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Overview"><strong>HTTP </strong></a><strong>service</strong> that can be called using standard web tools and libraries.</p><h4>REST API Model:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vOdikmFLZGkP0GQnSf9GeQ.png" /><figcaption>REST API Model</figcaption></figure><h4>REST architectural constraints:</h4><ul><li><strong>Stateless</strong>: Each request from the client must contain all the information needed. The server does not store any session state between requests.</li><li><strong>Client–Server</strong>: Clients and servers are independent. The client handles the user interface, while the server manages data and logic, allowing each to evolve separately.</li><li><strong>Cacheable</strong>: Responses should indicate whether they are cacheable, enabling clients (or intermediaries) to reuse data and improve performance.</li><li><strong>Uniform Interface</strong>: Resources are accessed in a consistent way, typically through URIs and standard methods (like GET, POST). This abstraction decouples clients from implementation details.</li><li><strong>Layered System</strong>: A client may not directly connect to the end server. Instead, requests can pass through intermediaries such as <a href="https://en.wikipedia.org/wiki/Proxy_server">proxies</a>, gateways, or <a href="https://en.wikipedia.org/wiki/Load_balancing_(computing)">load balancers.</a></li><li><strong>Code on Demand (optional)</strong>: Servers can extend client functionality by sending executable code (e.g., <a href="https://www.w3schools.com/js/">JavaScript</a>).</li></ul><blockquote><strong>Important:</strong> REST is not a strict specification. It’s a set of <strong>guidelines and principles</strong> for designing networked systems.</blockquote><h3>What are the benefits of REST APIs?</h3><p>REST APIs offer four main benefits:</p><h3>1. Integration</h3><p>APIs are used to integrate new applications with existing software systems. This increases development speed because each functionality doesn’t have to be written from scratch. You can use APIs to leverage existing code.</p><h3>2. Innovation</h3><p>Entire industries can change with the arrival of a new app. Businesses need to respond quickly and support the rapid deployment of innovative services. They can do this by making changes at the API level without having to re-write the whole code.</p><h3>3. Expansion</h3><p>APIs present a unique opportunity for businesses to meet their clients’ needs across different platforms. For example, maps API allows map information integration via websites, Android,iOS, etc. Any business can give similar access to their internal databases by using free or paid APIs.</p><h3>4. Ease of maintenance</h3><p>The API acts as a gateway between two systems. Each system is obliged to make internal changes so that the API is not impacted. This way, any future code changes by one party do not impact the other party.</p><h3>REST APIs and Web Services</h3><p>A <strong>REST web service</strong> is simply a web service that follows REST’s architectural constraints. These services make their data available to external applications through an <strong>API (Application Programming Interface)</strong>.</p><p>A <strong>REST API</strong> provides access to this data via public web URLs, typically returning results in formats like <strong>JSON</strong> or <strong>XML</strong>.</p><p>For example, GitHub exposes its data through a <a href="https://docs.github.com/en/free-pro-team@latest/rest">REST API</a>. Here’s one of its endpoints:</p><pre>https://api.github.com/users/octocat/repos</pre><p>This URL gives you access to information about a specific <strong>GitHub user</strong>. To interact with a REST API, you send an <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods"><strong>HTTP request</strong></a> to a particular URL (called an <em>endpoint</em>) and then process the <strong>response</strong> the server sends back.</p><p>For instance, a simple GET request to the GitHub API endpoint:</p><pre>https://api.github.com/users/octocat</pre><ul><li>returns JSON data describing the user octocat — including details like their profile name, bio, and number of public repositories.</li></ul><h3>HTTP Methods</h3><p>REST APIs rely on <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods"><strong>HTTP methods</strong></a> (also called <em>verbs</em>) such as GET, POST, and DELETE to determine what kind of action to perform on a <strong>resource</strong>.</p><p>A <strong>resource</strong> is any piece of data the web service exposes — for example, a user profile, an image, or a list of products. The HTTP method specifies <strong>how</strong> the API should interact with that resource: retrieve it, create it, update it, or remove it.</p><p>Although the HTTP standard defines many methods, the following <strong>five are the most commonly used in REST APIs</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*bNI8qiFi0KK1HvyyIv1eng.png" /><figcaption>HTTP standard methods</figcaption></figure><p>The following table lists HTTP request methods and their categorization in terms of safety, cacheability, and idempotency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*hmM7VC3g8mHvf_bzYRcZFw.png" /><figcaption>HTTP request methods and their categorization</figcaption></figure><p>(<a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods">Reference</a>)</p><h3>Status Codes</h3><p>When a REST API processes an HTTP request, it sends back an <strong>HTTP response</strong>. Every response includes a <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status"><strong>status code</strong></a>, which tells you the outcome of the request.</p><p>Status codes are crucial because they let applications decide what to do next — for example, show a success message, retry the request, or handle an error gracefully.</p><p>Here are the most common status codes you’ll see when working with REST APIs:</p><figure><img alt="HTTP response status code Table" src="https://cdn-images-1.medium.com/max/828/1*qJPWA3P111TH-tqDMHGCLQ.png" /><figcaption>common status codes</figcaption></figure><p>These ten status codes represent only a small subset of the available HTTP status codes. Each code belongs to a <strong>category</strong> based on the first digit:</p><figure><img alt="statusCodeCategoryTable" src="https://cdn-images-1.medium.com/max/821/1*fOOTIYLj6cYf8MwcT9Medg.png" /><figcaption>common status codes (Category Based)</figcaption></figure><p>HTTP status codes come in handy when working with REST APIs as you’ll often need to perform different logic based on the results of the request.</p><h3>API Endpoints</h3><p>A <strong>REST API</strong> provides a set of public URLs that client applications can use to access and interact with the resources of a web service. These URLs are called <a href="https://www.ibm.com/think/topics/api-endpoint"><strong>endpoints</strong></a>. Each endpoint corresponds to a specific function or resource within the API.</p><p>For example, consider a <strong>library management system</strong>. Below is a table showing API endpoints for managing the Book resource hypothetical <a href="https://en.wikipedia.org/wiki/Customer_relationship_management">CRM</a> system, which represents the books available in the library:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*1Gdw9_yzycBDOs_e5nDZAA.png" /><figcaption><strong><em>Common REST API endpoints for managing books in a library system</em></strong></figcaption></figure><p>Each of the endpoints above performs a different action based on the HTTP method.</p><blockquote><strong>Note:</strong> The <strong>base URL</strong> for the endpoints has been omitted here for brevity. In practice, you’ll need the full URL to access an API endpoint.</blockquote><blockquote>For example:</blockquote><pre>https://api.example.com/books</pre><blockquote>This is the full URL for the <em>books</em> resource. The <strong>base URL</strong> is everything before <em>/books</em> (in this case, <a href="https://api.example.com)./"><em>https://api.example.com</em>).</a></blockquote><p>You’ll notice that some endpoints include &lt;book_id&gt; at the end. This notation means you need to append a numeric <strong>book ID</strong> to the URL to tell the REST API exactly which book you want to work with.</p><p>Also, keep in mind that the endpoints shown above represent just <strong>one resource</strong> in the system. In a real-world, production-ready REST API, there could be <strong>dozens or even hundreds of endpoints</strong>, each managing different resources and operations within the web service.</p><h3>REST and Python: Consuming APIs</h3><p>When it comes to interacting with REST APIs in Python, most developers rely on the <a href="https://pypi.org/project/requests/"><strong>requests</strong></a>library. It simplifies the process of sending HTTP requests, hiding all the messy low-level details, and is so useful it almost feels like part of the standard library.</p><p>Before you can use it, you’ll need to install it using pip:</p><pre>$ python -m pip install requests</pre><p>Once installed, you’re ready to start sending HTTP requests and working with API data directly in Python.</p><h3>GET</h3><p><strong>GET</strong> is one of the most common HTTP methods you’ll use when working with REST APIs. This method allows you to <strong>retrieve resources</strong> from a given API. GET is a <strong>read-only operation</strong>, so it should never be used to modify an existing resource.</p><p>To test out GET and the other methods in this section, beginners often use <a href="https://jsonplaceholder.typicode.com/"><strong>JSONPlaceholder</strong></a>. This free service provides fake API endpoints that return sample data, making it easy to experiment without setting up your own server.</p><p>For a real-world example, consider a books API. You can try this in the Python REPL like so:</p><pre>&gt;&gt;&gt; import requests<br>&gt;&gt;&gt; api_url = &quot;https://api.example.com/books/1&quot;<br>&gt;&gt;&gt; response = requests.get(api_url)<br>&gt;&gt;&gt; response.json()<br>{&#39;id&#39;: 1, &#39;title&#39;: &#39;The Great Gatsby&#39;, &#39;author&#39;: &#39;F. Scott Fitzgerald&#39;, &#39;available&#39;: True}</pre><p>This code calls requests.get() to send a GET request to /books/1, which responds with the book that has <strong>ID 1</strong>. You can then call <a href="https://docs.python-requests.org/en/master/user/quickstart/#json-response-content">.json()</a> on the response object to access the data returned by the API.</p><p>The response is formatted as <a href="https://www.json.org/json-en.html"><strong>JSON</strong></a>, a key-value structure similar to a <a href="https://realpython.com/python-dicts/"><strong>Python dictionary</strong></a>. JSON is extremely popular and has become the <strong>de facto standard</strong> for data exchange in most REST APIs, making it easy for applications to communicate and share information efficiently.</p><p>Beyond just viewing the JSON data returned by the API, you can also inspect other aspects of the response:</p><pre>&gt;&gt;&gt; response.status_code<br>200<br>&gt;&gt;&gt; response.headers[&quot;Content-Type&quot;]<br>&#39;application/json; charset=utf-8&#39;</pre><p>Here, response.status_code lets you see the <strong>HTTP status code</strong>, which indicates whether the request was successful (200) or if there was an error.</p><p>You can also check the response’s <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers"><strong>HTTP headers</strong></a> with response.headers. This dictionary contains metadata about the response, such as the Content-Type, which in this case confirms that the response is <strong>JSON</strong>.</p><h3>POST</h3><p>Next, let’s see how to use requests to <strong>POST data</strong> to a REST API in order to <strong>create a new resource</strong>. We’ll reference <strong>JSONPlaceholder</strong> again for testing, but in a real-world scenario, we’ll use our books API.</p><p>When sending a POST request, you include <strong>JSON data</strong> in the request body. For example, here’s the data you might send to create a new book:</p><p>When sending a POST request, you include <strong>JSON data</strong> in the request body. For example, here’s the data you might send to create a new book:</p><pre>new_book = {<br>    &quot;title&quot;: &quot;1984&quot;,<br>    &quot;author&quot;: &quot;George Orwell&quot;,<br>    &quot;available&quot;: True<br>}</pre><p>You can then send this data to the API with requests.post():</p><pre>&gt;&gt;&gt; import requests<br>&gt;&gt;&gt; api_url = &quot;https://api.example.com/books&quot;<br>&gt;&gt;&gt; response = requests.post(api_url, json=new_book)<br>&gt;&gt;&gt; response.status_code<br>201<br>&gt;&gt;&gt; response.json()<br>{&#39;id&#39;: 2, &#39;title&#39;: &#39;1984&#39;, &#39;author&#39;: &#39;George Orwell&#39;, &#39;available&#39;: True}</pre><p>Here’s what’s happening:</p><p>First, you create a dictionary containing the data for your new book. Then you pass this dictionary to the json keyword argument of requests.post().</p><p>When you do this, requests.post() automatically sets the request’s <strong>Content-Type</strong> header to application/json and serializes the dictionary into a JSON string, which it includes in the body of the request.</p><p>If you don’t use the json keyword argument, you need to <strong>manually set the Content-Type</strong> and serialize the JSON. Here’s an equivalent version of the previous POST request for creating a new book:</p><pre>&gt;&gt;&gt; import requests<br>&gt;&gt;&gt; import json<br>&gt;&gt;&gt; api_url = &quot;https://api.example.com/books&quot;<br>&gt;&gt;&gt; new_book = {&quot;title&quot;: &quot;Brave New World&quot;, &quot;author&quot;: &quot;Aldous Huxley&quot;, &quot;available&quot;: True}<br>&gt;&gt;&gt; headers = {&quot;Content-Type&quot;: &quot;application/json&quot;}<br>&gt;&gt;&gt; response = requests.post(api_url, data=json.dumps(new_book), headers=headers)<br>&gt;&gt;&gt; response.json()<br>{&#39;id&#39;: 3, &#39;title&#39;: &#39;Brave New World&#39;, &#39;author&#39;: &#39;Aldous Huxley&#39;, &#39;available&#39;: True}</pre><pre>&gt;&gt;&gt; response.status_code<br>201</pre><p>In this version, you create a headers dictionary containing a single header, <strong>Content-Type</strong>, set to application/json. This informs the REST API that the request body contains JSON data.</p><p>You then call requests.post(), but instead of passing the dictionary to the json argument, you first serialize it with json.dumps(new_book) and pass the result to the data argument. The data argument tells requests what data to include in the request body. You also pass the headers dictionary to manually set the HTTP headers.</p><p>Using requests.post() this way has the <strong>same effect</strong> as the previous approach with json=, but it gives you <strong>more control</strong> over the request, such as setting custom headers or using a different serialization method.</p><blockquote><strong>Note:</strong> <a href="https://docs.python.org/3/library/json.html#json.dumps"><em>json.dumps()</em></a> comes from Python’s built-in <strong><em>json</em></strong> module. This module provides convenient methods for working with JSON data in Python, including <strong>serialization</strong> (converting Python objects to JSON strings) and <strong>deserialization</strong> (converting <strong>JSON </strong>strings back into Python objects).</blockquote><ul><li>Once the API responds, you can call response.json() to view the returned JSON. The JSON includes a server-generated <strong>ID</strong> for the new book. The <strong>201 status code</strong> indicates that a new resource was successfully created on the server.</li></ul><h3>PUT</h3><p>Beyond GET and POST, requests also supports all other HTTP methods used in REST APIs. <strong>PUT</strong> is used to <strong>update an existing resource</strong>, and any data sent with a PUT request <strong>completely replaces the existing values</strong> of that resource.</p><p>For example, let’s update a book in our books API. Here’s how it works:</p><pre>&gt;&gt;&gt; import requests<br>&gt;&gt;&gt; api_url = &quot;https://api.example.com/books/10&quot;<br>&gt;&gt;&gt; response = requests.get(api_url)<br>&gt;&gt;&gt; response.json()<br>{&#39;id&#39;: 10, &#39;title&#39;: &#39;Old Book Title&#39;, &#39;author&#39;: &#39;Unknown&#39;, &#39;available&#39;: True}</pre><pre>&gt;&gt;&gt; updated_book = {&quot;title&quot;: &quot;Clean Code&quot;, &quot;author&quot;: &quot;Robert C. Martin&quot;, &quot;available&quot;: True}<br>&gt;&gt;&gt; response = requests.put(api_url, json=updated_book)<br>&gt;&gt;&gt; response.json()<br>{&#39;id&#39;: 10, &#39;title&#39;: &#39;Clean Code&#39;, &#39;author&#39;: &#39;Robert C. Martin&#39;, &#39;available&#39;: True}<br>&gt;&gt;&gt; response.status_code<br>200</pre><p>Here’s what’s happening:</p><p>Here, you first call requests.get() to view the current contents of the existing resource (a todo in JSONPlaceholder or a book in a real API). Next, you call requests.put() with new JSON data to <strong>replace the existing values</strong>. You can confirm the update by calling response.json(), which will show the new data.</p><blockquote>Note that a successful <strong><em>PUT</em></strong><em> </em>request always returns <strong>200</strong>, not <strong>201</strong>, because you are updating an existing resource rather than creating a new one.</blockquote><h3>PATCH</h3><p>Next, you’ll use requests.patch() to <strong>modify specific fields</strong> on an existing resource. Unlike <strong>PUT</strong>, PATCH <strong>does not completely replace</strong> the resource—it only updates the fields included in the JSON you send.</p><p>For beginners, you can experiment with <strong>JSONPlaceholder</strong>, but here’s a real-world example using a books API. Suppose the current state of a book looks like this:</p><pre>{&#39;id&#39;: 12, &#39;title&#39;: &#39;Introduction to Algorithms&#39;, &#39;author&#39;: &#39;Cormen et al.&#39;, &#39;available&#39;: True}</pre><p>Now, you want to <strong>update only the availability</strong> of the book without changing the title or author:</p><pre>&gt;&gt;&gt; import requests<br>&gt;&gt;&gt; api_url = &quot;https://api.example.com/books/12&quot;<br>&gt;&gt;&gt; patch_data = {&quot;available&quot;: False}<br>&gt;&gt;&gt; response = requests.patch(api_url, json=patch_data)<br>&gt;&gt;&gt; response.json()<br>{&#39;id&#39;: 12, &#39;title&#39;: &#39;Introduction to Algorithms&#39;, &#39;author&#39;: &#39;Cormen et al.&#39;, &#39;available&#39;: False}</pre><pre>&gt;&gt;&gt; response.status_code<br>200</pre><p>When you call response.json(), you can see that <strong>only the </strong><strong>available field was updated</strong>, while all other data remains unchanged. A successful PATCH request returns <strong>200</strong>, just like PUT, since the resource already exists.</p><h3>DELETE</h3><p>Finally, if you want to <strong>completely remove a resource</strong>, you use <strong>DELETE</strong>. For beginners, you can experiment with <strong>JSONPlaceholder</strong>, but here’s how it would look for a real-world books API:</p><pre>&gt;&gt;&gt; import requests<br>&gt;&gt;&gt; api_url = &quot;https://api.example.com/books/12&quot;<br>&gt;&gt;&gt; response = requests.delete(api_url)<br>&gt;&gt;&gt; response.json()<br>{}<br>&gt;&gt;&gt; response.status_code<br>200</pre><p>Here’s what happens:</p><p>You call requests.delete() with an API URL that includes the <strong>ID of the book</strong> you want to remove.</p><p>This sends a DELETE request to the REST API, which then deletes the matching resource.</p><p>After the deletion, the API responds with an <strong>empty JSON object</strong>, signaling that the resource has been successfully removed.</p><p>The <strong>requests</strong> library is a versatile and essential tool for working with REST APIs in Python. With it, you can easily perform CRUD operations — GET, POST, PUT, PATCH, and DELETE — making it a cornerstone of any Python developer’s toolkit.</p><p>In the next section, we’ll shift focus to explore what it takes to <strong>build a REST API</strong> from scratch.</p><h3>REST and Python: Building APIs</h3><p>Designing a REST API is a broad topic with many layers, and, as with most areas in technology, there are multiple opinions on the “best” approach. In this section, we’ll focus on <strong>recommended steps and best practices</strong> to help you build a clean, functional, and maintainable API in Python.</p><h3>Identify Resources</h3><p>The first step in building a REST API is to <strong>identify the resources</strong> your API will manage. Resources are typically described as <strong>plural nouns</strong>, such as books, events, or transactions. By listing these resources, you define the different types of data users can interact with through your API.</p><p>It’s also important to consider <strong>nested resources</strong>. For example, a book might have reviews, or an event might have attendees. Mapping out these hierarchies early will make it much easier to <strong>design clear and intuitive endpoints</strong> for your API.</p><h3>Define Your Endpoints</h3><p>Once you’ve identified the resources in your web service, the next step is to <strong>define the API endpoints</strong> that allow clients to interact with those resources. Endpoints typically combine the <strong>resource name</strong> with the appropriate <strong>HTTP method</strong> to perform specific actions.</p><p>For example, consider a books resource in a library management API:</p><figure><img alt="APIendPoints" src="https://cdn-images-1.medium.com/max/685/1*GSzhfg837Cb0EBmrWV0Yww.png" /><figcaption><strong>API endpoints for managing books in a library system</strong></figcaption></figure><p>These six endpoints cover all the operations you’ll need to <strong>create, read, update, and delete books</strong> in the library system.</p><p>Each resource in your web service would have a similar set of endpoints, tailored to the actions users can perform on that resource. By planning these endpoints carefully, you ensure that your API is <strong>consistent, intuitive, and easy to use</strong>.</p><p><strong>Note:</strong> An endpoint shouldn’t contain verbs. Instead, you should select the appropriate HTTP methods to convey the endpoint’s action. For example, the endpoint below contains an unneeded verb:</p><pre>GET /getBooks</pre><p>Here, get is included in the endpoint when it isn’t needed. The HTTP method GET already provides the semantic meaning for the endpoint by indicating the action. You can remove get from the endpoint:</p><pre>GET /books</pre><p>This endpoint contains only a plural noun, and the HTTP method GET communicates the action.</p><h3>Example: Nested Resources</h3><p>Sometimes a resource exists <strong>within another resource</strong>, creating a hierarchy. For instance, a library system might manage <strong>chapters nested under books</strong>. Here’s how the endpoints could look:</p><figure><img alt="Nested Resources Table" src="https://cdn-images-1.medium.com/max/826/1*xGe-fMjOYTbxYYH-mvZFpA.png" /><figcaption><strong>Hierarchical API design: Handling chapters under books.</strong></figcaption></figure><p>With these endpoints, you can manage <strong>chapters for a specific book</strong> in the system.</p><p>This isn’t the only way to structure nested resources. Some developers prefer using <a href="https://en.wikipedia.org/wiki/Query_string"><strong>query strings</strong></a> to access a nested resource. A query string lets you send additional parameters with your HTTP request.</p><p>For example, you could fetch chapters for a specific book using a query string like this:</p><pre>GET /chapters?book_id=214</pre><p>Here, the query string parameter book_id=214 specifies that you want to retrieve only the chapters belonging to book 214. This approach can be useful for filtering or searching, but it’s important to balance readability and RESTful conventions when designing your API.</p><blockquote><strong>Note:</strong> It’s very unlikely that a REST API will remain unchanged throughout the life of a web service. Resources evolve, and endpoints may need updates to reflect these changes. This is where <strong>API versioning</strong> comes in — it allows you to modify your API <strong>without breaking existing integrations</strong>.</blockquote><blockquote>There are several popular strategies for versioning an API, and the right choice depends on your specific requirements:</blockquote><blockquote><a href="https://docs.microsoft.com/en-us/azure/architecture/best-practices/api-design#uri-versioning"><strong>URI versioning</strong></a><strong>:</strong> Include the version number in the URL, e.g., <em>/v1/books</em>.</blockquote><blockquote><a href="https://docs.microsoft.com/en-us/azure/architecture/best-practices/api-design#header-versioning"><strong>HTTP header versioning</strong></a><strong>:</strong> Specify the version in request headers.</blockquote><blockquote><a href="https://docs.microsoft.com/en-us/azure/architecture/best-practices/api-design#query-string-versioning"><strong>Query string versioning</strong></a><strong>:</strong> Use query parameters like <em>?version=1</em>.</blockquote><blockquote><a href="https://docs.microsoft.com/en-us/azure/architecture/best-practices/api-design#media-type-versioning"><strong>Media type versioning</strong></a><strong>:</strong> Include version information in the <em>Content-Type</em> or <em>Accept</em> headers.</blockquote><blockquote>Regardless of the method, <strong>API versioning is essential</strong> for ensuring your API can adapt to changing requirements while continuing to support existing users.</blockquote><p>Now that you’ve covered endpoints, in the next section you’ll look at some options for formatting data in your REST API.</p><h3>Pick Your Data Interchange Format</h3><p>When designing a REST API, you need to decide <strong>how your data will be formatted</strong>. Two common options are <a href="https://en.wikipedia.org/wiki/XML"><strong>XML</strong></a> and <strong>JSON</strong>. Historically, XML was widely used with <a href="https://en.wikipedia.org/wiki/SOAP"><strong>SOAP</strong></a> APIs, but <strong>JSON has become the preferred format for REST APIs</strong> because it’s lightweight, easy to read, and integrates seamlessly with JavaScript and most programming languages.</p><p>To illustrate, here’s an example of a book resource formatted as <strong>XML</strong>:</p><pre>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;<br>&lt;book&gt;<br>    &lt;title&gt;Deep Learning Essentials&lt;/title&gt;<br>    &lt;page_count&gt;420&lt;/page_count&gt;<br>    &lt;pub_date&gt;2022-08-10&lt;/pub_date&gt;<br>    &lt;authors&gt;<br>        &lt;author&gt;<br>            &lt;name&gt;Aurélien Géron&lt;/name&gt;<br>        &lt;/author&gt;<br>        &lt;author&gt;<br>            &lt;name&gt;Francois Chollet&lt;/name&gt;<br>        &lt;/author&gt;<br>    &lt;/authors&gt;<br>    &lt;isbn13&gt;978-1617296864&lt;/isbn13&gt;<br>    &lt;genre&gt;Technology&lt;/genre&gt;<br>&lt;/book&gt;</pre><p>Here, the XML structure uses nested elements to represent hierarchical data, such as multiple authors for a book. While XML is very expressive, it tends to be <strong>more verbose</strong> than JSON, which is why many REST APIs today favor JSON.</p><p>Now, here’s the same book represented in <strong>JSON</strong>:</p><pre>{<br>    &quot;title&quot;: &quot;Deep Learning Essentials&quot;,<br>    &quot;page_count&quot;: 420,<br>    &quot;pub_date&quot;: &quot;2022-08-10&quot;,<br>    &quot;authors&quot;: [<br>        {&quot;name&quot;: &quot;Aurélien Géron&quot;},<br>        {&quot;name&quot;: &quot;Francois Chollet&quot;}<br>    ],<br>    &quot;isbn13&quot;: &quot;978-1617296864&quot;,<br>    &quot;genre&quot;: &quot;Technology&quot;<br>}</pre><p>JSON stores data in <strong>key-value pairs</strong>, similar to a Python dictionary. Like XML, JSON supports <strong>nested structures</strong>, allowing you to model complex relationships, such as multiple authors for a single book.</p><p>While neither JSON nor XML is inherently “better,” <strong>JSON is generally preferred in REST APIs</strong>, especially when paired with front-end frameworks like <a href="https://reactjs.org/"><strong>React</strong></a>, <a href="https://reactjs.org/"><strong>Vue</strong></a>, or Angular. Its simplicity, readability, and native compatibility with JavaScript make it the go-to choice for modern web development.</p><h3>Design Success Responses</h3><p>Once you’ve chosen a data format, the next step is deciding <strong>how your API will respond to HTTP requests</strong>. All responses should follow a <strong>consistent structure</strong> and include the appropriate <strong>HTTP status code</strong>.</p><p>Let’s look at an example of a GET request to /books in a library API. We’ll examine the raw HTTP request and response rather than using a library like requests for clarity:</p><p><strong>HTTP Request:</strong></p><pre>GET /books HTTP/3<br>Host: api.example.com</pre><p>This request has four parts:</p><ol><li>GET – the HTTP method.</li><li>/books – the API endpoint.</li><li>HTTP/1.1 – the HTTP version.</li><li>Host: api.example.com – the API host.</li></ol><p><strong>HTTP Response:</strong></p><pre>HTTP/3 200 OK<br>Content-Type: application/json</pre><pre>[<br>    {<br>        &quot;id&quot;: 1,<br>        &quot;title&quot;: &quot;Deep Learning Essentials&quot;,<br>        &quot;author&quot;: &quot;Aurélien Géron&quot;,<br>        &quot;published_year&quot;: 2022,<br>        &quot;isbn&quot;: &quot;978-1617296864&quot;,<br>        &quot;genre&quot;: &quot;Technology&quot;<br>    },<br>    {<br>        &quot;id&quot;: 2,<br>        &quot;title&quot;: &quot;Clean Code&quot;,<br>        &quot;author&quot;: &quot;Robert C. Martin&quot;,<br>        &quot;published_year&quot;: 2008,<br>        &quot;isbn&quot;: &quot;978-0132350884&quot;,<br>        &quot;genre&quot;: &quot;Software&quot;<br>    },<br>    {<br>        &quot;id&quot;: 3,<br>        &quot;title&quot;: &quot;Artificial Intelligence: A Modern Approach&quot;,<br>        &quot;author&quot;: &quot;Stuart Russell&quot;,<br>        &quot;published_year&quot;: 2016,<br>        &quot;isbn&quot;: &quot;978-0134610993&quot;,<br>        &quot;genre&quot;: &quot;Education&quot;<br>    }<br>]</pre><p>Here’s what you can learn from this response:</p><ul><li>The <strong>200 OK</strong> status code indicates the request was successful.</li><li>The <strong>Content-Type</strong> header specifies application/json, telling the client to parse the response as JSON.</li><li>The response body contains a <strong>list of books</strong>, each represented as a JSON object with consistent fields.</li></ul><p>By maintaining a consistent response structure, your API becomes <strong>predictable, reliable, and easier to consume</strong> for developers.</p><blockquote><strong>Note:</strong> When working with a real API, you’ll encounter <strong>additional HTTP headers</strong> beyond what’s shown here. These headers can vary between APIs, so they’ve been omitted in these examples for simplicity.</blockquote><p>It’s essential to <strong>set the correct </strong><strong>Content-Type header</strong> in your API responses. If your API returns JSON, use Content-Type: application/json; if it returns XML, use Content-Type: application/xml. This header informs the client <strong>how to parse the response data correctly</strong>.</p><p>Additionally, always include an <strong>appropriate HTTP status code</strong>. For a successful GET request, return <strong>200 OK</strong> — this communicates clearly that the request was processed successfully and the data is valid.</p><p>Take a look at another GET request, this time for a <strong>single book</strong>:</p><p><strong>HTTP Request:</strong></p><pre>GET /books/1 HTTP/3<br>Host: api.example.com</pre><p>This request queries the API for book with ID 1.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 200 OK<br>Content-Type: application/json<br>{<br>    &quot;id&quot;: 1,<br>    &quot;title&quot;: &quot;Deep Learning Essentials&quot;,<br>    &quot;author&quot;: &quot;Aurélien Géron&quot;,<br>    &quot;published_year&quot;: 2022,<br>    &quot;isbn&quot;: &quot;978-1617296864&quot;,<br>    &quot;genre&quot;: &quot;Technology&quot;<br>}</pre><p>Here, the response contains a <strong>single JSON object</strong> with the book’s data. Since it’s a single object, there’s no need to wrap it in a list. As before, the <strong>200 OK</strong> status code confirms that the request was successful.</p><blockquote><strong>Note:</strong> A <em>GET</em> request should never modify an existing resource. If the request contains data, then this data should be ignored and the API should return the resource unchanged.</blockquote><p>Next, let’s look at a POST request to <strong>add a new book</strong>:</p><p><strong>HTTP Request:</strong></p><pre>POST /books HTTP/3<br>Host: api.example.com<br>Content-Type: application/json<br>{<br>    &quot;title&quot;: &quot;Python for Data Analysis&quot;,<br>    &quot;author&quot;: &quot;Wes McKinney&quot;,<br>    &quot;published_year&quot;: 2018,<br>    &quot;isbn&quot;: &quot;978-1491957660&quot;,<br>    &quot;genre&quot;: &quot;Data Science&quot;<br>}</pre><p>This POST request includes JSON data for the new book and sets the <strong>Content-Type header</strong> to application/json so the API knows how to interpret the request. The API will create a new book using this data.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 201 Created<br>Content-Type: application/json<br>{<br>    &quot;id&quot;: 4,<br>    &quot;title&quot;: &quot;Python for Data Analysis&quot;,<br>    &quot;author&quot;: &quot;Wes McKinney&quot;,<br>    &quot;published_year&quot;: 2018,<br>    &quot;isbn&quot;: &quot;978-1491957660&quot;,<br>    &quot;genre&quot;: &quot;Data Science&quot;<br>}</pre><p>Key points from this response:</p><ul><li>The <strong>201 Created</strong> status code indicates that a new resource has been successfully created.</li><li>The response includes a <strong>copy of the new book</strong>, with an id generated by the API. Including the ID is important so the client can reference or update the resource later.</li></ul><blockquote><strong>Note:</strong> It’s important to always send back a copy of a resource when a user creates it with <em>POST</em> or modifies it with <em>PUT</em> or <em>PATCH</em>. This way, the user can see the changes that they’ve made.</blockquote><p>Now, let’s look at a PUT request to <strong>update a book</strong>:</p><p><strong>HTTP Request:</strong></p><pre>PUT /books/4 HTTP/3<br>Host: api.example.com<br>Content-Type: application/json<br>{<br>    &quot;title&quot;: &quot;Advanced Python Programming&quot;,<br>    &quot;author&quot;: &quot;Joe Marini&quot;,<br>    &quot;published_year&quot;: 2021,<br>    &quot;isbn&quot;: &quot;978-1492051376&quot;,<br>    &quot;genre&quot;: &quot;Programming&quot;<br>}</pre><p>This request uses the id of the previously created book to <strong>replace all its fields</strong> with new data. Remember, <strong>PUT replaces the entire resource</strong>, so every field should be included in the request.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 200 OK<br>Content-Type: application/json<br>{<br>    &quot;id&quot;: 4,<br>    &quot;title&quot;: &quot;Advanced Python Programming&quot;,<br>    &quot;author&quot;: &quot;Joe Marini&quot;,<br>    &quot;published_year&quot;: 2021,<br>    &quot;isbn&quot;: &quot;978-1492051376&quot;,<br>    &quot;genre&quot;: &quot;Programming&quot;<br>}</pre><p>The <strong>200 OK</strong> status code confirms that the update was successful, and the response body returns the updated book.</p><p>The same concept applies to a <strong>PATCH request</strong>, which is used to <strong>update only part of a resource</strong>:</p><p><strong>HTTP Request:</strong></p><pre>PATCH /books/4 HTTP/3<br>Host: api.example.com<br>Content-Type: application/json<br>{<br>    &quot;published_year&quot;: 2022,<br>    &quot;genre&quot;: &quot;Software Development&quot;<br>}</pre><p>This request updates only the published_year and genre fields of the book with ID 4, leaving all other fields unchanged.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 200 OK<br>Content-Type: application/json<br>{<br>    &quot;id&quot;: 4,<br>    &quot;title&quot;: &quot;Advanced Python Programming&quot;,<br>    &quot;author&quot;: &quot;Joe Marini&quot;,<br>    &quot;published_year&quot;: 2022,<br>    &quot;isbn&quot;: &quot;978-1492051376&quot;,<br>    &quot;genre&quot;: &quot;Software Development&quot;<br>}</pre><p>The response returns a <strong>full copy of the book</strong>, but only the fields included in the PATCH request (published_year and genre) have been updated.</p><p>Finally, let’s see how a REST API should respond to a <strong>DELETE request</strong>. Here’s an example to remove a book:</p><p><strong>HTTP Request:</strong></p><pre>DELETE /books/4 HTTP/3<br>Host: api.example.com</pre><p>This request instructs the API to delete the book with ID 4.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 204 No Content</pre><p>The response includes only the <strong>204 No Content</strong> status code, which indicates that the operation was successful but <strong>no content is returned</strong>. This makes sense because the book has been deleted — there’s no need to send it back.</p><p>The examples above demonstrate how to handle <strong>successful requests</strong>, but what happens when something goes wrong? In the next section, we’ll cover <strong>how your REST API should respond to errors</strong>.</p><h3>Design Error Responses</h3><p>Even the best-designed APIs can encounter errors. It’s important to define a <strong>consistent error response format</strong> that includes both a descriptive message and the appropriate <strong>HTTP status code</strong>.</p><p>For example, consider a request for a resource that doesn’t exist in the API:</p><p><strong>HTTP Request:</strong></p><pre>GET /magazines HTTP/3<br>Host: api.example.com</pre><p>Here, the client requests the /magazines endpoint, which doesn’t exist.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 404 Not Found<br>Content-Type: application/json<br>{<br>    &quot;error&quot;: &quot;The requested resource was not found.&quot;<br>}</pre><ul><li>The <strong>404 Not Found</strong> status code indicates that the resource does not exist.</li><li>The response body contains a <strong>descriptive error message</strong> in JSON format. Providing clear error messages gives developers <strong>more context</strong> and helps them debug or correct their requests.</li></ul><p>Next, consider an error response when a client sends an <strong>invalid request</strong>:</p><p><strong>HTTP Request:</strong></p><pre>POST /books HTTP/3<br>Host: api.example.com<br>Content-Type: application/json<br>{<br>    &quot;title&quot;: &quot;Learning Python&quot;,<br>    &quot;author&quot;: &quot;Mark Lutz&quot;,<br>    &quot;published_year&quot;: 2013,<br>    &quot;genre&quot;: &quot;Programming&quot;</pre><p><em>Note:</em> This JSON is <strong>malformed</strong>, missing the closing curly brace (}), so the API cannot process it.</p><p><strong>HTTP Response:</strong></p><pre>HTTP/3 400 Bad Request<br>Content-Type: application/json<br>{<br>    &quot;error&quot;: &quot;This request was not properly formatted. Please send again.&quot;<br>}</pre><p>The <strong>400 Bad Request</strong> status code tells the client that their request could not be processed.The response body contains a <strong>descriptive error message</strong>, helping the client understand <strong>what went wrong</strong> and how to correct it.</p><p>There are several ways a request can fail, even if it’s <strong>formatted correctly</strong>.</p><p><strong>1. Unsupported Media Type:</strong><br> Suppose a client sends a POST request using XML, but the API only supports JSON:</p><p><strong>HTTP Request:</strong></p><pre>POST /books HTTP/3<br>Host: api.example.com<br>Content-Type: application/xml<br>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; ?&gt;<br>&lt;book&gt;<br>    &lt;title&gt;Python Crash Course&lt;/title&gt;<br>    &lt;author&gt;Eric Matthes&lt;/author&gt;<br>    &lt;published_year&gt;2019&lt;/published_year&gt;<br>    &lt;genre&gt;Programming&lt;/genre&gt;<br>&lt;/book&gt;</pre><p><strong>HTTP Response:</strong></p><pre>HTTP/3 415 Unsupported Media Type<br>Content-Type: application/json<br>{<br>    &quot;error&quot;: &quot;The application/xml media type is not supported.&quot;<br>}</pre><ul><li><strong>415 Unsupported Media Type</strong> indicates that the request included a data format the API cannot process.</li></ul><p><strong>2. Invalid Data in Correct Format:</strong><br> Even if the JSON is valid, the API might reject unexpected or unsupported fields:</p><p><strong>HTTP Request:</strong></p><pre>POST /books HTTP/3<br>Host: api.example.com<br>Content-Type: application/json<br>{<br>    &quot;title&quot;: &quot;Python Crash Course&quot;,<br>    &quot;author&quot;: &quot;Eric Matthes&quot;,<br>    &quot;pages&quot;: 550,<br>    &quot;extraField&quot;: &quot;Invalid data&quot;<br>}</pre><p><strong>HTTP Response:</strong></p><pre>HTTP/3 422 Unprocessable Entity<br>Content-Type: application/json<br>{<br>    &quot;error&quot;: &quot;Request had invalid or missing data.&quot;<br>}</pre><ul><li>The <strong>422 Unprocessable Entity</strong> status code indicates that the request was syntactically correct, but the <strong>data itself was invalid</strong>. A well-designed REST API should always <strong>validate incoming data</strong> and clearly inform users of any errors.</li></ul><p>Responding to requests — both successful and erroneous — is one of the most important jobs of a REST API. An <strong>intuitive API with accurate responses</strong> makes it much easier for developers to build applications around your web service. Fortunately, several <strong>Python web frameworks</strong> handle much of the complexity of processing HTTP requests and returning structured responses. In the next section, we’ll explore <strong>three popular frameworks</strong> that simplify building REST APIs.</p><h3>REST and Python: Tools of the Trade</h3><p>In this section, we’ll explore three popular <strong>Python frameworks</strong> for building REST APIs. Each framework has its strengths and trade-offs, so you’ll need to decide which one best suits your project. To demonstrate, we’ll build a similar API in each framework that manages a <strong>collection of books</strong>.</p><p>Each book will have the following fields:</p><ul><li><strong>title</strong> — the name of the book.</li><li><strong>author</strong> — the author of the book.</li><li><strong>pages</strong> — the number of pages in the book.</li></ul><p>These fields store essential information about each book.</p><p>Most of the time, REST APIs fetch data from a <strong>database</strong>, but connecting to a database is beyond the scope of this tutorial. For our examples, we’ll store the data in a <strong>Python list</strong>, except for the Django REST framework example, which uses Django’s built-in <strong>SQLite database</strong>.</p><p>This approach keeps the examples <strong>simple and focused</strong>, letting you learn the framework mechanics before adding database complexity.</p><blockquote><strong>Note:</strong> It’s a good practice to create <strong>separate folders</strong> for each framework example to keep the source files organized. Additionally, you should use <strong>Python virtual environments</strong> for each project to isolate dependencies and avoid conflicts between frameworks.</blockquote><p>To keep things consistent, we’ll use <strong>books</strong> as the main endpoint for all three frameworks. We’ll also use <strong>JSON</strong> as the data format across the examples.</p><p>Now that the background is set, let’s dive into <strong>building a REST API in Flask</strong>.</p><h3>Flask</h3><p><a href="http://geeksforgeeks.org/python/flask-tutorial/"><strong>Flask</strong></a> is a lightweight Python microframework for building web applications and REST APIs. It provides a solid foundation while giving you the flexibility to structure your application the way you want. At its core, Flask <strong>handles HTTP requests</strong> and <strong>routes them</strong> to the appropriate function in your application.</p><p>For our example, the API will manage a collection of books, each with the fields:</p><ul><li>title</li><li>author</li><li>pages</li></ul><p>We’ll store these books in a Python list for simplicity, allowing us to focus on <strong>Flask mechanics</strong> without worrying about databases.</p><blockquote><strong>Note:</strong> The code in this section uses the <strong>new Flask 2 syntax</strong>. If you’re running an older version of Flask, you’ll need to adjust the route decorators.</blockquote><p>For example, instead of using:</p><pre>@app.get(&quot;/books&quot;)<br>@app.post(&quot;/books&quot;)</pre><p>in <strong>Flask 1.x</strong>, you should use:</p><pre>@app.route(&quot;/books&quot;)<br>@app.route(&quot;/books&quot;, methods=[&quot;POST&quot;])</pre><p>This handles GET and POST requests to the /books endpoint in older Flask versions.</p><h3>Example Flask Application: Books API</h3><pre># app.py<br>from flask import Flask, request, jsonify<br>app = Flask(__name__)<br>books = [<br>    {&quot;id&quot;: 1, &quot;title&quot;: &quot;Python Crash Course&quot;, &quot;author&quot;: &quot;Eric Matthes&quot;, &quot;pages&quot;: 550},<br>    {&quot;id&quot;: 2, &quot;title&quot;: &quot;Automate the Boring Stuff&quot;, &quot;author&quot;: &quot;Al Sweigart&quot;, &quot;pages&quot;: 600},<br>    {&quot;id&quot;: 3, &quot;title&quot;: &quot;Fluent Python&quot;, &quot;author&quot;: &quot;Luciano Ramalho&quot;, &quot;pages&quot;: 790},<br>]<br>def _find_next_id():<br>    return max(book[&quot;id&quot;] for book in books) + 1<br>@app.get(&quot;/books&quot;)<br>def get_books():<br>    return jsonify(books)<br>@app.post(&quot;/books&quot;)<br>def add_book():<br>    if request.is_json:<br>        book = request.get_json()<br>        book[&quot;id&quot;] = _find_next_id()<br>        books.append(book)<br>        return book, 201<br>    return {&quot;error&quot;: &quot;Request must be JSON&quot;}, 415</pre><h3>How It Works</h3><ul><li><strong>GET /books</strong>: Returns the full list of books.</li><li><strong>POST /books</strong>: Adds a new book to the list. The API expects <strong>JSON data</strong>, assigns a new id automatically, and returns a <strong>201 Created</strong> response. If the request is not JSON, it returns a <strong>415 Unsupported Media Type</strong> error.</li></ul><blockquote><strong>Note:</strong> This Flask application includes functions to handle only two types of requests to the API endpoint, <em>/countries</em>. In a full REST API, you’d want to expand this to include functions for all the required operations.</blockquote><p>You can try out this application by first installing Flask with pip:</p><pre>$ python -m pip install flask</pre><p>Once installed, save the code in a file called app.py. To run the Flask application, you need to set an <strong>environment variable</strong> called FLASK_APP pointing to app.py. This tells Flask which file contains your application.</p><p>On <strong>macOS or Linux</strong>, run:</p><pre>$ export FLASK_APP=app.py</pre><p>Optionally, you can enable <strong>debug mode</strong> to get helpful error messages and automatic reloads whenever you change your code:</p><pre>$ export FLASK_ENV=development</pre><blockquote><strong>Note:</strong> The above commands work on macOS or Linux. If you’re running this on Windows, then you need to set <em>FLASK_APP</em> and <em>FLASK_ENV</em> like this in the Command Prompt:</blockquote><pre>C:\&gt; set FLASK_APP=app.py<br>C:\&gt; set FLASK_ENV=development</pre><p>Now FLASK_APP and FLASK_ENV are set inside the Windows shell.</p><p>With all environment variables set, you can start the Flask development server by running:</p><pre>$ flask run</pre><p>You’ll see output like this:</p><pre>* Serving Flask app &quot;app.py&quot; (lazy loading)<br>* Environment: development<br>* Debug mode: on<br>* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)</pre><p>This launches a server running your application. Open your browser and navigate to <a href="http://127.0.0.1:5000/books">http://127.0.0.1:5000/books</a>, and you’ll see the following JSON response:</p><pre>[<br>    {&quot;id&quot;: 1, &quot;title&quot;: &quot;Python Crash Course&quot;, &quot;author&quot;: &quot;Eric Matthes&quot;, &quot;pages&quot;: 550},<br>    {&quot;id&quot;: 2, &quot;title&quot;: &quot;Automate the Boring Stuff&quot;, &quot;author&quot;: &quot;Al Sweigart&quot;, &quot;pages&quot;: 600},<br>    {&quot;id&quot;: 3, &quot;title&quot;: &quot;Fluent Python&quot;, &quot;author&quot;: &quot;Luciano Ramalho&quot;, &quot;pages&quot;: 790}<br>]</pre><p>This response contains the three books defined at the start of app.py.</p><p>Here’s how it works in the code:</p><pre>@app.get(&quot;/books&quot;)<br>def get_books():<br>    return jsonify(books)</pre><p>The @app.get() decorator <strong>connects GET requests</strong> to the get_books() function. When you access /books, Flask calls this function to handle the request and return a JSON response with all the book data.</p><p>In the code above, get_books() takes books, which is a Python list, and converts it to JSON using jsonify(). This JSON is then returned in the HTTP response.</p><blockquote><strong>Note:</strong> Most of the time, you can return a <strong>Python dictionary</strong> directly from a Flask function. Flask will automatically convert it to JSON. For example:</blockquote><pre>@app.get(&quot;/book&quot;)<br>def get_book():<br>    return books[1]</pre><p>Here, the function returns the second dictionary from the books list. Flask automatically converts this dictionary to JSON. When you request /book, you’ll see:</p><pre>{<br>    &quot;id&quot;: 2,<br>    &quot;title&quot;: &quot;Automate the Boring Stuff&quot;,<br>    &quot;author&quot;: &quot;Al Sweigart&quot;,<br>    &quot;pages&quot;: 600<br>}</pre><p>This is the JSON version of the Python dictionary returned by get_book().</p><p>For get_books(), you <strong>must use </strong><strong>jsonify()</strong> because it returns a <strong>list of dictionaries</strong>. Flask does not automatically convert lists to JSON.</p><p>Next, let’s look at add_book(), which handles <strong>POST requests</strong> to /books and allows you to add a new book to the list. It uses the Flask request object to access the incoming HTTP request and extract JSON data.</p><pre>@app.post(&quot;/books&quot;)<br>def add_book():<br>    if request.is_json:<br>        book = request.get_json()<br>        book[&quot;id&quot;] = _find_next_id()<br>        books.append(book)<br>        return book, 201<br>    return {&quot;error&quot;: &quot;Request must be JSON&quot;}, 415</pre><p>This function performs several operations:</p><ol><li>Uses <a href="https://flask.palletsprojects.com/en/1.1.x/api/?highlight=is_json#flask.Request.is_json"><strong>request.is_json</strong></a><strong> </strong>to verify that the incoming request contains JSON.</li><li>Creates a new book instance with request.get_json().</li><li>Finds the next available id using _find_next_id() and sets it on the book.</li><li>Appends the new book to the books list.</li><li>Returns the newly created book in the response along with a <strong>201 Created</strong> status code.</li><li>Returns an error message and a <strong>415 Unsupported Media Type</strong> status code if the request isn’t JSON.</li></ol><p>The helper function _find_next_id() determines the ID for the new book:</p><pre>def _find_next_id():<br>    return max(book[&quot;id&quot;] for book in books) + 1</pre><p>This function uses a generator expression to select all existing book IDs and then finds the largest value. Adding 1 gives the next available ID.</p><p>You can test this endpoint using <strong>curl</strong> from the command line to simulate a POST request:</p><pre>$ curl -i http://127.0.0.1:5000/books \<br>-X POST \<br>-H &#39;Content-Type: application/json&#39; \<br>-d &#39;{&quot;title&quot;:&quot;Clean Code&quot;, &quot;author&quot;:&quot;Robert C. Martin&quot;, &quot;pages&quot;:364}&#39;</pre><p>The response will look like this:</p><pre>HTTP/3 201 CREATED<br>Content-Type: application/json<br>...<br>{<br>    &quot;id&quot;: 4,<br>    &quot;title&quot;: &quot;Clean Code&quot;,<br>    &quot;author&quot;: &quot;Robert C. Martin&quot;,<br>    &quot;pages&quot;: 364<br>}</pre><p><strong>Curl options to note:</strong></p><ul><li>-X specifies the HTTP method.</li><li>-H sets an HTTP header.</li><li>-d provides the request data.</li></ul><p>With these options, curl sends JSON data in a POST request, and the API returns <strong>201 Created</strong> along with the JSON for the new book.</p><blockquote><strong>Note:</strong> In this example, <em>add_book()</em> doesn’t validate that the incoming JSON matches the expected format for a book. If you want to ensure the request contains valid fields like <em>title</em>, <em>author</em>, and <em>pages</em>, you can use packages such as <a href="https://pypi.org/project/Flask-Expects-JSON/"><strong>flask-expects-json</strong></a> to enforce schema validation in Flask.</blockquote><blockquote>This helps prevent users from sending malformed or incomplete data, making your API more robust and predictable.</blockquote><p>You can use curl to send a GET request to /books to confirm that the new book was added. If you don’t use -X in your curl command, it defaults to a GET request:</p><pre>$ curl -i http://127.0.0.1:5000/books</pre><p><strong>Response:</strong></p><pre>HTTP/3 200 OK<br>Content-Type: application/json<br>...<br>[<br>    {<br>        &quot;id&quot;: 1,<br>        &quot;title&quot;: &quot;Python Basics&quot;,<br>        &quot;author&quot;: &quot;David Amos&quot;,<br>        &quot;pages&quot;: 635<br>    },<br>    {<br>        &quot;id&quot;: 2,<br>        &quot;title&quot;: &quot;Advanced ML&quot;,<br>        &quot;author&quot;: &quot;Joanna Jablonski&quot;,<br>        &quot;pages&quot;: 480<br>    },<br>    {<br>        &quot;id&quot;: 3,<br>        &quot;title&quot;: &quot;Deep Learning 101&quot;,<br>        &quot;author&quot;: &quot;Dan Bader&quot;,<br>        &quot;pages&quot;: 520<br>    },<br>    {<br>        &quot;id&quot;: 4,<br>        &quot;title&quot;: &quot;Flask for APIs&quot;,<br>        &quot;author&quot;: &quot;Fletcher Heisler&quot;,<br>        &quot;pages&quot;: 300<br>    }<br>]</pre><p>This returns the full list of books in the system, with the newest book appearing at the bottom.</p><p>This example demonstrates just a fraction of what Flask can do. You could expand this application to support all HTTP methods, and leverage Flask’s rich ecosystem of extensions for tasks like database integration, authentication, or background processing.</p><h3>Django REST Framework</h3><p>Another popular choice for building REST APIs in Python is <a href="https://www.django-rest-framework.org/"><strong>Django REST framework (DRF)</strong></a>. DRF is a plugin for Django that adds powerful REST API functionality on top of an existing Django project.</p><p>To use Django REST framework, you first need a Django project. If you already have one, you can integrate DRF directly. Otherwise, follow along to create a new project and add DRF.</p><p>Start by installing Django and Django REST framework:</p><pre>$ python -m pip install Django djangorestframework</pre><p>Once installed, create a new Django project:</p><pre>$ django-admin startproject bookapi</pre><p>This creates a bookapi folder in your current directory containing all the necessary files for the project. Next, create a new Django <strong>application</strong> to handle your books API. In Django, each app manages a distinct part of the project:</p><pre>$ cd bookapi<br>$ python manage.py startapp books</pre><p>This creates a books folder inside your project. Inside it, you’ll find the base files for the application.</p><p>Finally, tell Django about your new app. Open bookapi/settings.py and add &quot;books&quot; and &quot;rest_framework&quot; to INSTALLED_APPS:</p><pre>INSTALLED_APPS = [<br>    &quot;django.contrib.admin&quot;,<br>    &quot;django.contrib.auth&quot;,<br>    &quot;django.contrib.contenttypes&quot;,<br>    &quot;django.contrib.sessions&quot;,<br>    &quot;django.contrib.messages&quot;,<br>    &quot;django.contrib.staticfiles&quot;,<br>    &quot;rest_framework&quot;,<br>    &quot;books&quot;,<br>]</pre><blockquote><strong><em>Note:</em></strong><em> This tutorial only scratches the surface of Django’s capabilities. To dive deeper, explore the official </em><a href="https://www.w3schools.com/django/"><strong><em>Django tutorials </em></strong></a><em>and documentation.</em></blockquote><p>You may be wondering why rest_framework needs to be added. It’s because Django REST framework is just another Django app—like any other plugin. Django treats it as a standard application that provides additional functionality for your project.</p><p>The next step is to create a <strong>Django model</strong> to define the fields of your data. Inside the books application, update models.py:</p><pre># books/models.py<br>from django.db import models<br>class Book(models.Model):<br>    title = models.CharField(max_length=200)<br>    author = models.CharField(max_length=100)<br>    page_count = models.IntegerField(help_text=&quot;Number of pages&quot;)<br>    published_date = models.DateField()</pre><p>This defines a Book model. Django will use this to create the database table and columns for your book data.</p><p>Run the following commands to update the database:</p><pre>$ python manage.py makemigrations<br>Migrations for &#39;books&#39;:<br>  books/migrations/0001_initial.py<br>    - Create model Book</pre><pre>$ python manage.py migrate<br>Operations to perform:<br>  Apply all migrations: admin, auth, contenttypes, books, sessions<br>Running migrations:<br>  Applying contenttypes.0001_initial... OK<br>  Applying auth.0001_initial... OK<br>  Applying books.0001_initial... OK</pre><p>These commands create a new database table for books.</p><p>The table starts empty, but it’s useful to have some initial data for testing DRF. You can use a <strong>Django fixture</strong> to load sample books. Save the following JSON as books.json inside the books directory:</p><pre>[<br>    {<br>        &quot;model&quot;: &quot;books.book&quot;,<br>        &quot;pk&quot;: 1,<br>        &quot;fields&quot;: {<br>            &quot;title&quot;: &quot;Python Basics&quot;,<br>            &quot;author&quot;: &quot;David Amos&quot;,<br>            &quot;page_count&quot;: 635,<br>            &quot;published_date&quot;: &quot;2021-03-16&quot;<br>        }<br>    },<br>    {<br>        &quot;model&quot;: &quot;books.book&quot;,<br>        &quot;pk&quot;: 2,<br>        &quot;fields&quot;: {<br>            &quot;title&quot;: &quot;Data Science Fundamentals&quot;,<br>            &quot;author&quot;: &quot;Joanna Jablonski&quot;,<br>            &quot;page_count&quot;: 512,<br>            &quot;published_date&quot;: &quot;2020-07-10&quot;<br>        }<br>    },<br>    {<br>        &quot;model&quot;: &quot;books.book&quot;,<br>        &quot;pk&quot;: 3,<br>        &quot;fields&quot;: {<br>            &quot;title&quot;: &quot;Machine Learning 101&quot;,<br>            &quot;author&quot;: &quot;Dan Bader&quot;,<br>            &quot;page_count&quot;: 400,<br>            &quot;published_date&quot;: &quot;2022-01-05&quot;<br>        }<br>    }<br>]</pre><p>This fixture gives you some initial book records for testing your API. You can later load it with:</p><pre>$ python manage.py loaddata books.json</pre><p>This will populate the database with sample books so you can start building endpoints with Django REST framework.</p><p>This JSON contains database entries for three books. Call the following command to load this data into the database:</p><pre>$ python manage.py loaddata books.json<br>Installed 3 object(s) from 1 fixture(s)</pre><p>This adds three rows to the database.</p><p>With that, your Django application is now set up and populated with initial data. You can start integrating Django REST framework.</p><p>Django REST framework converts Django models into JSON for your REST API using <strong>serializers</strong>. A serializer defines how to transform a model instance into JSON and which fields to include.</p><p>Create a file called serializers.py inside the books application. Add the following code:</p><pre># books/serializers.py<br>from rest_framework import serializers<br>from .models import Book<br>class BookSerializer(serializers.ModelSerializer):<br>    class Meta:<br>        model = Book<br>        fields = [&quot;id&quot;, &quot;title&quot;, &quot;author&quot;, &quot;page_count&quot;, &quot;published_date&quot;]</pre><p>This BookSerializer subclasses serializers.ModelSerializer to automatically generate JSON from the Book model fields. By specifying the fields list, you control exactly which fields are exposed via the API.</p><p>Just like Django, Django <strong>REST framework</strong> uses <strong>views</strong> to query the database and display data via the API. Instead of manually writing each REST API view, you can subclass ModelViewSet, which comes with built-in views for common REST API operations.</p><blockquote><strong><em>Note:</em></strong><em> In DRF documentation, these built-in views are called </em><strong><em>actions</em></strong><em>.</em></blockquote><p>Here’s a table of ModelViewSet actions with their equivalent HTTP methods for a books API:</p><figure><img alt="HTTP-Method-Action Table" src="https://cdn-images-1.medium.com/max/679/1*m-TqDU5ntFn-4ZkSvXRhBQ.png" /><figcaption>REST API Actions Provided by Django REST Framework’s ModelViewSet for the Book Resource</figcaption></figure><p>As you can see, these actions map to the standard HTTP methods you’d expect in a REST API. You can <a href="https://www.django-rest-framework.org/api-guide/viewsets/#example">override these actions</a> in your subclass or <a href="https://www.django-rest-framework.org/api-guide/viewsets/#marking-extra-actions-for-routing">add additional actions</a> based on the requirements of your API.</p><p>As you can see, these actions map to the standard HTTP methods you’d expect in a REST API. You can override these actions in your subclass or add additional actions based on the requirements of your API.</p><p>Below is the code for a <strong>ModelViewSet</strong> subclass called BookViewSet. This class will generate the views needed to manage <strong>Book</strong> data. Add the following code to views.py inside the books application:</p><pre># books/views.py<br>from rest_framework import viewsets<br>from .models import Book<br>from .serializers import BookSerializer<br>class BookViewSet(viewsets.ModelViewSet):<br>    serializer_class = BookSerializer<br>    queryset = Book.objects.all()</pre><p>In this class, serializer_class is set to BookSerializer and queryset is set to Book.objects.all(). This tells Django REST framework which serializer to use and how to query the database for this specific set of views.</p><p>Once the views are created, they need to be mapped to the appropriate URLs or endpoints. To do this, Django REST framework provides a <strong>DefaultRouter</strong> that will automatically generate URLs for a ModelViewSet.</p><p>Create a urls.py file in the books application and add the following code to the file:</p><pre># books/urls.py<br>from django.urls import path, include<br>from rest_framework.routers import DefaultRouter<br>from .views import BookViewSet<br>router = DefaultRouter()<br>router.register(r&quot;books&quot;, BookViewSet)<br>urlpatterns = [<br>    path(&quot;&quot;, include(router.urls))<br>]</pre><p>This code creates a <a href="https://www.django-rest-framework.org/api-guide/routers/#defaultrouter"><strong>DefaultRouter</strong></a> and registers BookViewSet under the books URL. This will place all the URLs for BookViewSet under /books/.</p><blockquote><strong>Note:</strong> Django REST framework automatically appends a forward slash (<em>/</em>) to the end of any endpoints generated by <em>DefaultRouter</em>. You can disable this behavior like so:</blockquote><pre>router = DefaultRouter(trailing_slash=False)</pre><blockquote>This will disable the forward slash at the end of endpoints.</blockquote><p>Finally, you need to update the project’s base urls.py file to include all the <strong>books</strong> URLs in the project. Update the urls.py file inside your project folder (e.g., bookapi/urls.py) with the following code:</p><pre># bookapi/urls.py<br>from django.contrib import admin<br>from django.urls import path, include<br>urlpatterns = [<br>    path(&quot;admin/&quot;, admin.site.urls),<br>    path(&quot;&quot;, include(&quot;books.urls&quot;)),<br>]</pre><p>This puts all the URLs under /books/. Now you’re ready to try out your Django-backed REST API. Run the following command in the root bookapi directory to start the Django development server:</p><pre>$ python manage.py runserver</pre><p>The development server is now running. Go ahead and send a GET request to /books/ to get a list of all the books in your Django project:</p><pre>$ curl -i http://127.0.0.1:8000/books/ -w &#39;\n&#39;</pre><p>Example response:</p><pre>HTTP/3 200 OK<br>...<br>[<br>    {<br>        &quot;id&quot;: 1,<br>        &quot;title&quot;: &quot;The Pragmatic Programmer&quot;,<br>        &quot;author&quot;: &quot;Andrew Hunt&quot;,<br>        &quot;pages&quot;: 352<br>    },<br>    {<br>        &quot;id&quot;: 2,<br>        &quot;title&quot;: &quot;Clean Code&quot;,<br>        &quot;author&quot;: &quot;Robert C. Martin&quot;,<br>        &quot;pages&quot;: 464<br>    },<br>    {<br>        &quot;id&quot;: 3,<br>        &quot;title&quot;: &quot;Effective Python&quot;,<br>        &quot;author&quot;: &quot;Brett Slatkin&quot;,<br>        &quot;pages&quot;: 256<br>    }<br>]</pre><p>Django REST framework sends back a JSON response with the three books you added earlier. The response above is formatted for readability, so your response may look slightly different.</p><p>The DefaultRouter you created in books/urls.py provides URLs for requests to all the standard API endpoints:</p><ul><li>GET /books/ → list all books</li><li>GET /books/&lt;id&gt;/ → retrieve a single book</li><li>POST /books/ → create a new book</li><li>PUT /books/&lt;id&gt;/ → update all fields of a book</li><li>PATCH /books/&lt;id&gt;/ → update some fields of a book</li><li>DELETE /books/&lt;id&gt;/ → remove a book</li></ul><p>You can try out a POST request to /books/ to create a new book in your Django project:</p><pre>$ curl -i http://127.0.0.1:8000/books/ \<br>-X POST \<br>-H &#39;Content-Type: application/json&#39; \<br>-d &#39;{&quot;title&quot;:&quot;Python Crash Course&quot;, &quot;author&quot;:&quot;Eric Matthes&quot;, &quot;pages&quot;:544}&#39; \<br>-w &#39;\n&#39;</pre><p>Example response:</p><pre>HTTP/3 201 Created<br>...<br>{<br>    &quot;id&quot;: 4,<br>    &quot;title&quot;: &quot;Python Crash Course&quot;,<br>    &quot;author&quot;: &quot;Eric Matthes&quot;,<br>    &quot;pages&quot;: 544<br>}</pre><p>This creates a new <strong>Book</strong> with the JSON you sent in the request. Django REST framework returns a 201 Created status code along with the newly created book.</p><blockquote><strong>Note:</strong> By default, the response doesn’t include a new line at the end. This means that the JSON may run into your command prompt. The curl command above includes <em>-w &#39;\n&#39;</em> to add a newline character after the JSON to fix this issue.</blockquote><p>You can view an existing <strong>Book</strong> by sending a request to GET /books/&lt;book_id&gt;/ with an existing id. Run the following command to get the first Book:</p><pre>$ curl -i http://127.0.0.1:8000/books/1/ -w &#39;\n&#39;</pre><p>Example response:</p><pre>HTTP/3 200 OK<br>...<br>{<br>    &quot;id&quot;: 1,<br>    &quot;title&quot;: &quot;The Pragmatic Programmer&quot;,<br>    &quot;author&quot;: &quot;Andrew Hunt&quot;,<br>    &quot;pages&quot;: 352<br>}</pre><p>The response contains the information for the first <strong>Book</strong>.</p><p>These examples only covered <strong>GET</strong> and <strong>POST</strong> requests. Feel free to try out <strong>PUT</strong>, <strong>PATCH</strong>, and <strong>DELETE</strong> requests on your own to see how you can fully manage your model from the REST API.</p><h3>FastAPI</h3><p><a href="https://www.geeksforgeeks.org/python/fastapi-introduction/"><strong><em>FastAPI</em></strong></a> is a Python web framework optimized for building APIs. It uses Python type hints and has built-in support for async operations. FastAPI is built on top of <a href="https://www.starlette.io/"><strong>Starlette </strong></a>and <a href="https://pydantic-docs.helpmanual.io/"><strong>Pydantic </strong></a>and is very performant.</p><blockquote>What is an <em>async</em> function in Python, and how does it differ from a regular function? Give an example of its usage.</blockquote><blockquote>An <em>async</em> function in Python is a function defined using the <em>async def</em> syntax that allows asynchronous execution. It does not run immediately but returns a coroutine object. It can pause its execution at <em>await</em> expressions to let other code run concurrently, which is useful for I/O-bound tasks like network requests or file operations.</blockquote><p>It is a high-performance web framework for building HTTP-based service APIs in Python 3.8+. It uses Pydantic and type hints to validate, serialize, and deserialize data. FastAPI also automatically generates OpenAPI documentation for APIs built with it.</p><p>Below is an example of a REST API for books built with FastAPI:</p><pre># app.py<br>from fastapi import FastAPI<br>from pydantic import BaseModel, Field</pre><pre>app = FastAPI()</pre><pre>def _find_next_id():<br>    return max(book.book_id for book in books) + 1<br>class Book(BaseModel):<br>    book_id: int = Field(default_factory=_find_next_id, alias=&quot;id&quot;)<br>    title: str<br>    author: str<br>    pages: int<br>books = [<br>    Book(id=1, title=&quot;The Pragmatic Programmer&quot;, author=&quot;Andrew Hunt&quot;, pages=352),<br>    Book(id=2, title=&quot;Clean Code&quot;, author=&quot;Robert C. Martin&quot;, pages=464),<br>    Book(id=3, title=&quot;Deep Learning&quot;, author=&quot;Ian Goodfellow&quot;, pages=775),<br>]</pre><pre>@app.get(&quot;/books&quot;)<br>async def get_books():<br>    return books</pre><pre>@app.post(&quot;/books&quot;, status_code=201)<br>async def add_book(book: Book):<br>    books.append(book)<br>    return book</pre><p>This application uses FastAPI features to build a REST API for <strong>books</strong>, similar to how we handled countries in previous examples.</p><p>You can try this application by installing FastAPI with pip:</p><pre>$ python -m pip install fastapi</pre><p>You’ll also need to install uvicorn[standard], a server that can run FastAPI applications:</p><pre>$ python -m pip install uvicorn[standard]</pre><p>Once both FastAPI and Uvicorn are installed, save the code above in a file called app.py. Run the following command to start a development server:</p><pre>$ uvicorn app:app --reload<br>INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)</pre><p>The server is now running. Open up a browser and go to <a href="http://127.0.0.1:8000/books">http://127.0.0.1:8000/books</a>. You’ll see FastAPI respond with this:</p><pre>[<br>    {<br>        &quot;id&quot;: 1,<br>        &quot;title&quot;:&quot;The Pragmatic Programmer&quot;,<br>        &quot;author&quot;:&quot;Andrew Hunt&quot;,<br>        &quot;pages&quot;:352<br>    },<br>    {<br>        &quot;id&quot;: 2,<br>        &quot;title&quot;:&quot;Clean Code&quot;,<br>        &quot;author&quot;:&quot;Robert C. Martin&quot;,<br>        &quot;pages&quot;:464<br>    },<br>    {<br>        &quot;id&quot;: 3,<br>        &quot;title&quot;:&quot;Deep Learning&quot;,<br>        &quot;author&quot;:&quot;Ian Goodfellow&quot;,<br>        &quot;pages&quot;:775<br>    }<br>]</pre><p>FastAPI responds with a JSON array containing a list of books. You can also add a new book by sending a POST request to /books:</p><pre>$ curl -i http://127.0.0.1:8000/books \<br>-X POST \<br>-H &#39;Content-Type: application/json&#39; \<br>-d &#39;{&quot;title&quot;:&quot;Python Crash Course&quot;, &quot;author&quot;:&quot;Eric Matthes&quot;, &quot;pages&quot;:544}&#39; \<br>-w &#39;\n&#39;</pre><pre>HTTP/3 201 Created<br>content-type: application/json<br>...<br>{&quot;id&quot;:4,&quot;title&quot;:&quot;Python Crash Course&quot;,&quot;author&quot;:&quot;Eric Matthes&quot;,&quot;pages&quot;:544}</pre><p>You added a new book. You can confirm this with GET /books:</p><pre>$ curl -i http://127.0.0.1:8000/books -w &#39;\n&#39;</pre><pre>HTTP/3 200 OK<br>content-type: application/json<br>...<br>[<br>    {<br>        &quot;id&quot;:1,<br>        &quot;title&quot;:&quot;The Pragmatic Programmer&quot;,<br>        &quot;author&quot;:&quot;Andrew Hunt&quot;,<br>        &quot;pages&quot;:352<br>    },<br>    {<br>        &quot;id&quot;:2,<br>        &quot;title&quot;:&quot;Clean Code&quot;,<br>        &quot;author&quot;:&quot;Robert C. Martin&quot;,<br>        &quot;pages&quot;:464<br>    },<br>    {<br>        &quot;id&quot;:3,<br>        &quot;title&quot;:&quot;Deep Learning&quot;,<br>        &quot;author&quot;:&quot;Ian Goodfellow&quot;,<br>        &quot;pages&quot;:775<br>    },<br>    {<br>        &quot;id&quot;:4,<br>        &quot;title&quot;:&quot;Python Crash Course&quot;,<br>        &quot;author&quot;:&quot;Eric Matthes&quot;,<br>        &quot;pages&quot;:544<br>    }<br>]</pre><p>FastAPI returns a JSON list including the new book you just added.</p><p>You’ll notice that the FastAPI application looks similar to the Flask application. Like Flask, FastAPI has a focused feature set. It doesn’t try to handle all aspects of web application development. It’s designed to build APIs with modern Python features like type hints, async support, and automatic OpenAPI docs.</p><p>If you look near the top of app.py, you’ll see a class called Book that extends BaseModel. The Book class describes the structure of the data in the REST API:</p><pre>class Book(BaseModel):<br>    book_id: int = Field(default_factory=_find_next_id, alias=&quot;id&quot;)<br>    title: str<br>    author: str<br>    pages: int</pre><p>This is a <a href="https://pydantic-docs.helpmanual.io/usage/models/"><strong>Pydantic model</strong></a>. Pydantic models provide some super helpful features in FastAPI. They use Python type annotations to enforce the data type for each field. This allows FastAPI to automatically generate JSON with the correct data types for API endpoints, and also validate incoming JSON.</p><blockquote><strong>What is a Pydantic model in FastAPI, and why is it important?</strong></blockquote><blockquote>A <strong>Pydantic model</strong> is a Python class used to define <strong>data structures</strong> and enforce <strong>data validation</strong> in FastAPI applications. It uses Python <strong>type hints</strong> to automatically validate, serialize, and deserialize data. This ensures that the data received in requests or sent in responses has the correct type and format.</blockquote><blockquote><strong>Key Features of a Pydantic model:</strong></blockquote><blockquote><strong>1.Data Validation</strong>: Ensures the incoming JSON matches the expected types.</blockquote><blockquote><strong>2.Serialization / Deserialization</strong>: Converts Python objects to JSON and vice versa automatically.</blockquote><blockquote><strong>3.Default Values &amp; Aliases</strong>: Allows you to set defaults or output a field with a different JSON key.</blockquote><blockquote><strong>4.Error Reporting</strong>: Returns clear error messages with HTTP status code <em>422 Unprocessable Entity</em> if validation fails.</blockquote><p>Let’s zoom into the first line:</p><pre>book_id: int = Field(default_factory=_find_next_id, alias=&quot;id&quot;)</pre><ul><li>book_id stores an integer for the ID of the Book.</li><li><a href="https://pydantic-docs.helpmanual.io/usage/schema/#field-customisation"><strong>Field</strong></a><strong> </strong>from Pydantic modifies the behavior of book_id.</li><li>default_factory=_find_next_id specifies a function to run whenever a new Book is created. The return value of _find_next_id() will be assigned to book_id.</li><li>alias=&quot;id&quot; tells FastAPI to output the key &quot;id&quot; instead of &quot;book_id&quot; in the JSON:</li></ul><pre>{<br>    &quot;id&quot;: 1,<br>    &quot;title&quot;: &quot;The Pragmatic Programmer&quot;,<br>    &quot;author&quot;: &quot;Andrew Hunt&quot;,<br>    &quot;pages&quot;: 352<br>}</pre><p>This alias also allows you to use id when creating a new Book. You can see this in the initial books list:</p><pre>books = [<br>    Book(id=1, title=&quot;The Pragmatic Programmer&quot;, author=&quot;Andrew Hunt&quot;, pages=352),<br>    Book(id=2, title=&quot;Clean Code&quot;, author=&quot;Robert C. Martin&quot;, pages=464),<br>    Book(id=3, title=&quot;Deep Learning&quot;, author=&quot;Ian Goodfellow&quot;, pages=775),<br>]</pre><p>This list contains three instances of Book for the initial data in the API.</p><p>Now take a look at the two API functions in this application. The first, get_books(), returns a list of books for GET requests to /books:</p><pre>@app.get(&quot;/books&quot;)<br>async def get_books():<br>    return books</pre><p>FastAPI automatically creates JSON based on the fields in the Pydantic model and sets the correct JSON data types from the Python type hints.</p><p>The <strong>Pydantic model</strong> also provides a benefit when you make a POST request to /books. You can see in the second API function below that the parameter book has a Book annotation:</p><pre>@app.post(&quot;/books&quot;, status_code=201)<br>async def add_book(book: Book):<br>    books.append(book)<br>    return book</pre><p>This type annotation tells FastAPI to <strong>validate the incoming JSON against the Book model</strong>. If it doesn’t match, FastAPI will automatically return an error.</p><p>For example, try sending a POST request missing the pages field:</p><pre>$ curl -i http://127.0.0.1:8000/books \<br>-X POST \<br>-H &#39;Content-Type: application/json&#39; \<br>-d &#39;{&quot;title&quot;:&quot;The Clean Coder&quot;, &quot;author&quot;:&quot;Robert C. Martin&quot;}&#39; \<br>-w &#39;\n&#39;</pre><p>FastAPI responds with:</p><pre>HTTP/3 422 Unprocessable Entity<br>content-type: application/json<br>...<br>{<br>    &quot;detail&quot;: [<br>        {<br>            &quot;loc&quot;: [&quot;body&quot;,&quot;pages&quot;],<br>            &quot;msg&quot;: &quot;field required&quot;,<br>            &quot;type&quot;: &quot;value_error.missing&quot;<br>        }<br>    ]<br>}</pre><p>Here, the JSON request was missing a value for pages. FastAPI returned a <strong>422 Unprocessable Entity</strong> status code and provided detailed information about the error.</p><p>This validation is handled automatically thanks to the <strong>Pydantic model</strong>. You don’t have to write manual checks for required fields or data types — FastAPI does it for you.</p><p>FastAPI’s <strong>high performance</strong>, <strong>async support</strong>, and <strong>automatic documentation generation</strong> make it an excellent choice for modern REST APIs.</p><h3>Conclusion</h3><p>Building REST APIs in Python has never been more approachable. Whether you choose <strong>Flask</strong> for its simplicity, <strong>Django REST Framework</strong> for full-featured projects, or <strong>FastAPI</strong> for high-performance, modern applications, the core principles remain the same: <strong>design intuitive endpoints, validate incoming data, and respond consistently with clear status codes</strong>.</p><p>Throughout this article, we explored <strong>CRUD operations, error handling, data validation, and JSON serialization</strong>, highlighting best practices and practical examples. We also saw how frameworks like <strong>Pydantic and serializers</strong> simplify data validation and ensure your API is robust and maintainable.</p><p>By adhering to these standards and leveraging Python’s rich ecosystem, you can build APIs that are <strong>scalable, reliable, and developer-friendly</strong>, making it easier for clients and applications to interact with your services efficiently.</p><p>Ultimately, mastering REST API development is about <strong>clarity, consistency, and thoughtful design</strong> — tools and frameworks are just enablers. With the foundations laid here, you’re ready to build APIs that are not only functional but also elegant and professional.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3227ce4cc119" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/python-and-rest-apis-interacting-with-web-services-using-python-and-rest-apis-3227ce4cc119">The Art of Making the Internet Talk Back : Python × APIs</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>