<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ertuğrul Demir on Medium]]></title>
        <description><![CDATA[Stories by Ertuğrul Demir on Medium]]></description>
        <link>https://medium.com/@ertugruldemir?source=rss-c2ba54d15a24------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vpZN8RoCNs5fZslUDi5Gpg.jpeg</url>
            <title>Stories by Ertuğrul Demir on Medium</title>
            <link>https://medium.com/@ertugruldemir?source=rss-c2ba54d15a24------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 19:32:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ertugruldemir/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Skills over System Prompts: Building an Anki Tutor with the Antigravity SDK]]></title>
            <link>https://ertugruldemir.medium.com/skills-over-system-prompts-building-an-anki-tutor-with-the-antigravity-sdk-c237aafccb70?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/c237aafccb70</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[google-antigravity]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Fri, 26 Jun 2026 09:51:30 GMT</pubDate>
            <atom:updated>2026-06-26T09:51:30.446Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WniIu2hhz1xBNMVvj47jmQ.png" /></figure><p>AI has made me a little lazier.</p><p>Not dramatically lazy. Not “the robots will do everything” lazy. More like: once you get used to asking an agent to do boring work, every small manual workflow starts looking suspicious.</p><p>Anki is a perfect example.</p><p>Anki is great. I use it to remember things I study, subjects I work on, and the weird little decisions hidden inside codebases. Spaced repetition works. The problem is not Anki.</p><p>The problem is me.</p><p>I can already see the rot setting in. On complex cards, my brain starts negotiating with itself. “Yeah, I basically knew that.” “Close enough.” “I would have remembered it in context.” Then I press Good and move on.</p><p>That is not studying. That is self-certified vibes.</p><p>What I actually wanted was a study buddy sitting on top of my real Anki collection. Someone to ask the card, wait for my answer, reveal the real answer, compare it honestly, explain the gap, and only then help decide whether it was Again, Hard, Good, or Easy.</p><p>AI is annoyingly good for that.</p><p>It is also useful when taking over a new project. When I enter a repo, I do not only want a summary. I want to be quizzed later on the key decisions, the architecture, the gotchas, and the “why is it like this?” parts. Anki is great for that too.</p><p>But I am still lazy.</p><p>I am not going to manually write every card. I am not going to keep every deck updated by hand. And if I am studying from my phone, I am definitely not going to type long answers into a chat just so the agent can grade me. Voice needs to work too.</p><p>So the project quickly stopped being “connect Gemini to Anki.”</p><p>It became a small agent system:</p><ul><li>a terminal tutor for focused review sessions</li><li>a Telegram tutor for studying from my phone, including voice answers</li><li>a deck builder that creates cards from web research or a local codebase</li><li>a watch mode that can notice code changes and create cards while I work</li></ul><p>That is a lot of behavior.</p><p>My first instinct was the usual one: write a bigger system prompt. Tell the agent how to run a study session. Tell it how to write good flashcards. Tell it how to inspect a codebase and turn architecture into cards. Tell it how to behave differently in Telegram. Tell it not to touch scheduling unless I approve.</p><p>That works for about ten minutes.</p><p>Then the system prompt becomes a junk drawer.</p><p>The hard part was not giving the agent tools.</p><p>The hard part was giving it habits.</p><p>That is where the Google Antigravity SDK fit really well. It gives you the agent runtime as a Python library: custom tools, reusable skills, lifecycle hooks, safety policies, streaming, triggers, and multiple ways to run the same agent logic from different surfaces.</p><h3>What the Antigravity SDK Gives You</h3><p>The Antigravity SDK is not just a wrapper around a chat model.</p><p>It gives you programmatic access to the same agent runtime behind Google Antigravity 2.0 and the Antigravity CLI, but from Python.</p><p>That matters because a real agent is not only a model call. A real agent needs:</p><ul><li>tools</li><li>memory across turns</li><li>permissions</li><li>hooks</li><li>skills</li><li>streaming</li><li>triggers</li><li>safety around side effects</li></ul><p>The SDK puts those behind one main abstraction: Agent.</p><p>The smallest useful version really is tiny:</p><pre>import asyncio<br>from google.antigravity import Agent, LocalAgentConfig</pre><pre>async def main():<br>    config = LocalAgentConfig()<br>    async with Agent(config) as agent:<br>        response = await agent.chat(&quot;What files are in the current directory?&quot;)<br>        print(await response.text())</pre><pre>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(main())</pre><p>Install it with:</p><pre>pip install google-antigravity</pre><p>Then set a Gemini API key from Google AI Studio:</p><pre>export GEMINI_API_KEY=&quot;your-key-here&quot;</pre><p>That is the hello world.</p><p>The useful version starts when you compose the runtime features around a real workflow.</p><p>In this project, the Antigravity SDK pieces mapped like this:</p><ul><li><strong>Agent / </strong><strong>LocalAgentConfig</strong> — the terminal tutor, Telegram tutor, and deck builder all run on the same agent runtime</li><li><strong>Custom Python tools</strong> — AnkiConnect actions like get_due_cards, show_answer, rate_card, and add_notes</li><li><strong>skills_paths</strong> — shared review-buddy, plain-cards, and codebase-cards behavior packages</li><li><strong>Lifecycle hooks</strong> — sync on session start/end, deck backup before writes, audit log after scheduling changes, tool-error recovery</li><li><strong>Safety policies</strong> — practice mode blocks rate_card so cram sessions cannot change real scheduling</li><li><strong>Streaming</strong> — the deck builder prints progress while the agent researches and creates cards</li><li><strong>Triggers</strong> — watch mode reacts to .py file changes and asks the agent to card important changes</li><li><strong>Built-in read-only tools</strong> — codebase mode lets the agent inspect a repo without editing it</li></ul><p>That list is the reason this worked better as an SDK project than as one giant prompt around a model call.</p><p>Now, the first useful step: give the agent hands.</p><h3>Giving the Agent Hands: Anki as Python Tools</h3><p>Anki already has an HTTP API through the AnkiConnect add-on. The entire bridge is basically one POST to localhost:</p><pre>def invoke(action: str, **params):<br>    response = requests.post(<br>        &quot;http://localhost:8765&quot;,<br>        json={&quot;action&quot;: action, &quot;version&quot;: 6, &quot;params&quot;: params},<br>        timeout=30,<br>    )<br>    response.raise_for_status()<br>    payload = response.json()<br>    if payload[&quot;error&quot;]:<br>        raise RuntimeError(payload[&quot;error&quot;])<br>    return payload[&quot;result&quot;]</pre><p>From there, the agent tools are just normal Python functions.</p><p>A simplified version:</p><pre>def list_decks() -&gt; str:<br>    &quot;&quot;&quot;List all Anki decks with their due counts.&quot;&quot;&quot;<br>    decks = invoke(&quot;deckNames&quot;)<br>    stats = invoke(&quot;getDeckStats&quot;, decks=decks)<br>    return json.dumps(stats)</pre><pre>def get_due_cards(deck: str = &quot;&quot;, limit: int = 5) -&gt; str:<br>    &quot;&quot;&quot;Return due cards without revealing the answer side.&quot;&quot;&quot;<br>    query = f&#39;deck:&quot;{deck}&quot; is:due&#39; if deck else &quot;is:due&quot;<br>    card_ids = invoke(&quot;findCards&quot;, query=query)[:limit]<br>    cards = invoke(&quot;cardsInfo&quot;, cards=card_ids)<br>    return json.dumps(cards)</pre><pre>def rate_card(card_id: int, rating: int) -&gt; str:<br>    &quot;&quot;&quot;Submit a user-confirmed Anki rating: 1 Again, 2 Hard, 3 Good, 4 Easy.&quot;&quot;&quot;<br>    invoke(&quot;answerCards&quot;, answers=[{&quot;cardId&quot;: card_id, &quot;ease&quot;: rating}])<br>    return json.dumps({&quot;rated&quot;: card_id, &quot;rating&quot;: rating})</pre><p>Then register them with the SDK:</p><pre>from google.antigravity import LocalAgentConfig</pre><pre>config = LocalAgentConfig(<br>    tools=[list_decks, get_due_cards, rate_card],<br>)</pre><p>That is one of the nicest parts of the SDK: custom tools do not require a separate server. For this version, I did not need MCP, a framework, a schema generator, or a second process.</p><p>The agent can call plain Python.</p><p>In the real project I ended up with more tools:</p><ul><li>list_decks</li><li>get_due_cards</li><li>show_answer</li><li>rate_card</li><li>find_notes</li><li>add_note</li><li>add_notes</li><li>update_note</li><li>suspend_card</li><li>unsuspend_card</li><li>undo</li><li>get_stats</li><li>sync</li></ul><p>That was enough to make the tutor useful.</p><p>This is the first pattern:</p><blockquote><em>Put capabilities in tools.</em></blockquote><p>Tools are the agent’s hands. But hands are not behavior.</p><p>For behavior, I used skills.</p><h3>The Problem with Giant System Prompts</h3><p>At first, I tried to describe everything in the agent’s system instructions.</p><p>The tutor needs to know how to run a review session:</p><ol><li>show the question</li><li>wait for my answer</li><li>reveal the answer</li><li>compare my answer</li><li>suggest a rating</li><li>wait for confirmation</li><li>only then update Anki scheduling</li></ol><p>It also needs to know how to write good cards:</p><ul><li>one fact per card</li><li>answer-first backs</li><li>no trivia padding</li><li>no vague questions</li><li>no giant essay cards</li></ul><p>Then the deck builder needs another workflow:</p><ul><li>research a topic</li><li>extract the important facts</li><li>create cards</li><li>verify they exist in Anki</li></ul><p>Then the codebase deck builder needs a different workflow:</p><ul><li>inspect the repo breadth-first</li><li>find key abstractions</li><li>explain responsibilities and data flow</li><li>avoid making cards for random syntax</li></ul><p>Then Telegram needs shorter replies because nobody wants a wall of Markdown on a phone.</p><p>You can put all of that into one system prompt.</p><p>But you should not.</p><p>A giant system prompt has three problems:</p><ol><li><strong>It pollutes every task.</strong> The agent is thinking about codebase exploration while you are reviewing Spanish verbs.</li><li><strong>It is hard to reuse.</strong> The same card-writing rules need to appear in the terminal tutor, Telegram tutor, and deck builder.</li><li><strong>It rots.</strong> Every new behavior gets pasted into the same blob until nobody knows which rule controls what.</li></ol><p>This is exactly the problem skills solve.</p><p>The shape changed from this:</p><pre>system prompt = tutor rules<br>              + card-writing rules<br>              + codebase-exploration rules<br>              + Telegram style rules<br>              + safety reminders<br>              + whatever I forgot last week</pre><p>Into this:</p><pre>system prompt = identity + hard safety floor<br>review-buddy   = study-session behavior<br>plain-cards    = card-writing behavior<br>codebase-cards = repo-exploration behavior<br>hooks/policies = enforcement and receipts</pre><p>That is the real pattern behind the title.</p><p>Not “make the prompt better.”</p><p>Make the prompt smaller.</p><h3>Skills over System Prompts</h3><p>A skill is a folder with a SKILL.md file inside it.</p><p>My project has three:</p><pre>.agents/skills/<br>  plain-cards/<br>    SKILL.md<br>  review-buddy/<br>    SKILL.md<br>  codebase-cards/<br>    SKILL.md</pre><p>Each skill starts with a tiny bit of frontmatter.</p><p>For example, the review skill begins like this:</p><pre>---<br>name: review-buddy<br>description: Playbook for running an interactive Anki review session - quiz one card at a time, grade recall together, submit ratings, repair noisy or broken cards.<br>---</pre><p>That description is not just documentation for humans. It is the lightweight discovery layer. The agent can see what skills exist, then load the full instructions only when the task calls for them.</p><p>A skill is not a service. It is not an MCP server. It is not a deployment. It is a behavior package sitting on disk, ready to be pulled into the agent when needed.</p><p>Then the SDK loads the skill directory:</p><pre>config = LocalAgentConfig(<br>    system_instructions=SYSTEM_INSTRUCTIONS,<br>    tools=ALL_TOOLS,<br>    skills_paths=[&quot;.agents/skills&quot;],<br>)</pre><p>The key idea is simple:</p><blockquote><em>The system prompt says who the agent is. Skills say what job it is currently doing.</em></blockquote><p>For this project, the system prompt stays small. It says the agent is a friendly flashcard tutor working with a real Anki collection.</p><p>The details live in skills.</p><h3>review-buddy: the study session playbook</h3><p>This skill describes how to run a review session.</p><p>It covers the rhythm:</p><ul><li>ask one card at a time</li><li>hide the answer until the user attempts it</li><li>reveal and teach briefly</li><li>suggest a rating</li><li>wait for confirmation</li><li>handle noisy or broken cards</li><li>close with a recap</li></ul><p>This is not code. It is behavioral protocol.</p><p>That distinction matters. The review flow is not tied to terminal I/O, Telegram messages, or AnkiConnect. It is just the way a good tutor should behave.</p><h3>plain-cards: the card-writing style guide</h3><p>This skill handles card quality.</p><p>It tells the agent to write cards that are:</p><ul><li>atomic</li><li>answer-first</li><li>lean</li><li>verified</li><li>free of filler</li><li>easy to review months later</li></ul><p>A bad flashcard is worse than no flashcard. It creates fake progress. The model can generate ten cards in seconds, but without a style guide it will happily generate ten vague cards that future me will hate.</p><p>So card writing became a skill.</p><h3>codebase-cards: the repo exploration protocol</h3><p>This one is for turning source code into Anki cards.</p><p>The agent is told to inspect the repo breadth-first, identify architecture, data flow, responsibilities, and gotchas, then turn only the useful findings into cards.</p><p>That skill powers code mode in the deck builder:</p><pre>python deck_builder.py &quot;overall architecture&quot; --path ~/my/project --count 6</pre><p>The focus hint changes, but the exploration protocol stays the same.</p><p>This is the second pattern:</p><blockquote><em>Put reusable behavior in skills.</em></blockquote><p>Not in the system prompt. Not duplicated across entrypoints. Not buried in Python conditionals.</p><p>A skill is just a file, but it changes the shape of the whole project.</p><h3>One Behavior Layer, Three Surfaces</h3><p>Once the behavior lived in skills, adding new surfaces became much easier.</p><p>The architecture looked like this:</p><pre>                  .agents/skills/<br>          ┌──────────────┼──────────────┐<br>          │              │              │<br>     review-buddy   plain-cards   codebase-cards<br>          │              │              │<br>          └──────────────┼──────────────┘<br>                         │<br>                  LocalAgentConfig<br>                         │<br>       ┌─────────────────┼─────────────────┐<br>       │                 │                 │<br>  terminal tutor   Telegram tutor    deck builder<br>    tutor.py      telegram_tutor.py  deck_builder.py</pre><p>The terminal tutor is the simplest surface:</p><pre>async with Agent(config) as agent:<br>    await run_interactive_loop(agent)</pre><p>The Telegram tutor uses the same agent differently:</p><pre>async def chat_response(agent: Agent, prompt: str) -&gt; str:<br>    response = await agent.chat(prompt)<br>    return &quot;&quot;.join([token async for token in response])</pre><p>The deck builder streams output as it works:</p><pre>response = await agent.chat(message)<br>async for token in response:<br>    print(token, end=&quot;&quot;, flush=True)</pre><p>Different surfaces. Same runtime. Same skills.</p><p>That is the part I liked most. Telegram did not need a copied review prompt. The deck builder did not need its own card-writing manifesto. The codebase mode did not need a separate app-specific doctrine.</p><p>They all loaded the same skill directory.</p><h3>The Terminal Tutor</h3><p>The terminal version is the baseline.</p><p>Start Anki, run the tutor, and ask naturally:</p><pre>python tutor.py</pre><p>Then:</p><pre>quiz me on XYZ</pre><p>The tutor lists due cards, asks one question, waits for my answer, reveals the real Anki answer, compares, teaches, and suggests a rating.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*WKTLoBKkU6of4cwj.png" /></figure><p>The review loop is human-in-the-loop by design:</p><pre>Agent: I would rate this Good (3). You had the main idea but missed the date.<br>User: yes<br>Agent: rated 3. Next card...</pre><p>Or I can override it:</p><pre>Agent: I would rate this Hard (2).<br>User: actually 1<br>Agent: rated Again (1). Let&#39;s reinforce it.</pre><p>Spaced repetition is stateful. A bad rating affects the future schedule. So the model can suggest, but I decide.</p><p>That is not just a prompt preference. It is the product boundary.</p><h3>The Telegram Tutor</h3><p>The second surface was Telegram.</p><p>Not because Telegram is fancy. Because the best study app is the one I actually open.</p><p>The Telegram bot long-polls the Bot API, sends messages into the same Antigravity agent, and returns the response. It also supports voice notes: speak the answer, transcribe it, and feed the transcript back into the tutor as text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/505/0*AOeP8KuQ29Y4ToNi.png" /></figure><p>The agent gets a small extra instruction:</p><pre>TELEGRAM_INSTRUCTIONS = &quot;&quot;&quot;<br>You are chatting through Telegram on a phone. Keep replies short and plain<br>text only - no markdown headers, tables, or code fences. One card per message.<br>&quot;&quot;&quot;</pre><p>Everything else stays shared.</p><p>Same Anki tools. Same hooks. Same skills.</p><p>I also added due-card nudges without spending model tokens. Every 30 minutes, plain Python checks Anki deck counts. If cards are waiting, the bot sends a short reminder:</p><pre>25 cards waiting (X 5, Y 8). Say &#39;quiz me&#39; to start.</pre><p>No LLM needed. No reasoning needed. Just deterministic code.</p><p>This became a useful design rule:</p><blockquote><em>Do not use the model for work a </em><em>for loop can do.</em></blockquote><p>The agent is for tutoring. The nudge is just a counter.</p><h3>The Deck Builder</h3><p>The third surface is a deck builder.</p><p>It has two modes.</p><p>Web mode:</p><pre>python deck_builder.py &quot;Ottoman Empire&quot; --deck &quot;History&quot; --count 8</pre><p>Codebase mode:</p><pre>python deck_builder.py &quot;error handling and edge cases&quot; --path ~/my/project --count 6</pre><p>Web mode gives the agent a small research toolset: Wikipedia search, Wikipedia read, and URL fetch. Then it asks the agent to create cards using the plain-cards skill.</p><p>Codebase mode is more interesting. The SDK can give the agent built-in file tools scoped to a workspace. I enabled read-only access:</p><pre>from google.antigravity.types import BuiltinTools, CapabilitiesConfig</pre><pre>config = LocalAgentConfig(<br>    tools=[add_notes, list_decks],<br>    workspaces=[code_path],<br>    capabilities=CapabilitiesConfig(<br>        enabled_tools=BuiltinTools.read_only()<br>    ),<br>    skills_paths=[&quot;.agents/skills&quot;],<br>)</pre><p>That means the agent can inspect the target repo, but not edit it.</p><p>For a deck builder, that is the right permission boundary. It needs to read code and create Anki notes. It does not need to modify the project.</p><p>This is where codebase-cards activates. The agent explores the repo, identifies the concepts worth remembering, then writes cards through add_notes.</p><p>At the end, I do not trust the model’s narration. The script queries Anki to verify the cards exist.</p><pre>def cards_in_anki(deck: str) -&gt; int:<br>    result = json.loads(find_notes(f&#39;deck:&quot;{deck}&quot; tag:auto-researched&#39;, 100))<br>    return len(result) if isinstance(result, list) else 0</pre><p>If the model says it created cards but Anki has zero, the script nudges it to try again.</p><p>That became another rule:</p><blockquote><em>Trust the system receipt, not the model narration.</em></blockquote><h3>Turning It Ambient with Triggers</h3><p>The SDK also supports triggers: background tasks that react to external events and push messages into the agent.</p><p>I used a file-change trigger for codebase card generation.</p><p>The idea: while I work on a project, if a Python file changes, the agent can inspect the change and decide whether it introduced something worth remembering.</p><p>Simplified:</p><pre>from google.antigravity.triggers import on_file_change</pre><pre>def make_watch_trigger(path, deck, tag):<br>    async def on_change(ctx, changes):<br>        paths = sorted({c.path for c in changes if c.path.endswith(&quot;.py&quot;)})<br>        if not paths:<br>            return<br>        await ctx.send(<br>            f&quot;These files changed: {&#39;, &#39;.join(paths)}. &quot;<br>            f&quot;Create cards in deck {deck} if the change is worth remembering.&quot;<br>        )<br>    return on_file_change(path, on_change)</pre><p>Run it like this:</p><pre>python deck_builder.py &quot;as I work&quot; --path ~/my/project --watch</pre><p>This is where the project started feeling less like a chatbot and more like a sidecar.</p><p>I edit code. The trigger wakes the agent. The codebase skill tells it how to inspect the change. The card-writing skill tells it how to write good cards. The Anki tool creates the notes.</p><p>No new server. No custom scheduler. No giant prompt.</p><p>Just SDK triggers plus skills.</p><h3>The Part I Refused to Trust to the Model</h3><p>Skills are guidance.</p><p>Policies and hooks are enforcement.</p><p>That line is the difference between a fun demo and a tool I can leave connected to my real Anki collection.</p><p>The Antigravity SDK has declarative safety policies and lifecycle hooks. I used both.</p><h3>Practice mode blocks scheduling writes</h3><p>Sometimes I want to cram without touching Anki scheduling.</p><p>A prompt instruction is not enough for that. If the agent forgets and calls rate_card, the schedule changes.</p><p>So practice mode denies the tool at the harness level:</p><pre>from google.antigravity.hooks import policy</pre><pre>policies = policy.confirm_run_command()<br>if practice_mode:<br>    policies = policies + [<br>        policy.deny(&quot;rate_card&quot;, name=&quot;practice_mode&quot;)<br>    ]</pre><p>Now rate_card is blocked even if the model tries to call it.</p><p>That is the kind of safety I want: not vibes, not trust, not “please don’t”. A runtime boundary.</p><h3>Hooks sync, back up, audit, and recover</h3><p>The SDK hook system lets you observe or intervene at lifecycle points.</p><p>I used session hooks to sync Anki:</p><pre>@hooks.on_session_start<br>async def sync_on_start():<br>    sync_anki()</pre><pre>@hooks.on_session_end<br>async def sync_on_end():<br>    sync_anki()</pre><p>I used a pre-tool-call Decide hook to back up a deck before note writes:</p><pre>@hooks.pre_tool_call_decide<br>async def backup_before_note_writes(tool_call):<br>    if tool_call.name in (&quot;add_note&quot;, &quot;add_notes&quot;):<br>        backup_deck(tool_call.args[&quot;deck&quot;])<br>    return hooks.HookResult(allow=True)</pre><p>I used a post-tool-call Inspect hook to audit scheduling changes:</p><pre>@hooks.post_tool_call<br>async def audit_scheduling_changes(result):<br>    if result.name in {&quot;rate_card&quot;, &quot;undo&quot;, &quot;suspend_card&quot;, &quot;unsuspend_card&quot;}:<br>        append_jsonl(&quot;backups/scheduling_audit.jsonl&quot;, result)</pre><p>And I used a Transform hook to turn ugly tool errors into recovery hints the model can act on:</p><pre>@hooks.on_tool_error<br>async def recover_from_tool_error(error):<br>    if isinstance(error, requests.Timeout):<br>        return &quot;AnkiConnect timed out. Ask the user to check Anki, then retry.&quot;<br>    return None</pre><p>This is one of the strongest parts of the SDK.</p><p>The model does not need to remember to audit itself. The harness does it.</p><p>The model does not need to remember to back up a deck before writing. The hook does it.</p><p>The model does not get to bypass practice mode. The policy blocks it.</p><p>The pattern became clear:</p><ul><li><strong>tools</strong> give the agent capabilities</li><li><strong>skills</strong> give the agent reusable behavior</li><li><strong>policies</strong> define what must never happen</li><li><strong>hooks</strong> add system-level guarantees around the agent</li></ul><p>That separation is the architecture.</p><h3>What Worked</h3><p>A few things worked better than expected.</p><h3>Plain Python tools were enough</h3><p>I originally thought I might need to build an MCP server immediately.</p><p>I did not.</p><p>For one application, custom Python functions were simpler. The SDK already knows how to expose them as tools. That kept the first version small.</p><p>MCP is still useful when you want the same tools available across multiple clients. But for an SDK-native app, Python functions are the shortest path.</p><h3>Skills kept the project from becoming prompt soup</h3><p>This was the biggest win.</p><p>The base system instructions stayed focused. The detailed workflows moved into skills.</p><p>When I improved card-writing rules, terminal, Telegram, and deck builder all benefited. I did not need to update three prompts.</p><h3>Hooks made side effects less scary</h3><p>Anki is not a toy database. It is my real spaced-repetition schedule.</p><p>The hooks gave me a deterministic layer around model behavior:</p><ul><li>sync at session boundaries</li><li>backup before writes</li><li>audit after scheduling changes</li><li>recover from tool failures</li></ul><p>That made the agent feel much less like a random chatbot with database access.</p><h3>Triggers changed the feel of the app</h3><p>The file watcher was small, but it changed the mental model.</p><p>The agent was no longer only something I talked to. It could react to work happening around it.</p><p>That is where SDK agents get interesting: not just chat, but event-driven labor.</p><h3>What Did Not Work Perfectly</h3><p>A few caveats.</p><h3>Skills are not hard guarantees</h3><p>Skills are instructions. They improve behavior, but they are still model-read guidance.</p><p>If something must be impossible, use a policy or remove the tool.</p><p>That is why practice mode denies rate_card instead of merely asking the model not to call it.</p><h3>AnkiConnect has sharp edges</h3><p>AnkiConnect is simple, but it has quirks.</p><p>For example, answerCards can return success even for bad card IDs unless you pre-check the card. Some note updates silently fail if the note is open in Anki&#39;s browser window. AnkiConnect also runs inside Anki&#39;s Qt process, so you should not treat it like a high-concurrency API.</p><p>The fix is boring and important: validate inside tools.</p><h3>Voice was simpler outside the agent loop</h3><p>The Telegram bot supports voice answers, but I kept transcription outside the agent loop. A direct Gemini transcription call turns the voice note into text, then the transcript goes into the tutor.</p><p>That was simpler and more reliable for this build.</p><p>The lesson: use the SDK where it makes the architecture cleaner. Do not force every feature through the agent if a direct call is simpler.</p><h3>How to Build Something Similar</h3><p>If you want to build your own version of this pattern, I would do it in this order.</p><h3>1. Start with one real workflow</h3><p>Do not start with a platform.</p><p>Pick one annoying workflow with real state behind it:</p><ul><li>flashcards</li><li>GitHub issues</li><li>CRM updates</li><li>personal knowledge base</li><li>support tickets</li><li>finance records</li></ul><p>The state matters. Agents get interesting when they can act on something real.</p><h3>2. Wrap the system as small Python tools</h3><p>Keep the tools boring.</p><pre>def search_items(query: str) -&gt; str:<br>    &quot;&quot;&quot;Search the user&#39;s records.&quot;&quot;&quot;<br>    ...</pre><pre>def create_item(title: str, body: str) -&gt; str:<br>    &quot;&quot;&quot;Create a new record after user approval.&quot;&quot;&quot;<br>    ...</pre><p>Register them:</p><pre>config = LocalAgentConfig(<br>    tools=[search_items, create_item],<br>)</pre><p>Make tools validate inputs. Do not rely on the model to pass perfect IDs.</p><h3>3. Move task behavior into skills</h3><p>Create a skill folder:</p><pre>.agents/skills/my-workflow/SKILL.md</pre><p>A minimal skill:</p><pre>---<br>name: my-workflow<br>description: Use when helping the user process and update records in this system.<br>---</pre><pre># My Workflow</pre><pre>1. Inspect the current record before changing it.<br>2. Propose the change in plain language.<br>3. Wait for user confirmation before writing.<br>4. After writing, verify the record exists.</pre><p>Then load it:</p><pre>config = LocalAgentConfig(<br>    tools=TOOLS,<br>    skills_paths=[&quot;.agents/skills&quot;],<br>)</pre><p>This is the move: do not keep growing the system prompt forever.</p><h3>4. Add policies for non-negotiables</h3><p>If a tool should never run in a mode, deny it.</p><pre>policies = [<br>    policy.deny(&quot;delete_record&quot;, name=&quot;no_deletes&quot;),<br>]</pre><p>If shell execution should require confirmation, keep the default guard:</p><pre>policies = policy.confirm_run_command()</pre><p>The model can misunderstand a skill. It cannot ignore a denied tool.</p><h3>5. Add hooks for receipts</h3><p>Use hooks for things that should happen regardless of whether the model remembers them:</p><ul><li>audit logs</li><li>backups</li><li>sync</li><li>metrics</li><li>sanitization</li><li>error recovery</li></ul><pre>@hooks.post_tool_call<br>async def audit(result):<br>    write_log({<br>        &quot;tool&quot;: result.name,<br>        &quot;result&quot;: result.result,<br>        &quot;error&quot;: result.error,<br>    })</pre><h3>6. Add another surface only after the behavior is reusable</h3><p>Once the behavior lives in tools and skills, a second surface becomes much cheaper.</p><p>Terminal first. Then Telegram, Slack, web, cron, or file triggers.</p><p>The surface should be thin. The agent behavior should not live there.</p><h3>The Bigger Point</h3><p>The old way to build an AI feature was to write a large prompt and hope the model followed it.</p><p>That is not enough for real agents.</p><p>A real agent needs separation of concerns:</p><pre>Capabilities       -&gt; tools<br>Reusable behavior  -&gt; skills<br>Hard boundaries    -&gt; policies<br>System guarantees  -&gt; hooks<br>External events    -&gt; triggers<br>User interface     -&gt; thin surface</pre><p>This is what the Antigravity SDK made pleasant. I could build one agent runtime and reuse it across terminal, Telegram, and deck generation. I could keep the tutoring behavior in SKILL.md files instead of duplicating it. I could wrap real side effects with policies and hooks instead of trusting the model to behave.</p><p>The Anki tutor is just the concrete example.</p><p>The pattern generalizes.</p><p>A support agent could keep triage behavior in a skill, expose ticket updates as tools, deny destructive writes by policy, and audit every status change by hook.</p><p>A code review agent could keep review rubrics in skills, expose GitHub as tools, require approval before comments, and verify every posted review.</p><p>A research agent could keep extraction protocols in skills, use file triggers to process new papers, and write structured outputs only after validation.</p><p>The skill is the portable behavior module.</p><p>The SDK is the harness that lets it act.</p><h3>Resources</h3><ul><li>Google Antigravity SDK GitHub repository: <a href="https://github.com/google-antigravity/antigravity-sdk-python">https://github.com/google-antigravity/antigravity-sdk-python</a></li><li>Google Antigravity: <a href="https://antigravity.google/">https://antigravity.google/</a></li><li>Antigravity SDK overview: <a href="https://antigravity.google/docs/sdk-overview">https://antigravity.google/docs/sdk-overview</a></li><li>Google AI Studio API keys: <a href="https://aistudio.google.com/apikey">https://aistudio.google.com/apikey</a></li><li>Anki: <a href="https://apps.ankiweb.net/">https://apps.ankiweb.net/</a></li></ul><h3>Closing</h3><p>I started this because I was too lazy to open Anki.</p><p>That sounds like a joke, but most useful automation starts there. Not with a grand platform vision. With a small workflow that keeps not happening because the friction is just high enough.</p><p>The surprising part was not that an LLM could quiz me.</p><p>The surprising part was how clean the architecture became.</p><p>Tools gave the agent hands. Skills gave it habits. Policies gave it boundaries. Hooks gave it receipts. Triggers made it wake up when something changed.</p><p>That is the version of agents I trust more: not one giant prompt pretending to be an application, but a small runtime with clear layers.</p><p>The future of agent apps is not monolithic complex systems.</p><p>It is smaller prompts, sharper tools, reusable skills, and a harness that refuses to let the model pretend a side effect happened when it did not.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c237aafccb70" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI Tutorial Assistant for Europa Universalis V]]></title>
            <link>https://ertugruldemir.medium.com/building-an-ai-tutorial-assistant-for-europa-universalis-v-e8f7928603ce?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/e8f7928603ce</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[gaming]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Sun, 14 Dec 2025 11:41:01 GMT</pubDate>
            <atom:updated>2025-12-14T11:41:01.732Z</atom:updated>
            <content:encoded><![CDATA[<h4>How I tackled the steepest learning curve in strategy gaming using Google ADK, Gemini Pro 3, and the Interactions API</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UkgaineKKQw_7V-sDN1qUA.png" /></figure><p>It’s 1337. I’m commanding a medieval kingdom at the height of its power, just before the bubonic plague changes everything. I’ve just declared war on a neighboring realm, and I have no idea if my army composition is going to get me massacred or not.</p><p>I could pause the game and alt-tab to google it. Again. For the fifteenth time this session. Or I could just… ask.</p><p>That’s the dream, anyway. Europa Universalis V is one of those games where even as a developer — someone who literally builds things for a living — the learning curve feels insurmountable. The in-game tutorial shows you the basics. Everything else? Wiki-diving, YouTube rabbit holes, and trial-by-fire defeats.</p><p>I love this game, but I was spending more time pausing and alt-tabbing to research than actually playing. So I did what developers do: I automated the annoying part.</p><h3>The Problem: Information Overload</h3><p>EU5 is deceptively complex. The tutorial teaches you how to move armies and collect taxes. It doesn’t teach you optimal army composition for different tech levels, when to take loans versus raise war taxes, or why your trade income just tanked after you conquered that province.</p><p>The official wiki has the answers. So do dozens of YouTube guides. But accessing them means breaking flow state, searching, context-switching, and trying to remember what you were doing when you finally find the answer.</p><p>I needed something that could:</p><ul><li>Answer questions without breaking flow</li><li>Pull from the entire wiki’s knowledge base</li><li>Search YouTube for meta strategies and edge-case tactics</li><li>Remember context across multiple questions in a session so it could follow my campaign.</li></ul><p>Basically, an AI agent that understood EU5 as well as the wiki contributors do.</p><h3>The Stack: ADK + Gemini 3 + Interactions API</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*licUpDN0WH9fQnuZvmZ_ZA.png" /></figure><p><strong>YouTube Data API v3</strong>: For fetching video guides and meta content. Requires an API key from Google Cloud Console, but it’s free tier is generous enough for personal use.</p><p><strong>Agent Development Kit (ADK)</strong>: Google’s framework for building multi-agent systems. It handles the orchestration — tool selection, routing between agents, managing execution flow. Clean abstraction layer that lets you focus on what your agent does rather than how it coordinates. Bonus: adk web spins up a ready-to-use frontend for testing and interacting with your agent without writing any UI code.</p><p><strong>Gemini 3 Pro</strong>: The actual reasoning engine. I tested both Gemini 2.5 and 3, and the difference was noticeable. Gemini 3 synthesized game mechanics much better, especially for complex queries like “why did this battle go badly” where it needs to connect multiple wiki concepts.</p><p><strong>Interactions API</strong>: This is brand new (just launched) and designed specifically for stateful, multi-turn agentic workflows. The key difference from the old generateContent approach: it manages conversation state server-side.</p><p>What this means practically: the agent remembers our conversation history without me having to manually track and send the full message chain every time. When I ask “what about heavy cavalry instead?” it knows we’re still discussing army composition from three questions ago. The API also has native support for “thoughts” (the agent’s reasoning process) separate from its final response, and can handle long-running background tasks without timeout issues.</p><p>Basically, it’s built for exactly this use case — persistent assistants that follow complex, branching conversations.</p><h3>Setting Up the Knowledge Base</h3><p><strong>API Keys</strong>: Created a Gemini API key from AI Studio and a YouTube Data API v3 key from Google Cloud Console. Both stored as environment variables.</p><p><strong>Wiki Scraping</strong>: Wrote a Python scraper to recursively download all EU5 wiki pages as structured JSON. The scraper extracts page content, tables, internal links, and metadata — everything needed to answer game mechanics questions.</p><p>Rather than using semantic search or live URL fetching, I opted to download everything upfront. This gives the agent complete offline access to the wiki and faster lookups.</p><p><strong>Indexing</strong>: After scraping ~500 pages, I used Gemini Flash (cheaper/faster model) to generate descriptions and keywords for each page, creating a searchable JSON index. The agent queries this index first to find relevant pages, then reads the full content.</p><p>Looking at your agent code, here’s a concise technical explanation for the blog:</p><h3>The Agent Architecture</h3><p>The EU5 assistant is built around a <strong>deep research pattern</strong> — it doesn’t just answer questions, it investigates them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4uvHC3AO7Af6w1fajHpkqw.png" /></figure><p><strong>Core Components:</strong></p><p><strong>Knowledge Base</strong>: ~500 scraped wiki pages plus 10,000+ rows of raw game data (Grotaclas tables). The wiki pages explain mechanics, the Grotaclas data contains exact values pulled from game files.</p><p><strong>Tool Arsenal</strong>: The agent has 20+ tools organized by purpose:</p><ul><li><strong>Navigation</strong>: get_page_index, get_linked_pages (explore wiki structure)</li><li><strong>Data access</strong>: get_grotaclas_data (raw game stats), get_page_content (full articles)</li><li><strong>Extraction</strong>: extract_modifiers, extract_requirements (regex parsing for values in prose)</li><li><strong>Search</strong>: search_wiki, search_all_for_modifier (exhaustive lookups)</li><li><strong>External</strong>: YouTube search and video analysis agents</li></ul><p><strong>YouTube Integration</strong>: Two specialized capabilities here:</p><ol><li><strong>Video Search</strong> (search_eu5_videos): Uses YouTube Data API v3 to find relevant guides and strategies. Filters to videos under 30 minutes and batches metadata requests to minimize API quota usage.</li><li><strong>Video Analysis Agent</strong>: A sub-agent that uses Gemini’s native video understanding to extract strategies, mechanics, and specific numbers from gameplay videos. It looks for undocumented mechanics and meta strategies not in the wiki.</li></ol><p><strong>Research Protocol</strong>: The system prompt enforces a multi-pass approach. For complex questions, it must:</p><ol><li>Check the page index to find relevant content</li><li>Fetch 3–5 related pages (not just one)</li><li>Follow internal links to connected mechanics</li><li>Pull exact numbers from Grotaclas tables</li><li>Search for one short YouTube video to visualize complex mechanics</li><li>Cross-reference all sources</li></ol><p><strong>The Interactions API Bug</strong>: During development, I hit an issue where ADK tools with zero parameters caused the Interactions API to throw invalid_request errors. The API expects every tool to have at least one parameter in its JSON schema—passing an empty parameter list breaks schema validation.</p><p>My workaround: added dummy parameters (like refresh_cache: bool or show_details: bool) to tools that didn&#39;t naturally need arguments. The parameters are ignored in the function body, but they satisfy the schema requirement. Not elegant, but it unblocked me. I reported this upstream and it&#39;ll likely get an official fix, but as of writing, you need at least one parameter per tool when using ADK with Interactions API.</p><p><strong>The Key Insight</strong>: Simple questions get direct answers. Complex questions trigger deep research — the agent explores the knowledge graph, validates numbers across sources, and synthesizes comprehensive answers with full citations.</p><p>Example: “How do I maximize discipline?” triggers searches across government reforms, estates, religions, holy sites, cultures, advances (2500+ tech options), buildings, and more. It returns every source with exact values, not just top results.</p><h3>Meta-Gaming with AI</h3><p>Here’s where it got interesting. I was playing as the Karamanids — a small Turkish beylik in Anatolia, squeezed between the Ottomans and Mamluks. Not exactly a powerhouse. I had gold but limited manpower, so I needed maximum damage per regiment.</p><p>I asked the agent: <strong>“Give me a meta high-damage doomstack composition for the Age of Reformation.”</strong></p><p>This is genuinely hard. Army composition in EU5 depends on unit stats, combat phases, terrain, and era-specific mechanics like the Bombard Phase. Even experienced players debate optimal builds.</p><p>The agent synthesized data from unit type tables, combat mechanics pages, and YouTube meta guides. Its recommendation:</p><p><strong>The “Doomstack” Composition:</strong></p><ul><li><strong>100% Cavalry frontline</strong> (Pistoleers/Hussars) — 4.0 Combat Power, 4.0 Initiative</li><li><strong>Artillery backline</strong> (Falconets) — 5.0 Combat Power for the Bombard Phase</li><li><strong>Minimal/zero infantry</strong> — statistically inferior in this era</li></ul><p>The reasoning: Cavalry’s high initiative (4.0 vs infantry’s 1.0) means they attack 4x more often per engagement phase and deal 3x damage per hit. Artillery dominates the opening Bombard Phase with 5.0 Combat Power and 20% efficiency bonus.</p><p>I tried it. <strong>It worked shockingly well.</strong></p><p>Here are some of my conversations with the AI assistant:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v-grd_asGU3AE1Koavhb1g.png" /></figure><p>The agent didn’t just give me unit names — it explained <em>why</em> this composition exploits Age of Reformation combat mechanics, how to deploy the “Stackwipe Split” formation, and even caught edge cases like Spain’s Tercios being an exception due to unique morale modifiers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*XRoixTn1X2zLnB5VAey9dg.png" /></figure><p>This is meta-knowledge I wouldn’t have discovered through casual wiki reading. Having instant access to synthesized, cross-referenced game data while playing is transformative.</p><h3>It Actually Works</h3><p>I followed the agent’s cavalry-artillery “doomstack” recommendation exactly. The results were… dramatic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-kdCKZWTQ6EeOku5QRBbfg.png" /><figcaption>Battle of Harput: Decisive victory against Georgia with minimal losses</figcaption></figure><p>And here comes the magnum opus of the doom stack:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y3uJfGmxT00URYFbB92UDQ.png" /><figcaption>Battle of Kars: 10,000 troops defeat 55,000 Mamluk forces</figcaption></figure><p>That second screenshot isn’t a typo. <strong>My 10k army defeated a 55k Mamluk force.</strong></p><p>I tested this against two nations with different military traditions to see if it was a fluke. It wasn’t. The composition exploited Age of Reformation combat mechanics so effectively that numerical superiority didn’t matter.</p><p>This is meta-knowledge that typically takes experienced players dozens of hours to discover through trial and error. Having instant access to it while playing — without breaking flow — fundamentally changed my campaign. I’m not just learning faster; I’m experimenting with strategies I wouldn’t have attempted otherwise.</p><h3>Conclusion</h3><p>Building a game tutorial assistant used to mean months of development — UI, state management, API integration. The Google stack (ADK + Gemini + Interactions API) compressed this to hours.</p><p>A weekend of work gave me:</p><ul><li>Comprehensive EU5 knowledge base with instant lookup</li><li>Multi-turn conversations that remember context</li><li>YouTube integration for community strategies</li><li>Working interface via adk web</li></ul><p><strong>The tech delivers.</strong> Gemini 3 synthesizes complex game mechanics across multiple sources. The Interactions API maintains conversation state without manual history management. ADK orchestrates 20+ tools seamlessly. Together, they turn a 500-page wiki into an interactive expert.</p><p>Instead of pausing every 15 minutes to research, I get answers in-flow. Instead of trial-and-error builds, I get meta strategies backed by exact game data. The 55k-vs-10k battle screenshot speaks for itself.</p><p>AI assistants excel at knowledge-intensive games: grand strategy, 4X, competitive multiplayer — anything with hundreds of interconnected systems. The barrier to building one is now low enough that enthusiasts can do it in a weekend.</p><p>If you’re tackling steep learning curves — games, technical docs, internal knowledge bases — this stack works. Gemini handles reasoning, Interactions API manages state, ADK orchestrates tools. You define what your agent knows, Google’s infrastructure handles the rest.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e8f7928603ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vibing My Way Back in Time: Building an AI Time Machine with Gemini 3]]></title>
            <link>https://ertugruldemir.medium.com/vibing-my-way-back-in-time-building-an-ai-time-machine-with-gemini-3-b67f38077e91?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/b67f38077e91</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[gemini]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Mon, 01 Dec 2025 05:22:53 GMT</pubDate>
            <atom:updated>2025-12-01T05:30:17.476Z</atom:updated>
            <content:encoded><![CDATA[<h4>An adventure into the new Gemini environment</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J8uXuIrdgFHX2G3i-6Ow4g.png" /></figure><h3>Table of Contents</h3><ul><li>Why Gemini 3 (and Why Now)</li><li>A Backend Engineer Walks Into a Frontend</li><li>Let’s Get Our Hands Dirty</li><li>Adding Maps to the Mix</li><li>Vibing Finally</li><li>Coordinates: Not Enough!</li><li>Teaching Nano Banana to See the Past</li><li>The Sketch That Built Itself</li><li><a href="#7e7f">It Actually Works <em>(skip here if you’re not interested in setup)</em></a></li><li>Neo-Brutalism: Finding the Right Aesthetic</li><li>Nano Traveler in Action</li><li>The Verdict</li></ul><p>There’s a specific kind of itch that hits when a new AI model drops. Reading the announcement blog isn’t enough — you need to build something with it. Test whether those promised capabilities actually hold up when you’re coordinating multiple services into something real.</p><p>So when Google released the Gemini 3 series last week, I did what any reasonable person would do: cleared my weekend, mass-produced coffee, and decided to build a time machine.</p><p>Not a real time machine. But close enough. Beyond the usual race for impressive single-shot prompts, I wanted something more ambitious: what happens when you orchestrate these models into something complex? What becomes possible when you combine Gemini 3 Pro’s reasoning with Gemini 3 Pro Image’s spatial precision — better known by its delightful nickname, “Nano Banana Pro”?</p><p>The idea was simple on paper, beautifully chaotic in practice: point at any street corner on Google Maps, and the app shows you what that exact spot looked like during its historical peak. Not a stock photo from an archive — a generated image, informed by actual research, matching your precise camera angle. The elevator pitch: “Street View, but for the past.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QZCJnmE0syko5dtGn1NNpA.jpeg" /></figure><p>I called it “Nano Traveler” (partly as a nod to that Nano Banana Pro codename, partly because naming things at 2 AM after too much caffeine produces questionable results).</p><p>Here’s where it gets properly recursive: I didn’t just build an AI-powered app, I built it <em>with</em> AI. The frontend came together using Antigravity, Google’s new agentic IDE that lets you describe interfaces in natural language instead of wrestling with frontend frameworks. The workflow became comically layered: describing what I wanted to one AI, which generates React components, which call another AI to analyze Street View images, which hands off to yet another AI to reconstruct historical scenes. It’s AIs all the way down.</p><p>What followed was a few-hour sprint coordinating three Google services and a couple of AI models, with some genuinely impressive moments watching the models research historical contexts and reconstruct scenes with spatial awareness I didn’t expect. The app doesn’t just guess at history — it researches, reasons through architectural principles, then reconstructs the scene while maintaining your exact viewpoint.</p><p>This is the story of that build: what worked surprisingly well, what broke hilariously, and what it revealed about where this technology is heading.</p><h4><strong>Why Gemini 3 (and Why Now)</strong></h4><p>Why use Gemini 3 specifically? Historical reconstruction proved the perfect stress test. It requires analyzing images, researching context, reasoning across modalities, and generating spatially-precise outputs — all in a coordinated workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bi5gwWBvOg9_3l63eEbDiw.png" /></figure><p><strong>Multimodal reasoning:</strong> Gemini 3 Pro needed to analyze a Street View photograph, understand the architecture, research historical context, and translate that into precise generation instructions. This is a chain where any weak link breaks everything.</p><p><strong>Context understanding:</strong> Google emphasized how Gemini 3 “figures out the context and intent behind your request.” Would it understand that “1890s Istanbul” means Ottoman architecture, period-appropriate urban planning, accurate building proportions? Or would I get generic “old-timey city” vibes?</p><p><strong>Spatial precision with Nano Banana Pro:</strong> I’d been burned by “spatial precision” claims before. Every image model promises it until you need a generated scene to match a specific camera angle — then everything drifts. For a time travel app, that drift destroys the entire illusion. If I capture a Street View looking northeast at a 15-degree upward pitch, would the historical reconstruction maintain that exact viewpoint?</p><p>The combination made sense: Gemini 3 Pro to research and reason, Nano Banana Pro to render with spatial fidelity. The question was whether I could orchestrate this before the coffee ran out.</p><h4>A Backend Engineer Walks Into a Frontend</h4><p>I am not a frontend developer. I’m a data scientist and ML engineer. My natural habitat is Python scripts and backend services where “user interface” means a well-documented API endpoint. Buttons, layouts, CSS that breaks when you look at it wrong — that’s someone else’s problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VhBQuC1u3Hxqawzil-X3_A.jpeg" /></figure><p>But Nano Traveler needed a real frontend. Not a Gradio demo or Streamlit dashboard held together with prayers. If I was testing whether Gemini 3 could handle complex multimodal orchestration, I wanted the whole experience to feel like an actual product.</p><p>Which meant a choice: spend half my sprint fighting with TypeScript and React, or try something different.</p><h4><strong>Enter Antigravity.</strong></h4><p>Google released this new agentic IDE alongside Gemini 3: describe what you want, and agents work autonomously across your editor, terminal, and browser. You review artifacts, screenshots, implementation plans, working components instead of watching code stream by.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fuCXSvpEri82C4kyQJ_m-g.jpeg" /></figure><p>For someone who usually keeps tight control over every line of code, this felt like handing my car keys to a stranger. I call my usual approach <em>dictate code</em> instead of <em>vibe code. </em>I have something well-structured in mind before I instruct. Trusting an agent to turn “retro-futuristic interface with scan-line effects” into actual implementation went against every debugging instinct I’d built up.</p><p>But here’s why it made for a good experiment: testing Antigravity on a language I’m fluent in wouldn’t tell me much. I’d backseat drive the whole time. TypeScript and React? I know just enough to be dangerous. I’d have to actually <em>vibe code</em> and trust the system.</p><p>Could Antigravity turn a backend engineer into someone capable of shipping a polished frontend? And could I resist micromanaging an AI that definitely knows more about CSS grid than I ever will?</p><p>The answer to both surprised me.</p><h3>Let’s Get Our Hands Dirty</h3><p>First things first: we need an API to connect our app to Gemini. Two options here: Google AI Studio or GCP/Vertex AI. I went with AI Studio because it offers generous free usage limits. Getting started takes just a few steps:</p><ul><li>Go to <a href="https://aistudio.google.com/">https://aistudio.google.com/</a></li><li>Select “Playground” from the left side menu</li><li>From the model selection dropdown, set it to gemini-3-pro-preview</li></ul><p>These steps get you into a simple, useful chat GUI with Gemini 3 enabled. But we want to run this in our app, which means we need API access. A couple more steps:</p><ul><li>Go to <a href="https://aistudio.google.com/api-keys">https://aistudio.google.com/api-keys</a> to create your API key</li><li>Head back to the chat menu and select “&lt;&gt; Get code” from the top right (near where you selected your model)</li><li>You’ll see some getting-started copy-paste code for connecting your app to Gemini models, as shown below:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*f7C-f6LE46_WagZ8W53tYg.png" /></figure><p>While we’re in that chat screen, let’s run some basic tests with Gemini 3 Pro.</p><p>I started with what I already knew: Gemini models can work with coordinate pairs. Gemini 3 inherited this capability, so my proof of concept was straightforward — make a single API call with coordinates and see what comes back.</p><p>It worked out of the box:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*qKJXHrzWAoEMwLoS5Kwrxg.png" /></figure><p>But that was for a well-known location with rich history. When I tested more generic spots, the margin of error climbed and it started making mistakes. Expected, but not acceptable for what I was building.</p><p>Time for the next weapon: grounding. It’s on the same screen in AI Studio’s playground, right hand side:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/297/1*zJ1lcGBgP3GzGcVIoygKIg.png" /></figure><p>That little button in AI Studio (or Vertex) lets you leverage Google Search to enrich the model’s context. With grounding enabled, Gemini 3 Pro tried hard to narrow things down. After a lot of thinking, it got within a couple hundred meters. Better, but still not enough.</p><p>I needed a better approach. The obvious answer was asking users for the exact address, but then what’s the point? They could just Google the historical context themselves. Plus, users aren’t cartographers with GPS coordinates memorized. Most people think in terms of “that corner near the coffee shop,” not latitude/longitude pairs.</p><p>The real first step was obvious: use another service to handle location input. And for getting coordinates from a user-friendly interface, what’s better than Google Maps Platform?</p><h3>Adding Maps to the Mix</h3><p>We already need our first upgrade and we haven’t even written a single line of code yet. For this, we need to enable another service (or services, pretty soon), which adds a bit of complexity. Great for our case — I want to challenge the coding part using Antigravity as we include more services.</p><p>Here are the simple steps to follow on Google Cloud Platform to get your credentials:</p><ul><li>Open the Google Cloud Console and log in</li><li>Select your project or create a new one using the dropdown at the top</li><li>Set up a billing account for the project (required for Maps APIs)</li><li>Navigate to APIs &amp; Services &gt; Library and search for: Maps JavaScript API, Geocoding API, Street View Static API, Places API</li><li>Click the Enable button for all of them</li><li>Go to APIs &amp; Services &gt; Credentials, create a new API Key, and copy it</li></ul><h3>Vibing Finally</h3><p>Now we have starter example code from AI Studio and API keys for both Gemini and Maps. Time to launch Antigravity and experience vibe coding with Gemini 3 Pro.</p><p>After launching, you’ll be welcomed by the Agent Manager screen. Think of it as an agent orchestrator page where Antigravity can plan, create tasks, and run multiple agents simultaneously to edit multiple files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*luS3k_uBNvIdj1qDFx90xA.png" /></figure><p>You can also use the editor directly or even use autocomplete, which is closer to my usual dictate code style than vibe code. But for the true vibe experience, let’s leverage this new Antigravity Agent Manager. It also has a built-in browser where you can see and edit your frontend work immediately — perfect for testing the interface as it builds.</p><p>I started by telling the Agent Manager my idea in a clean, simple way: “I have this idea, I want to see these results, I have these examples, I have these API services and their keys.” Then Antigravity came up with a plan I could comment on, review, and give directions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*qcSQAL2OyO9jwwlczMthtw.png" /></figure><p>It created a detailed implementation plan in markdown for me to review. I commented and directed the model here and there to keep it on track. Turns out vibe coding doesn’t mean hands-off — it means working at a higher level of abstraction, guiding the agent through feedback rather than writing every line.</p><p>After a couple minutes of back and forth with the IDE and the AI, I got the simplest version of what I asked for: grab my location using maps, convert it into coordinates, send it to Gemini to get its actual and historical context. It worked on the first try. Even better, when I needed specific functionality, I could describe logic in Python (my comfort zone) and ask Antigravity to implement it in TypeScript.</p><h3>Coordinates: Not Enough!</h3><p>Alright, we have our baseline: a classic Google Maps minimap where you can select your location, get the address and coordinates, then send them to Gemini with a simple task: retrieve the historical context and use Google grounding to enhance the results.</p><p>It works. Sort of. It handles a really primitive task, but let’s be honest: we haven’t improved much over our initial chatbot query. We fixed some problems with Maps integration and gained precision, but we’re not there yet.</p><p>Some locations return minimal details from coordinates or grounding. Other historical spots are so dense that even with exact coordinates, there might be multiple points of interest within a close perimeter. How would the model know which building we’re looking at?</p><p>Then it hit me: what if we feed it a photo for extra context? Gemini has multimodal capabilities, so this shouldn’t be a problem.</p><p>Even better: we could give it just the image and let the model guess the location. Perfect for when you don’t have exact coordinates or want to use photos you took earlier. Suddenly, the app became more flexible.</p><p>But wait — we don’t even need users to take photos. We have Google Street View! We can pull specific angles, zoom levels, everything, and feed all of that into our knowledge pool to enrich the context.</p><h3>Teaching Nano Banana to See the Past</h3><p>Now we have rich context. Time to connect the dots and finalize our initial idea: get the location, generate the historical image. This is where Nano Banana Pro comes in. It needs a good prompt to generate the best experience, and here’s the clever part: most of the important bits of that prompt will be generated by our initial analysis prompt. Here’s the structured output format:</p><pre>{<br>  &quot;locationName&quot;: &quot;Official location name&quot;,<br>  &quot;historicalEra&quot;: &quot;Era name (e.g., Edo Period, Victorian Era)&quot;,<br>  &quot;imagePrompt&quot;: &quot;Comprehensive visual prompt incorporating all research&quot;,<br>  &quot;architecturalStyle&quot;: &quot;Specific architectural style&quot;,<br>  &quot;sourceCameraAngle&quot;: &quot;Precise camera angle with degrees if possible&quot;,<br>  // ... 15+ additional fields for detailed reconstruction<br>}</pre><p>Some of these fields go directly to the user in the frontend, while others feed into the image generation phase. We also pass the initial image to Nano Banana if provided (either direct user input or Street View). The full analysis prompt is too long to share here, but here’s the relevant piece for extracting camera information when an image is present:</p><pre>## PHASE 5: CAMERA &amp; SPATIAL ANALYSIS (For image input only)<br>19. Precisely describe the camera angle with numerical detail (e.g., &quot;eye-level with 5° upward tilt&quot;, &quot;15° worm&#39;s eye perspective&quot;).<br>20. Estimate the camera distance from the subject in meters (e.g., &quot;approximately 8 meters from facade&quot;, &quot;50 meters wide establishing shot&quot;).<br>21. Describe the field of view and focal length equivalent (e.g., &quot;wide angle ~24mm capturing full structure&quot;, &quot;normal ~50mm view&quot;, &quot;telephoto ~85mm&quot;).<br>22. Identify horizon line placement (e.g., &quot;horizon at lower third of frame&quot;, &quot;centered horizon&quot;, &quot;high horizon line at upper quarter&quot;).<br>23. CRITICAL FOR RUINS: If the current location shows ruins, partial structures, or archaeological remains:<br>    - Document which portions are PRESENT in the modern image<br>    - Document which portions are MISSING/DESTROYED<br>    - Note the EXACT position and orientation of remaining structures<br>    - This ensures historical reconstruction maintains spatial accuracy</pre><p>The workflow is elegant: simple location info goes into the model, rich historical data comes out, then feeds into the image generation model. Some information displays to the user, the rest guides Nano Banana’s reconstruction.</p><p>Quick note: to enable Nano Banana Pro in AI Studio, you need to link your API key to a billing-enabled Google Cloud Project. It won’t work on the free tier for image generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*LmlAHbCrEwU-yN79KRlRbg.png" /></figure><p>Once that’s set up, you can use the same example code pattern from AI Studio that we used for Gemini 3 Pro. Same API key, different model name. Easy.</p><h3>The Sketch That Built Itself</h3><p>The moment of vibe-truth arrived. I had an idea of how the app should look, but remember: I’m not a frontend developer. I scribbled some wireframes with pen and paper, then realized I’m not much of an illustrator either.</p><p>Then it hit me. Why not go full meta?</p><p>I took a photo of my scribbles and fed them directly to Nano Banana Pro. My prompt: clean up these notes and sketches, turn them into something more legible. Not a professional blueprint (though I’m sure it could do that too), but something clear enough for Antigravity to understand. I wanted to test how well these systems could interpret messy human intent.</p><p>Here’s what Nano Banana Pro produced from my chicken scratch:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r8jsq8dqFgJ8OFRlfaELIg.png" /></figure><p>Much cleaner than my original. Nano Banana Pro interpreted my messy wireframe surprisingly well.</p><p>Now came the tricky part. I know just enough TypeScript and React to articulate what I need and inadvertently break everything when reviewing code line-by-line.</p><p>I carefully gathered my requirements: what’s on my mind, what I expect, how it should behave. Then I handed the cleaned sketch to Antigravity’s Agent Manager along with my expectations and let it work.</p><p>My requirements were specific: a clean map interface for location selection, a split view showing current vs historical scenes side-by-side, and a panel displaying the historical context Gemini had researched. I wanted it retro-futuristic — something that felt like using advanced technology to look backwards in time.</p><h3>It Actually Works</h3><p>It’s alive! Well, sort of. Antigravity needed a while to wrangle some nasty bugs, and I had to step in here and there, but it managed to pull together something that actually worked.</p><p>Time for the real test. I navigated to my favorite team’s stadium on the map and ran the whole pipeline. Fingers crossed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*FGt1c4ao_XAy-LMspLfFlg.png" /></figure><p>It nailed the location details. Historical context, architectural notes, the works. Now for the moment of truth: the time travel results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zxcOcMEEDX6pItFp3eJRFg.png" /></figure><p>Genuinely impressive. The reconstruction was far better than I expected.</p><p>Look at the angle preservation. Nano Banana Pro replicated the camera perspective from the Street View image exactly, then populated the scene with classical Ottoman attire mixed with late modernization attempts. That’s a tricky era to nail historically — the transition period between traditional Ottoman and early Republic reforms. The building itself has minimal information online, but the model still reconstructed architecture that’s remarkably close to historical records.</p><p>This was just the start. Now came iteration.</p><p>Over multiple test runs, I worked with Antigravity to squash bugs, add small enhancements, and most importantly, refine the prompts. The generation prompt especially needed work.</p><p>Then I stumbled onto something interesting. I’d been converting the structured JSON output from Gemini 3’s analysis phase into plain text for Nano Banana Pro. But while researching prompting techniques, I found people getting better results by keeping the JSON format. Perfect for our case.</p><p>Here’s what the final generation prompt looked like:</p><pre>{<br>  &quot;task&quot;: &quot;temporal_transformation&quot;,<br>  &quot;Architecture&quot;: [<br>    &quot;${analysis.locationName} as it appeared during ${analysis.historicalEra}&quot;,<br>    &quot;Structural condition: ${analysis.structuralCondition}&quot;,<br>    &quot;Primary materials: ${analysis.buildingMaterials.join(&quot;, &quot;)}&quot;<br>  ],<br>  &quot;Camera&quot;: {<br>    &quot;perspective&quot;: &quot;EXACT match: ${analysis.sourceCameraAngle}&quot;,<br>    &quot;angle&quot;: &quot;${analysis.sourceCameraAngle}&quot;,<br>    &quot;distance&quot;: &quot;${analysis.sourceCameraDistance}&quot;<br>  },<br>  &quot;CriticalConstraints&quot;: [<br>    &quot;MAINTAIN exact camera angle, framing, and composition from source&quot;,<br>    &quot;PRESERVE spatial relationships and perspective geometry&quot;<br>  ],<br>  // ... additional sections for ColorPalette, HumanActivity, Environment, Lighting, Mood<br>}</pre><p>The final prompt structure organizes the analysis data into six major sections: Architecture (style, materials, condition), ColorPalette (period-accurate pigments), HumanActivity (demographics, clothing, typical activities), Environment (vegetation, climate, infrastructure), Lighting (era-specific technology), and Camera (precise angle matching from the source image). Each section includes 3–5 specific instructions synthesized from Gemini’s analysis.</p><p>Could it be refined further? Absolutely. But this gave me stable, consistent results. The core pipeline was working.</p><p>But we had a bigger problem. As a backend guy who likes everything structured, terminal-based, and boring, the UI was still rough. Time to see if we could vibe code our way to something actually polished.</p><h3>Neo-Brutalism: Finding the Right Aesthetic</h3><p>This was the most enjoyable part of the build. Although I don’t know web design principles, I had a specific aesthetic in mind. The problem? I didn’t know what to call it. After researching and exploring design trends, I finally found it: Neo-Brutalism.</p><p>In short, Neo-Brutalism (or Neubrutalism) is a modern digital design trend that blends the raw, unpolished aesthetic of 1950s brutalist architecture with contemporary colors, typography, and animation. It’s a reaction against the standard, highly polished “clean” look of modern web design — think flat design or minimalism, but deliberately rougher.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZL6vhnazZISaoBhmcIv0Dg.png" /><figcaption><a href="https://neo-brutalism-ui-library.vercel.app/">Source</a></figcaption></figure><p>It was a perfect fit for the look in my mind — raw and bold, matching the experimental nature of the project. I researched Neo-Brutalist design principles, studied color palettes and layout patterns, then condensed everything into a detailed prompt.</p><p>I fed this into Antigravity, honestly expecting the model to fight me and revert to the standard, polished ‘clean web’ look it’s likely trained on. Instead, it understood the assignment immediately. It nailed the high-contrast borders and the deliberate lack of padding on the first try, capturing the ‘raw’ vibe without making the app unusable.</p><p>With the help of Antigravity’s autonomous agents, a little bit of feedback, and some iterative fixes, I got something really close to what I wanted.</p><p>After all the iterations, the final interface came together:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nEQ3mfGGs91HoT-2rggwQg.png" /></figure><p>The transformation from the initial bland interface to this bold, striking design took about an hour of back-and-forth with Antigravity. Worth every minute.</p><h3>Nano Traveler in Action</h3><p>Now for the best part! Let’s test our vibe-coded app to find some historical places going back in time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*uC3D52kTkqzYq8BYeDcAfA.png" /></figure><p>I’m starting with a challenging location: the Theodosian Walls of Istanbul. This tests the system on multiple levels. First, the area has changed dramatically over time — the city expanded into the sea through land reclamation. Second, it’s surrounded by major historical landmarks in a tight area (Topkapı Palace, Hagia Sophia, etc.), so the model needs to be precise about which structure we’re looking at. The Street View angle adds another challenge: it’s shot from a modern road, but historically this viewpoint would have been from the water. Complex enough for a real test. Here’s the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BeBzOnpuSOQiEuD96PNqhg.png" /></figure><p>Impressive! Gemini selected the Byzantine era and nailed the historical details. The land reclamation is reversed, showing the walls extending to the water as they originally did. The soldier uniforms match Byzantine military dress, and the ship design fits the period. Best of all, it maintained the camera angle from the Street View — we’re looking at the walls from the same perspective, just with the modern road replaced by the Sea of Marmara.</p><p>Next, I’ll try the photo upload feature. This time it won’t have exact coordinates or Street View — instead, Gemini needs to identify the location from the image alone and reconstruct its historical peak. I’m testing with the Menkaure Pyramid, the smallest of the famous Pyramids of Giza. This is a good test because most image generation models struggle with pyramids — they show the inner stone core rather than the smooth limestone casing that covered them originally. Let’s see how it handles this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ac36S-uxjxsg2NPiNWerOA.png" /></figure><p>Excellent! The model identified the pyramid from the photo with high accuracy and, more impressively, reconstructed it with the original white limestone casing intact. The red granite blocks at the base — a distinctive feature of Menkaure’s pyramid — are accurately rendered. The camera angle matches the source photo well, maintaining the same perspective tourists would have had thousands of years ago.</p><p>Next, let’s test something a little different. Instead of a famous landmark, I’m testing a street where I grew up in Istanbul. This neighborhood had no particular historical significance — just a regular residential area that’s been completely transformed in the last 50 years. Let’s see if Nano Traveler can reconstruct something interesting from an ordinary location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ghno6xHCSt9rRty72qBWLA.png" /></figure><p>Fascinating result! It transformed this ordinary street into what appears to be late Ottoman Istanbul. The modern skyscrapers are gone, replaced by traditional wooden Ottoman houses. The wide modern road becomes a cobblestone street. Even without exact historical documentation of this specific location, the model reconstructed a plausible period scene based on Istanbul’s general urban character during that era.</p><p>I could share test results for hours — each location reveals something new about how the system handles different challenges. But this post is already longer than I planned, and I think these three examples demonstrate the range: famous landmarks, photo uploads, and ordinary streets all produce interesting reconstructions.</p><h3>The Verdict</h3><p>This was one of the more anxious coding experiences I’ve had. Letting go of line-by-line control, trusting an agent to build a React frontend while I watched from the sidelines, went against every instinct I’d developed as a developer. The “vibe coding” experience I’d been curious about turned out to be less about vibing and more about learning a new kind of control.</p><p>Let me be clear: there’s no magic “one-shot prompt” that builds complete applications. Even with Antigravity’s autonomous agents and Gemini 3’s reasoning, this wasn’t “describe an app, press enter, come back to a finished product.” I spent hours iterating, debugging, and guiding the agents through feedback loops. The difference is where I spent that time — reviewing artifacts and giving high-level direction rather than writing CSS selectors and debugging event handlers.</p><p>The real shift isn’t that AI writes all your code. It’s that you can operate at a higher level of abstraction. I described interfaces in terms of design principles rather than component hierarchies. I debugged by pointing at screenshots and saying “that spacing feels wrong” rather than hunting through stylesheets. I stayed in architect mode instead of constantly context-switching to implementer mode.</p><p>Would I have built Nano Traveler without these tools? Probably not. Not because it’s technically impossible — the individual pieces aren’t that complex — but because the friction would have been too high. Learning enough React to build a decent frontend, orchestrating the multi-stage pipeline, iterating on the UI… Each friction point is a place where weekend projects die.</p><p>Antigravity and Gemini 3 didn’t eliminate the work. They eliminated the friction. That’s the difference.</p><p>The system isn’t perfect. Historical reconstructions from well-documented locations (Hagia Sophia, Pyramids of Giza) are impressive. Obscure residential streets produce plausible but potentially inaccurate results. Camera angle matching works most of the time, but not always. Generation takes 30–60 seconds per image.</p><p>But perfect wasn’t the goal. Working was the goal. And it works.</p><p>A year ago, this project would have taken days and required expertise I don’t have. Today, it took a weekend and the willingness to experiment. That compression is remarkable.</p><p>We’re not at “anyone can build anything” yet. You still need developer instincts — knowing what’s possible, how to break down problems, when something feels wrong. But the barrier between “I have an idea” and “I have a working prototype” just got dramatically lower.</p><p>The weekend’s over. The coffee’s gone. And somewhere on my hard drive sits an app that lets me point at any street corner and glimpse what might have been there centuries ago. Not bad for an itch that needed scratching.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b67f38077e91" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reason with Your Images at Home]]></title>
            <link>https://ertugruldemir.medium.com/reason-with-your-images-at-home-a069eecd25d7?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/a069eecd25d7</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Mon, 27 Oct 2025 07:54:46 GMT</pubDate>
            <atom:updated>2025-10-27T08:09:33.377Z</atom:updated>
            <content:encoded><![CDATA[<h4>Use Gemma 3 as Visual Thinking Model</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c8mxzrbLC4K3zZhOX0FUQA.png" /><figcaption>Source: Generated by Author</figcaption></figure><p>Last year, we opened a similar discussion with a cautious statement: ‘Reasoning models are not entirely new.’ Today, we can drop the caution entirely. Why? Because reasoning capabilities are increasingly common for complex AI tasks. Major models now incorporate some form of extended thinking, and you’ve certainly encountered it in the wild, often called ‘thinking,’ ‘reasoning,’ or ‘test-time computing’ etc.</p><p>That’s our focus here: applying reasoning to vision language models. We’ll use reinforcement learning and practical optimization tricks to make it work in a home setup, training a small model on consumer hardware.</p><p>Here’s what we’ll cover:</p><ul><li><strong>Why visual reasoning demands a different approach</strong> — How image data’s density creates unique challenges that require detection before thinking, unlike pure text reasoning.</li><li><strong>How three-stage structured generation works</strong> — The detection-thinking-answer framework that prevents models from overlooking critical visual details or jumping to premature conclusions.</li><li><strong>Why GRPO excels at teaching structure without hard labels</strong> — How reward-based learning guides format compliance and correctness without requiring curated datasets of “perfect” reasoning chains.</li><li><strong>How GSPO and Dr. GRPO improve training stability</strong> — The sequence-level importance sampling and debiased loss formulations that fix subtle issues in the original GRPO algorithm.</li><li><strong>Why LoRA rank-1 is surprisingly sufficient</strong> — How reinforcement learning’s behavior-nudging nature allows even minimal parameter updates to achieve full fine-tuning performance.</li><li><strong>How to train vision-language models at home</strong> — The combination of Unsloth, vLLM, TRL, and optimization techniques that make 4B parameter model training feasible on consumer hardware in just 3–4 hours.</li></ul><p><strong>The Core Idea: How to ‘Think’</strong></p><p>For those who might be new to this idea, let’s quickly recap the core mechanics. At their heart, Large Language Models are just expert next-token predictors, focusing on maximizing the likelihood of the entire sequence, one word at a time. They do this by regenerating the compressed broad knowledge they are learnt from the immense pretraining and noisy data.</p><p>Now, imagine you were tasked with explaining a complex concept you know well, but you have only one chance to blurt it all out at once, without pausing. You might get confused, forget key details, or jump straight to the conclusion. Wouldn’t it be better to take a moment to <strong>‘think’</strong> about it first, or even jot down a few notes before you speak?</p><p>While the internal mechanics aren’t exactly the same as human thought, this is precisely how LLM “thinking” works. Research consistently shows that on complex tasks — especially arithmetic or multi-layered problems — giving the model space to “gather its thoughts” or validate its steps tends to produce dramatically better results. (Remember, though, the nature of LLMs: this is where they differ from human thinking. Here, thinking helps maintain salient information in the context window, allowing the model to condition its outputs on more relevant intermediate states.)</p><p>We saw this firsthand in our last article, where the model used this technique — breaking down and validating math questions with the “thinking budget” — to achieve a significant performance improvement over its standard, non-reasoning instruction-tuned version.</p><p><strong>Thinking over Images</strong></p><p>Now that the need for thinking is clear, let’s consider how we can leverage this capability for multimodal inputs — specifically, image inputs in our case. Many modern large language models can now understand different modalities: images, audio, video, and of course text. We won’t dive into the intricacies of processors or exact tokenization details here, but the key idea is that slices or patches of these different inputs can be processed like tokens by the model.</p><p>Although it would be fascinating to explore using other modalities in the thinking phase itself — such as generating intermediate sketches or diagrams maybe — we’re not quite there yet, so we’ll stick with text-based reasoning for now.</p><p>So back to our initial question: If we have a test-time compute budget for Vision Language Models, how should we spend it to improve final accuracy? The first instinct is to do what we did with text — just reason over the input before finalizing the answer.</p><p>But here’s the challenge: image data can pack an enormous amount of information into a small space, even for small objects. This creates two issues:</p><ul><li>Sometimes not all of that information is relevant to the task at hand</li><li>Other times, the relevant details are so subtle they risk going unnoticed</li></ul><p>This suggests adding an additional budget allocation where the model first takes note of important objects and visual elements. Then, the model can think over these observations in combination with the task itself, using the remaining budget to arrive at a well-grounded answer.</p><p><strong>An Image is Worth 16x16 Words</strong></p><p>Before going further with the visual reasoning training steps, I wanted to provide a short section on why images have great potential for so-called “language” models. <strong>The title of the </strong><a href="https://arxiv.org/abs/2010.11929"><strong>Vision Transformers paper from 2020</strong></a><strong> summarizes it perfectly: <em>An Image is Worth 16x16 Words</em>.</strong> A lot has changed since 2020, but the core idea stands: an image can encode significantly richer information density.</p><p><strong>While writing this article, a new paper was released by DeepSeek: </strong><a href="https://arxiv.org/html/2510.18234v1"><strong><em>DeepSeek-OCR: Contexts Optical Compression</em></strong></a><strong>. </strong>Although it’s not exactly about reasoning, it touches on a key concept we’re emphasizing throughout this article. The authors used images as a compression medium for text — instead of storing text as tokens, they rendered it as an image and compressed it into far fewer vision tokens, achieving 10–20× compression ratios.</p><p>Although this newer model leverages more sophisticated techniques such as object detection and convolutional compression, Gemini has a more primitive compression mechanism using SigLIP that converts a given image into 256 tokens — which is more than sufficient for our use case.</p><p><strong>Data and the Model</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*6wJjBw09Mjz-Frp_pFD6Eg.png" /></figure><p>Before we train a model to reason over images, we need to make two crucial decisions: which images to use, and which model to train. These choices are more constrained than they might first appear, especially when working from home without enterprise-scale compute.</p><p>To see the true effects of teaching a model to think, the data needs to strike a delicate balance. On one hand, it should be complex enough to genuinely challenge the base model — think diagrams with multiple labeled components, embedded text, or intricate spatial relationships between objects. On the other hand, it can’t be so chaotic or unstructured that it overwhelms a small model during training. Remember, we’re working with limited compute, which means we need a lightweight model that can still demonstrate reasoning improvements without requiring massive resources. (Though admittedly, this constraint adds an interesting challenge that makes the experiment more rewarding.)</p><p>For our model, we’re using<strong> Google’s Gemma 3 </strong>family, specifically a variant with around 4 billion parameters. This choice hits several sweet spots at once:</p><ul><li>It’s open source, so we can freely modify and train it</li><li>It’s genuinely lightweight — small enough to train at home, yet capable enough to show meaningful improvements</li><li>It uses SigLIP as its image encoder, which means it can process visual inputs natively</li><li>It offers a generous 128k context length, giving us plenty of room for the model to “think” without hitting token limits, although we won’t be using it all anyways</li></ul><p>For the dataset, we’re using the AI2 Diagram Dataset, known as AI2D. This is a collection of illustrative diagrams originally created for research on diagram understanding and question answering. It’s an ideal fit for our purposes. <strong>As the dataset authors aptly put it: “A Diagram Is Worth A Dozen Images.”</strong> Diagrams occupy that perfect middle ground — they’re more complex than simple photographs of objects, requiring the model to parse labels, arrows, relationships, and spatial hierarchies. Yet they’re structured enough that we can teach a model to systematically break them down into observable components, which is exactly what our detection-then-reasoning approach is designed to handle.</p><p><strong>The Task: Teaching Structure Through Reinforcement</strong></p><p>Our main tool for this endeavor will be reinforcement learning, though perhaps not in the way you might first expect. We’re not forcing the model to learn entirely new capabilities from scratch.<strong> Instead, we’re strategically shifting its behavior — nudging it to structure its generation process in a specific, deliberate way.</strong> We’ll guide this shift through carefully designed instructions and rewards that encourage the model to follow a particular pattern.</p><p>To make this concrete, let’s look at the structure we want the model to adopt. We define three clear stages using XML-style delimiters that mark where each phase begins and ends:</p><pre># Detection, reasoning, and solution delimiters<br>DETECTION_START = &quot;&lt;detection&gt;&quot;<br>DETECTION_END = &quot;&lt;/detection&gt;&quot;<br>REASONING_START = &quot;&lt;thinking&gt;&quot;<br>REASONING_END = &quot;&lt;/thinking&gt;&quot;<br>SOLUTION_START = &quot;&lt;answer&gt;&quot;<br>SOLUTION_END = &quot;&lt;/answer&gt;&quot;<br><br># Example expected output for a multiple choice question:<br># &lt;detection&gt;<br># Diagram shows: A cycle with arrows connecting elements<br># Key elements: Respiration box → Combustion box → arrows labeled with gases<br># Labels visible: &quot;Oxygen&quot;, &quot;Carbon dioxide&quot;, &quot;Nitrogen&quot;, &quot;Heat&quot;<br># Arrow from Respiration and Combustion point to: Carbon dioxide<br># &lt;/detection&gt;<br># &lt;thinking&gt;<br># The question asks what respiration and combustion give out.<br># In the diagram, arrows from both processes point to the same output.<br># Looking at the choices: A. Oxygen (input), B. Carbon dioxide (output), C. Nitrogen, D. Heat<br># Both respiration and combustion produce carbon dioxide as a byproduct.<br># &lt;/thinking&gt;<br># &lt;answer&gt;<br># B<br># &lt;/answer&gt;</pre><p>Notice how this structure mirrors the way a careful human would approach a diagram question:</p><ul><li>First, in the <strong>detection phase</strong>, the model inventories what it actually sees — the diagram’s components, visible labels, spatial relationships, and key elements. It’s not interpreting yet, just observing and cataloging. Think of this as the model building a mental index of available visual information.</li><li>Then, in the <strong>thinking phase</strong>, the model connects these raw observations to the actual question being asked. It reasons over the detected elements, considers the multiple choice options if applicable, and validates its logic step by step. This is where interpretation happens — where the model moves from “what I see” to “what this means for answering the question.”</li><li>Finally, in the <strong>answer phase</strong>, the model provides its well-reasoned final response, distilled from all the prior work.</li></ul><p>This three-stage breakdown serves a critical purpose. By forcing the model to explicitly separate observation from reasoning from conclusion, we’re essentially teaching it to “show its work” in a way that prevents it from jumping to premature conclusions or missing important visual details. The detection phase ensures nothing gets overlooked. The thinking phase ensures the logic is well thought and validated. The answer phase ensures we get a clean, usable result.</p><p>But here’s the key question: we want the model to follow this format and produce accurate answers. How do we simultaneously check for both format compliance and correctness, then communicate that feedback to the model during training? This is where reinforcement learning comes into play.</p><p><strong>The Rewards: How to Reinforce Structure and Accuracy</strong></p><p>Now we’ve instructed our model to generate outputs in our desired three-stage format. Thanks to its pretraining and existing capabilities, the model will do its best to produce these structured results. But here’s the reality: most of the time, it will fail to produce exactly what we instructed. Due to the stochastic nature of language models — the inherent randomness in how they sample tokens — sometimes the model gets close to what we asked for, and sometimes it nails it perfectly by chance. Our job is to recognize these successful moments and reinforce them, essentially telling the model <strong>“Yes, that’s exactly what we wanted, do more of that”</strong> by rewarding the model weights that led to those outcomes.</p><p>This is where Group Relative Policy Optimization, or GRPO, truly shines. Unlike traditional approaches, we don’t need:</p><ul><li>Meticulously curated datasets of exact output examples</li><li>Carefully constructed preference pairs comparing good and bad responses</li><li>A separate ranker model to judge quality</li></ul><p>Instead, we provide broad reward functions that define what “good” looks like, and then we let the model explore on its own. After the model generates multiple candidate responses, we simply tell it “Okay, I liked this one more than the others — keep doing more like this.”</p><p>Compare this to Supervised Fine-Tuning, where we essentially force-feed the model exact expected outputs and update its weights to mimic explicit examples we provide. <strong>GRPO operates with far more freedom</strong>, allowing the model to learn patterns almost on its own through exploration and feedback. This approach offers a crucial advantage: it helps prevent overfitting to narrow examples and reduces the risk of catastrophic forgetting, where the model loses its initial broad knowledge by being too rigidly constrained to specific patterns.</p><p>But keep in mind that reinforcement learning is not the answer to all of SFT’s shortcomings. <strong>As the “Father of Reinforcement Learning” Richard Sutton indicates</strong>, we don’t yet reward partial successes correctly, nor do we have robust ways to differentiate between multiple correct solution paths to reward the most efficient or generalizable approach. In practice (as you can see in the actual results), this can<strong> lead to reward hacking</strong> by the model.</p><p>Now we arrive at one of the most critical design decisions in our entire process. In order to tell the model “I liked this generation” when comparing multiple attempts, we need to define reward functions — mathematical expressions that return reward points or penalties based on the generated outputs. Designing these reward functions is genuinely enjoyable work, and you have considerable creative freedom to explore different approaches. For our case, we’ll use three straightforward reward functions with task-specific touches that guide the model toward both proper structure and correct answers:</p><ul><li>First, we have the <strong>formatting reward</strong>, which checks whether the model follows the expected detect-think-answer flow using our XML-style delimiters. If the model generates exactly one detection section, one thinking section, and one answer section — no more, no less — it can earn up to three reward points, one for each correctly formatted stage.</li><li>Second, we include a <strong>thinking reward</strong> that provides a small bonus based on the length and substance of the reasoning process. We encourage the model to think through problems thoroughly rather than rushing to conclusions. If the model produces a thinking section with a good amount of reasoning — roughly up to two hundred words of substantive thought — it earns an additional two reward points.</li><li>Third, and perhaps most importantly, we have the <strong>correctness reward</strong>. Since our AI2D dataset includes labeled ground truth answers for multiple choice questions, we can leverage this information to keep the model on the right track. This reward ensures that all our structured thinking actually leads somewhere useful — to the correct answer.</li></ul><p>Let’s examine the formatting reward function in detail to see how elegantly simple these mechanisms can be:</p><pre>def formatting_reward(completions, **_kwargs):<br>    &quot;&quot;&quot;<br>    Reward function for proper formatting with detection, reasoning, and solution delimiters.<br>    Awards points for each correctly formatted section.<br>    Maximum: 3.0 points per completion.<br>    &quot;&quot;&quot;<br>    import re<br><br>    # Define regex patterns to find each section<br>    detection_pattern = f&#39;{DETECTION_START}(.*?){DETECTION_END}&#39;<br>    reasoning_pattern = f&#39;{REASONING_START}(.*?){REASONING_END}&#39;<br>    solution_pattern = f&#39;{SOLUTION_START}(.*?){SOLUTION_END}&#39;<br><br>    scores = []<br>    for completion in completions:<br>        score = 0.0<br><br>        # Search for each section in the generated text<br>        detection_matches = re.findall(detection_pattern, completion, re.DOTALL)<br>        reasoning_matches = re.findall(reasoning_pattern, completion, re.DOTALL)<br>        solution_matches = re.findall(solution_pattern, completion, re.DOTALL)<br><br>        # Award 1.0 point for exactly one of each section (total = 3.0 possible)<br>        if len(detection_matches) == 1:<br>            score += 1.0  # Found exactly one detection section<br><br>        if len(reasoning_matches) == 1:<br>            score += 1.0  # Found exactly one reasoning section<br><br>        if len(solution_matches) == 1:<br>            score += 1.0  # Found exactly one answer section<br><br>        scores.append(score)<br><br>    return scores</pre><p>Notice what’s remarkable here: we’re not providing any ground truth labels about what the detection or thinking sections should contain. We’re not expressing preferences like “this reasoning is better than that reasoning.” We’re simply using straightforward regular expressions to check whether the generation follows the expected structural format. The function looks for our delimiter tags and verifies that each appears exactly once — not zero times (missing sections) and not multiple times (confused, repetitive output). That’s it. This simplicity is powerful because it allows the model to fill in the content of each section in whatever way works best, as long as it maintains the structure we’ve defined.</p><p><strong>Training at Home: Making It Work With Limited Resources</strong></p><p>Okay, we have our structure defined, and we have our reward functions ready to provide feedback. Now comes the practical question: how do we actually train this thing efficiently at home? The challenge is real — we’re working with limited compute resources, not a data center full of GPUs. To make this work, we need to deploy a combination of clever frameworks and optimization techniques that let us punch well above our weight class.</p><p>Our toolkit consists of three key pieces of infrastructure, each serving a specific purpose in our training pipeline:</p><ul><li>First, there’s <strong>Unsloth</strong>. This framework achieves its efficiency gains by overwriting parts of the standard PyTorch operations with manually optimized versions. The developers have painstakingly derived backpropagation steps by hand for common operations, resulting in implementations that use significantly less VRAM while often running faster than their standard counterparts. For us, this translates to being able to train models that would otherwise be out of reach on consumer hardware — and we get this benefit almost for free, simply by loading our model through Unsloth’s API.</li><li>Second, we have <strong>vLLM</strong>, a high-throughput, memory-efficient inference and serving engine specifically designed for large language models. While vLLM offers many features, we’re particularly interested in its ability to generate multiple candidate completions efficiently. Remember, our GRPO approach requires generating several responses per training example so we can compare them and reward the better ones. vLLM makes this generation phase much faster and more memory-efficient than naive implementations would be.</li><li>Third, we’re using <strong>TRL</strong>, a library specifically built for post-training foundation models using advanced techniques like GRPO, DPO, and PPO. Rather than implementing these complex algorithms from scratch, TRL provides battle-tested implementations that handle all the subtle details correctly. It already supports all the post-training techniques we need, allowing us to focus on our specific task rather than debugging reinforcement learning infrastructure.</li></ul><p>With our toolkit established, let’s start by loading our model using Unsloth:</p><pre>def setup_model():<br>    &quot;&quot;&quot;Initialize the vision-language model with LoRA configuration.&quot;&quot;&quot;<br>    max_seq_length = 4096<br><br>    # Load the base model with memory optimizations<br>    model, tokenizer = FastVisionModel.from_pretrained(<br>        model_name=&quot;unsloth/gemma-3-4b-it&quot;,<br>        max_seq_length=max_seq_length,<br>        load_in_4bit=True,  # Quantize to 4-bit precision<br>        fast_inference=True,  # Enable vLLM for generation<br>        gpu_memory_utilization=0.8,  # Use up to 80% of available VRAM<br>        offload_embedding=True  # Move embeddings to CPU when not needed<br>    )<br><br>    # Configure LoRA for parameter-efficient training<br>    model = FastVisionModel.get_peft_model(<br>        model,<br>        finetune_vision_layers=False,  # Keep vision encoder frozen<br>        finetune_language_layers=True,  # Train language model layers<br>        finetune_attention_modules=True,<br>        finetune_mlp_modules=True,<br>        r=1,  # LoRA rank - surprisingly, rank 1 is sufficient!<br>        lora_alpha=32,  # Scaling factor for LoRA weights<br>        lora_dropout=0,<br>        bias=&quot;none&quot;,<br>        random_state=3407,<br>        use_rslora=False,<br>        loftq_config=None,<br>        use_gradient_checkpointing=&quot;unsloth&quot;  # Trade computation for memory<br>    )<br><br>    return model, tokenizer</pre><p>Let’s unpack what’s happening in this setup, because several choices here are critical to making home training feasible.</p><p>We load our model in four-bit precision, which dramatically reduces memory requirements compared to the standard 16 or 32-bit. We also enable embedding offloading, which moves the large embedding tables to CPU memory when they’re not actively being used, and we activate gradient checkpointing, which trades some extra computation during backpropagation for significantly reduced memory usage during the forward pass. The fast_inference=True flag enables the vLLM engine for generation, which we&#39;ll use heavily during the reward collection phase of training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*poUpkwNHkppIElq_xMT6dw.png" /><figcaption>From: <a href="https://thinkingmachines.ai/blog/lora/">LoRA Without Regret</a></figcaption></figure><p>But the most important memory-saving technique here is our use of <strong>LoRA — Low-Rank Adaptation</strong>. Rather than training all the billions of parameters in Gemma 3, we instead train small, low-rank matrices that modify specific weight sets in the base model. Think of it this way: the base model contains large weight matrices that might be, say, four thousand by four thousand elements. LoRA adds two small matrices — one that’s four thousand by one, and another that’s one by four thousand — and their product modifies the original weights. We’re training these tiny matrices instead of the full weights, which means far fewer parameters to update and store gradients for.</p><p>You might be wondering: can such a tiny modification — especially at rank one, which is essentially just a pair of vectors — really be sufficient to teach the model our structured reasoning approach? The answer, surprisingly, is yes. <strong><em>Recent research from John Schulman in collaboration with others at Thinking Machines, published in their report “LoRA Without Regret” </em></strong>demonstrates that in many cases LoRA can match nearly full fine-tuning performance when you target all layers, which we’re doing here. Even more remarkably, they found that in reinforcement learning settings, where the learning signal is particularly sparse and subtle, even rank one can achieve the desired results. The intuition is that RL isn’t teaching the model entirely new capabilities — it’s nudging existing behaviors and shifting probability distributions over actions the model already knows how to take. For this kind of behavior modification, a low-rank intervention is surprisingly powerful. The researchers used an alpha value of thirty-two across their trials, so we replicate that choice here to benefit from their empirical findings.</p><p>With our model loaded and LoRA initialized, we next need to configure our training objective and optimization settings:</p><pre>def setup_training_config():<br>    &quot;&quot;&quot;Configure GRPO training parameters.&quot;&quot;&quot;<br>    return GRPOConfig(<br>        learning_rate=5e-5,  # 10x higher than typical<br>        adam_beta1=0.9,<br>        adam_beta2=0.99,<br>        weight_decay=0.1,<br>        warmup_ratio=0.0,<br>        lr_scheduler_type=&quot;cosine&quot;,<br>        optim=&quot;adamw_8bit&quot;,  # Memory-efficient 8-bit optimizer<br>        logging_steps=1,<br>        log_completions=False,<br>        per_device_train_batch_size=4,<br>        gradient_accumulation_steps=2,  # Effective batch size of 8<br>        num_generations=4,  # Generate 4 candidates per prompt<br>        max_prompt_length=1024,<br>        max_completion_length=2048,  # Room for detection, thinking, answer<br>        max_steps=250,<br>        save_steps=50,<br>        max_grad_norm=0.1,  # Gradient clipping for stability<br>        report_to=&quot;wandb&quot;,  # Track metrics in Weights &amp; Biases<br>        output_dir=&quot;outputs&quot;,<br>        importance_sampling_level=&quot;sequence&quot;,  # Use GSPO variant<br>        mask_truncated_completions=False,<br>        loss_type=&quot;dr_grpo&quot;,  # Use Dr. GRPO variant<br>    )</pre><p>Again, we’re using an 8-bit optimizer to save more VRAM — a pattern you’ll notice throughout our setup. We set our maximum completion length to two thousand forty-eight tokens, which gives the model plenty of room to write thorough detection observations, detailed reasoning, and a clear answer without running out of space.</p><p>The learning rate deserves special attention. We’re using 5e-5, which is roughly ten times higher than you’d typically use for standard fine-tuning. This might seem aggressive, but previous experiments with LoRA have shown that you can — and often should — use higher learning rates when training these low-rank adapters compared to full fine-tuning. The intuition is that because LoRA parameters start from a state where they have zero effect on the model (the adapter is initialized to produce the identity transformation), you need a stronger learning signal to get them moving in the right direction. That said, reinforcement learning typically demands more conservative learning rates than supervised learning, so we’ve balanced these two considerations by settling on this middle ground.</p><p>Before we assemble everything and start training, it’s worth pausing to discuss some important recent advancements in GRPO itself. In our previous work on mathematical reasoning, we used the standard GRPO algorithm as described in DeepSeek’s original paper. That worked well, but the field has moved quickly. Since then, researchers have introduced several refinements to the GRPO algorithm that significantly improve both training stability and final performance. We’re leveraging these newer variants in this project, and they’re worth understanding because they represent meaningful improvements over the baseline approach.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-uhACrzwfhmu3oO7f2W4Gw.jpeg" /><figcaption><a href="http://Group Sequence Policy Optimization">Group Sequence Policy Optimization</a></figcaption></figure><p>First, notice that we’ve set importance_sampling_level to &quot;sequence&quot;. This activates<strong> GSPO—Group Sequence Policy Optimization</strong>—which is a recent refinement of the standard GRPO algorithm. The key difference lies in how off-policy corrections are calculated. In the original GRPO, importance ratios and clipping are computed at the token level, which can sometimes lead to instability when individual tokens have wildly different importance weights. GSPO instead calculates these ratios at the sequence level, treating each complete generation as a unit. This produces more stable training dynamics and often better final performance, particularly in settings like ours where we care about the overall quality and structure of complete responses rather than individual token predictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*43jQ9BmUnddvKRfYksQ5vw.png" /><figcaption><a href="https://arxiv.org/pdf/2503.20783?">Understanding R1-Zero-Like Training: A Critical Perspective</a></figcaption></figure><p>Second, we’ve specified loss_type=&quot;dr_grpo&quot;, which uses a variant called <strong>Dr. GRPO—short for Debiased and Refined GRPO</strong>. This is another small but effective modification that addresses subtle biases in the original GRPO loss function. Specifically, Dr. GRPO removes the length normalization and standard deviation normalization terms that were present in the original formulation. While those normalizations were well-intentioned attempts to make rewards comparable across generations of different lengths, they can actually introduce unwanted biases that hurt performance. By removing them, Dr. GRPO produces cleaner learning signals. It&#39;s remarkable how these seemingly minor algorithmic tweaks—sequence-level importance sampling and debiased loss calculation—can have outsized impacts on training outcomes.</p><p>Finally, we bring all the pieces together and initialize our trainer:</p><pre># Our complete reward system:<br># - formatting_reward: up to +3.0 points (1.0 each for detection, reasoning, answer tags)<br># - reasoning_length_reward: up to +2.0 points (encourages thorough thinking, maxes at ~200 words)<br># - correctness_reward: +5.0 points (exact match to ground truth A/B/C/D answer)<br># Total possible reward: 0.0 to +10.0 points<br><br>training_args = setup_training_config()<br>trainer = GRPOTrainer(<br>    model=model,<br>    args=training_args,<br>    processing_class=tokenizer,<br>    reward_funcs=[formatting_reward, reasoning_length_reward, correctness_reward],<br>    train_dataset=train_dataset,<br>)<br><br># Start training<br>trainer.train()</pre><p>We pass in our configuration, our model with its LoRA adapters, the tokenizer, and critically, our three reward functions that define what “good” means in our training setup. The trainer will handle the complex dance of generating multiple candidate completions for each training example, computing rewards for each candidate, and updating the model to increase the probability of higher-rewarded responses while decreasing the probability of lower-rewarded ones.</p><p>With everything configured, we simply call trainer.train() and let the reinforcement learning process work its magic. Over the course of training, the model will gradually learn to structure its responses according to our three-stage format, to provide thorough visual observations in its detection phase, to reason carefully in its thinking phase, and ultimately to arrive at correct answers. All of this happens through the elegant mechanism of reward-based learning, without us ever having to provide a single example of what a &quot;perfect&quot; detection or reasoning section should look like.</p><h3><strong>Results</strong></h3><p>Well, here comes the moment of truth. After 3–4 hours of training and a reasonable number of training steps, we can already see the model catching on to what we want. The formatting requirement proves to be the easiest one to master — the model quickly learns to structure its outputs with the proper detection, thinking, and answer tags. Next comes the extended thinking phase, where the model gradually learns to produce more thorough reasoning. Finally, we have correctness — the hardest objective to achieve, especially for a small model working with complex diagrams.</p><p>But the trends are encouraging. We can see improvements in the correctness reward over time, along with decreasing variance across batches. Think of this as the model becoming more stable and consistent in generating correct answers rather than occasionally getting lucky. The upward trajectory suggests that with continued training, <strong>even our modest 4B parameter model can learn to reason reliably over visual inputs.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FsZ4uvoG7o3Whio9F-Kwew.png" /><figcaption>Reward Details</figcaption></figure><p>Remember, the total possible reward is 10 points across all three objectives. As you can see from the combined reward graph, the model reaches near-perfect scores after some initial struggle — an excellent result!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/1*3rpbgHGR1mWb6kbixMobNQ.png" /><figcaption>Total Rewards</figcaption></figure><p>Let’s evaluate the model’s performance on an actual dataset example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6HLZurH_1QoFacz8mfO3Jg.png" /><figcaption>Simple Plot of Trained Model Outputs</figcaption></figure><p>The results are impressive. The model successfully implements our three-stage framework: it systematically identifies objects and relationships in the detection phase, synthesizes these observations with the question and answer choices during reasoning, and delivers a concise final answer. Each stage appears within the correct delimiter tags as specified.</p><h3>Conclusion: Reasoning Over Images at Home</h3><p>We’ve demonstrated that a 4B parameter vision-language model can learn structured reasoning over diagrams. The approach is straightforward: define a three-stage format (detection, thinking, answer) and let the model learn to fill it through reward-based feedback.</p><p>After couple hours of training, the model consistently follows the expected format, produces extended reasoning, and shows improving correctness on the AI2D dataset. The progression — formatting first, then thinking length, finally correctness — matches our expectations for how these behaviors would develop.</p><p>The key advantage of this approach is simplicity. We didn’t provide labeled examples of ideal detections or reasoning chains. Three reward functions checking format compliance, thinking length, and answer correctness were sufficient to guide learning. GSPO’s sequence-level importance sampling and the debiased loss formulation provided stable training dynamics.</p><p>With frameworks like Unsloth, vLLM, and TRL, along with LoRA rank 1 and quantization, this training is feasible on consumer hardware. The explicit observation phase before reasoning appears to be particularly important for visual inputs where relevant information needs to be identified before interpretation.</p><p>For whole code you can explore and <a href="https://gist.github.com/ertugrul-dmr/de9b9ffe9b4ba1329c6f4dc1ca2b3617"><strong>modify this script here</strong></a><strong>.</strong></p><p>Future work could explore more complex visual tasks, alternative reward formulations, or applying this structured approach to other modalities.</p><h3>Resources:</h3><ul><li><a href="https://arxiv.org/abs/2501.12948">DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning</a></li><li><a href="https://arxiv.org/abs/2402.03300">DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models</a></li><li><a href="https://medium.com/@ertugruldemir/reasoning-models-at-home-760310732f6b">Reasoning Models at Home: How to Train Your Own Specialized Reasoning Model at Home</a></li><li><a href="https://unsloth.ai/blog/gemma3">Fine-tune &amp; Run Gemma 3</a></li><li><a href="https://arxiv.org/abs/2503.20783">Understanding R1-Zero-Like Training: A Critical Perspective</a></li><li><a href="https://arxiv.org/pdf/2507.18071">Group Sequence Policy Optimization</a></li><li><a href="https://thinkingmachines.ai/blog/lora/">LoRA Without Regret</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a069eecd25d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Your Own Agentic Newsletter]]></title>
            <link>https://ertugruldemir.medium.com/creating-your-own-agentic-newsletter-1864da937b77?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/1864da937b77</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Tue, 27 May 2025 14:43:22 GMT</pubDate>
            <atom:updated>2025-05-27T14:43:22.441Z</atom:updated>
            <content:encoded><![CDATA[<h4>Leverage Agent Development Kit and Gemini models to curate your own tailored Reddit summaries</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PXG_-APtubD47WjRDQyLBQ.png" /><figcaption>Source: Author</figcaption></figure><p>You’re probably hearing a lot about agents these days, with companies everywhere building “awesome” products based on them. Maybe you’re thinking, “Oh, that’s cool, but how do I get started with them personally?” And that’s where the complication lies: agents are cool to use, but when it comes to implementing them, things change a little bit.</p><p>In this article, I’m going to take a more practical approach and jump straight into the action. (I’ve previously shared some of my thoughts on agents and agentic workflows in another article, which you can find <a href="https://ertugruldemir.medium.com/flash-agents-building-efficient-and-effective-ai-with-gemini-7e440748dc8d">here</a>).</p><h3>The Use Case</h3><p>I believe defining the use case is one of the most underrated parts of implementing these systems. People often tend to jump into complex agents without first deciding what problem they’re solving. They either choose a truly trivial problem, like “What time is it?”, or an overly complex one, such as “Here’s my whole credit card, go book me the cheapest and best holiday in Paris.” While these might or might not work in practice, a balanced approach is best for learning, in my opinion. Plus, tackling a problem that solves your own daily issues, rather than just textbook examples, can really spark your creativity.</p><p>In this article, we’re going to implement a simple agentic system that solves a small problem of mine. We all know how hard it is to keep up with daily AI and LLM news. I’ve always wanted to get daily or curated summaries, tailored to my personal interests, about the trends in these areas.</p><p>For reliable, up-to-date information, I personally rely on specific subreddits and some social media users I follow. For this tutorial, we’ll leverage one of them — Reddit — which offers a more structured format and a somewhat pseudonymous environment. While we’ll start with a single source, it’s absolutely possible to merge multiple sources with agentic systems, as you’ll see throughout this tutorial.</p><h3>The Setup</h3><p>Our setup will be minimal, and thanks to ADK’s straightforward approach, getting started is quite easy. Here are the main packages we are going to use:</p><p><strong>Agent Development Kit (ADK):</strong> The Agent Development Kit (ADK) is an open-source framework from Google, designed to simplify the often-complex process of building and deploying advanced multi-agent AI applications. Think of it as a sophisticated orchestration layer that allows you to move beyond isolated AI models, enabling a team of specialized AI agents to work together intelligently, each contributing its unique capabilities to achieve a larger goal. This kit provides a comprehensive toolkit for developers, making it easier to manage the entire lifecycle of multi-agent systems. It offers remarkable flexibility, allowing seamless integration of diverse AI models and tools, including those from external libraries or even other agents as sub-tools. When it’s time to bring these systems to life, ADK simplifies deployment by supporting containerization and optimizing for Google Cloud environments, particularly leveraging services like Vertex AI and Gemini models.</p><p><strong>Python Reddit API Wrapper (PRAW):</strong> PRAW is a Python library that simplifies interacting with the Reddit API. It provides a user-friendly interface for accessing Reddit’s data and functionalities, making it easier to collect and analyze Reddit information. PRAW also handles the complexities of the API, like rate limits and authentication, so developers can focus on the data itself. This greatly simplifies our use case, especially when integrating with ADK.”</p><p>First, let’s create an environment of your choice and install these packages. I prefer a simple virtual environment, and then install:</p><pre>google-adk<br>praw<br>pydantic<br>python-dotenv</pre><p>Your agents should have their own folders, here is an example structure:</p><pre>parent_folder/<br>    multi_tool_agent/<br>        __init__.py<br>        agent.py<br>        .env</pre><p>And inside your __init__.py should look like this:</p><pre>from . import agent</pre><p>The agent.py will be the main one we will work in incoming parts, so you can leave as it is for now…</p><p><strong>Environment and Credentials:</strong></p><p>First, we need access to an LLM API, whether as a service or locally hosted, to power our agent. For simplicity, we’ll use Google AI Studio. It perfectly fits our use case since it’s easy to set up and get API keys. The good news is AI Studio lets you use up-to-date state of the art Gemini models for free, and its daily limits are certainly more than we’ll need. You can, of course, use other providers or Vertex AI if you prefer.</p><p>You can visit <a href="https://aistudio.google.com/apikey">https://aistudio.google.com/apikey</a> to get your API keys ready.</p><p>Next, we need to connect our personal Reddit account to fetch data. If you don’t have an account, you can create one easily. For the authentication part, you can follow this comprehensive guide: <a href="https://praw.readthedocs.io/en/stable/getting_started/authentication.html">https://praw.readthedocs.io/en/stable/getting_started/authentication.html</a></p><p>Once you have your credentials ready, create a .env file in your project folder (specifically within your ADK agent directory). It should look something like this:</p><pre>GOOGLE_GENAI_USE_VERTEXAI=FALSE<br>GOOGLE_API_KEY=&lt;your_studio_api_key&gt;<br><br><br>REDDIT_CLIENT_ID=&quot;YOUR_REDDIT_CLIENT_ID&quot;<br>REDDIT_CLIENT_SECRET=&quot;YOUR_REDDIT_CLIENT_SECRET&quot;<br>REDDIT_USER_AGENT=&quot;YOUR_REDDIT_USER_AGENT&quot;<br>REDDIT_USERNAME=&quot;YOUR_REDDIT_USERNAME&quot;<br>REDDIT_PASSWORD=&quot;YOUR_REDDIT_PASSWORD&quot;</pre><p>Okay, our setup is now complete, and we’re ready to dive in!</p><h3>Our first agent</h3><p>An LlmAgent is a core component of ADK that uses LLM models to think, decide, and act. There are a couple of types of agents in ADK, but we’ll be focusing on LLM-Based ones since they can reason and are quite fun to experiment with. For more robust and structured multi-step processes, you can also leverage ‘Workflow’ agents, which are great for orchestrating predefined sequences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lfVEcH8iAICGeYKFkswfdA.png" /><figcaption>Primary Agent Types from official documentation</figcaption></figure><p>Ultimately, we’ll have multiple agents that can interact with each other, call tools, and pass their results between them. But for starters, let’s talk about our ‘root agent’ (or ‘primary agent,’ ‘main agent,’ etc., whatever you prefer to call it). Think of this as our entry point — a mediator between the user and the entire system, an organizer to orchestrate tasks. It’s the agent that will have the power to delegate to other agents and tools.</p><p>But how does an agent know what to do? Well, it’s simple: as with many LLMs, they operate based on instructions. This might be the most important and creative part of working with LLM-Based agents. They need to know their role, purpose, and scope. Here are some simple rules of thumb from the official documentation for crafting these instructions:</p><blockquote><strong>Include:</strong></blockquote><blockquote>Its core task or goal.</blockquote><blockquote>Its personality or persona (e.g., “You are a helpful assistant,” “You are a witty pirate”).</blockquote><blockquote>Constraints on its behavior (e.g., “Only answer questions about X,” “Never reveal Y”).</blockquote><blockquote>How and when to use its tools. You should explain the purpose of each tool and the circumstances under which it should be called, supplementing any descriptions within the tool itself.</blockquote><blockquote>The desired format for its output (e.g., “Respond in JSON,” “Provide a bulleted list”).</blockquote><blockquote><strong>Tips for Effective Instructions:</strong></blockquote><blockquote><strong>Be Clear and Specific:</strong> Avoid ambiguity. Clearly state the desired actions and outcomes.</blockquote><blockquote><strong>Use Markdown:</strong> Improve readability for complex instructions using headings, lists, etc.</blockquote><blockquote><strong>Provide Examples (Few-Shot):</strong> For complex tasks or specific output formats, include examples directly in the instruction.</blockquote><blockquote><strong>Guide Tool Use:</strong> Don’t just list tools; explain <em>when</em> and <em>why</em> the agent should use them.</blockquote><p>Beyond the detailed instructions we just discussed, each agent also needs some metadata to define its identity and capabilities within the system. This helps other agents (and the overall system) understand when, how, and why to interact with it.</p><ul><li>Name: Choose a descriptive name.</li><li>Description: While optional, this is very useful. Try to provide a short summary of the agent’s capabilities here.</li><li>Model: This is a pretty obvious but crucial parameter, as it defines the ‘raw power’ of your agent. For simple tasks, for example, you can opt for smaller, more efficient models, while complex tasks might require more powerful ones.</li></ul><p>Here’s a simple, bare-bones agent instance example from the official documentation:</p><pre>capital_agent = LlmAgent(<br>    model=&quot;gemini-2.0-flash&quot;,<br>    name=&quot;capital_agent&quot;,<br>    description=&quot;Answers user questions about the capital city of a given country.&quot;<br>)</pre><p>Let’s assume a simple scenario: you have a root_agent and our capital_agent. When you ask, “What is the capital of Germany?”, instead of replying directly (which is also possible if not specifically instructed, especially for such an easy task), the root agent will delegate this task. It will look at its given list of tools and sub-agents (think of a sub-agent as an agent that can be invoked by its parent agent). Conveniently for our root agent, we happen to have an agent called capital_agent designed for exactly this! So, the root agent will send a request to the capital_agent and return its response to the user.</p><p>And that’s it — this is the simplest form of a multi-agent setup!</p><p>Before going further with agents, let’s talk about another important part: tools.</p><h3>Tools</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mNMt-7bPk756iv125_LKbA.png" /><figcaption>Simple tool included flow from the official documents</figcaption></figure><p>Tools are special functions or code components that allow your AI agent to perform actions and interact with the world beyond its core text generation and reasoning abilities. They enable it to:</p><ul><li><strong>Act &amp; Interact:</strong> Perform tasks like querying databases, calling APIs (e.g., for Reddit), searching the web, or running code.</li><li><strong>Extend Capabilities:</strong> Access real-time information and external systems, overcoming knowledge limits.</li><li><strong>Execute Defined Logic:</strong> Tools run specific, pre-programmed tasks; the AI (LLM) decides which tool to use and when.</li></ul><p>Similar to agents, it’s always good practice to clearly define your tools, as this will help your agents understand how to use them effectively.</p><p><strong>Designing Good Tool Functions:</strong></p><ul><li><strong>Name:</strong> Descriptive (e.g., get_subreddit).</li><li><strong>Docstring (Comments):</strong> Crucial! Explain what the tool does, its parameters, and what it returns. This is how the AI understands it.</li><li><strong>Parameters:</strong> Clear names, use type hints.</li><li><strong>Return Value:</strong> Always a dictionary (e.g., {‘status’: ‘success’, ‘details’: ‘…’}).</li><li><strong>Focus:</strong> One tool, one clear job.</li></ul><p>Throughout this project, we’ll be using many tools and agents. In fact, for some functionalities, we’ll even create simple agents that <em>wrap</em> other tools. This is a current workaround because, with the ADK’s current version, calling multiple tools simultaneously on AI Studio can sometimes be problematic. However, by wrapping these tools within another LLM instance, they tend to work quite well. This behavior is expected to be improved in future versions.</p><p><strong>Creating Agents and Tools</strong></p><p>So, let’s remember our core task: we want to feed user queries about a topic to our agent(s) and instruct them to provide summaries. Recalling our first principle for instructions — be clear and specific — we know that being too ambiguous is not recommended.</p><p>One of the most important parts is to identify the complex aspects of our task and create specific tools or sub-agents to solve those complexities. For this, I recommend you think as if you’re solving or orchestrating that problem yourself. In our case, we want to ask about a topic and get popular Reddit conversations about it, either recently or within a given timeframe.</p><p>First things first, you need to understand the capabilities of your LLM models. When a user asks our root agent about a topic, its underlying LLM will try to understand which subreddits might be related to that query. In our case, we simply need to consider these two questions:</p><ul><li>Is the model’s knowledge up-to-date?</li><li>Can a generic agent, without external tools, adequately solve the query and map it to relevant search parameters?</li></ul><p>For most queries, if not directly specified by the user, there are likely multiple subreddits. However, the challenge is that some are either well-forgotten in the pages of internet history, or too new to be within the LLM’s knowledge cut-off date. To address this, and for the sake of this tutorial, let’s start with a simple sub-agent that has a sole, clear, and focused task. And believe me, at their core, all of our agents will follow this same logic.</p><h4>The Analyze Agent</h4><p>Let’s create our first specialized agent in this multi-agent setup. (Technically, our root_agent was the first, but let’s consider this our first <em>task-specific</em> agent). Think about the scenario again: user_query leads to root_agent. This is exactly where we are now.</p><p>Assume an user asks: “What are the recent news about Google’s language models?” The root agent will take this, and as instructed, it’s not going to solve this problem itself (remember the agent’s instructions); it needs to delegate. What would be a good candidate for delegation here?</p><p>The agent knows it’s about Google and language models, but which family of models? Gemini or open-source models like Gemma? There are multiple subreddits for each. Even more generic but highly active subreddits like r/LocalLLaMA might include discussions about both. The Gemini models, for instance, were previously called ‘Bard’, and while some of their old subreddits are still active for Gemini-related discussions, there are also dedicated Gemini subreddits that aren’t as active.</p><p>These are many questions to answer. Let’s not burden the root agent with all that heavy thinking and instead create a new agent solely for this purpose: the analyze_agent:</p><pre>analyze_agent = Agent(<br>    name=&quot;analyze_agent&quot;,<br>    model=&quot;your_llm_name&quot;,<br>    instruction=&quot;You are an expert Reddit user with immense knowledge about subreddits and access to specialized web search capabilities. &quot;<br>    &quot;When presented with an user query, your responsibility is to identify and return a list of the most relevant, popular, and active subreddit names. &quot;,<br>    description=&quot;An expert Reddit user that analyzes queries to identify and recommend the most relevant, popular, and active subreddit names for any given topic or discussion area.&quot;,<br>)</pre><p>Alright, a simple agent for mapping queries into subreddits. It solves one of our problems with clear instructions. However, our model still has an LLM-induced limitation: up-to-date information.</p><p>This is where tools truly shine! We can instruct models to use specific tools and leverage their capabilities to augment their context. In our case, we can simply use the built-in google_search tool to fetch up-to-date information. For this, we need to update our analyze_agent:</p><pre>analyze_agent = Agent(<br>    name=&quot;analyze_agent&quot;,<br>    model=&quot;your_llm_name&quot;,<br>    instruction=&quot;You are an expert Reddit user with immense knowledge about subreddits and access to specialized web search capabilities. &quot;<br>    &quot;When presented with a user query, your responsibility is to identify and return a list of the most relevant, popular, and active subreddit names. &quot;<br>    &quot;Use the following approach: &quot;<br>    &quot;1. **Web Search First**: Use the &#39;google_search_agent&#39; to discover what subreddits are commonly recommended or mentioned in web discussions about the topic. &quot;<br>    &quot;2. **Knowledge Enhancement**: Combine the search results with your existing knowledge about subreddits to create a comprehensive list. &quot;<br>    &quot;3. **Quality Focus**: Prioritize subreddits that are likely to have active discussions related to the query, based on both search results and your knowledge. &quot;<br>    &quot;Your output should be a list of subreddit names in the format [&#39;r/subreddit1&#39;, &#39;r/subreddit2&#39;] along with brief reasoning from your search findings.&quot;,<br>    description=&quot;An expert Reddit user that analyzes queries to identify and recommend the most relevant, popular, and active subreddit names for any given topic or discussion area.&quot;,<br>    tools=[agent_tool.AgentTool(google_search_agent)] # wrapping for async tool call fix<br>)</pre><p>And there it is: our specialized agent, designed to break down queries and perform web searches for subreddit names, is ready! As you can see, we’ve equipped it with a tool and clearly instructed it on what the tool is for, how to use it, and what to return. Now, our root agent has another powerful teammate to help it create a list of likely subreddit names.”</p><h4>The Custom Tools</h4><p>The google_search_agent was built-in and easy to use. However, for Reddit, we’ll need some specialized custom tools. As I mentioned before, we’re going to use the PRAW library for this purpose. Remembering our best practices for designing good tool functions, we need clear naming, detailed docstrings, type hints, and a separate tool for each breakable task. It’s also great practice to add Pydantic schemas for tool inputs and outputs, which makes them more robust and self-documenting.</p><p>I’ll be including the full code in the GitHub repo when it’s clean enough to share =), but here’s a snippet to give you an idea:</p><pre>def find_subreddits(query: str, limit: int) -&gt; dict:<br>    &quot;&quot;&quot;<br>    Searches for and discovers relevant subreddits based on a query string, returning active public communities.<br><br>    This tool performs semantic search across Reddit&#39;s subreddit directory to find communities that match <br>    the search query. Results are filtered to include only public, accessible subreddits and ranked by <br>    active user count to prioritize vibrant communities.<br><br>    Args:<br>        query (str): The search term for finding relevant subreddits (e.g., &#39;machine learning&#39;, &#39;cooking&#39;, &#39;python programming&#39;).<br>        limit (int): The maximum number of subreddit suggestions to return (default: 25).<br><br>    Returns:<br>        dict: A dictionary with &quot;status&quot; and &quot;result&quot;.<br>              On success, &quot;result&quot; contains formatted subreddit information including names, descriptions, <br>              and active user counts, or a helpful message if none are found.<br>              On error, &quot;result&quot; contains a descriptive error message.<br>    &quot;&quot;&quot;<br><br>### ... rest of the function ...</pre><p>As you can see, the clearer and more descriptive your tool definitions are, the better your agents will be able to leverage them.</p><p>For this task, we’ll build the following custom tools:</p><ul><li><strong>find_subreddits:</strong> This tool is for searching and fetching subreddit information (including names, descriptions, etc.) for our agents to use.</li><li><strong>get_subreddit_posts: </strong>This tool fetches posts from any public subreddit(s) with flexible sorting options, and can optionally include top comments for the highest-scoring posts. It’s perfect for content summaries, trend analysis, and community insights. For our agents, it will return formatted post information including titles, URLs, scores, content, and optionally top comments with author details and scores.</li><li><strong>get_comments: </strong>This tool extracts comments from any accessible Reddit submission, providing insights into community discussions and reactions. Comments are filtered to exclude deleted content and include author information, scores, and direct permalinks for easy navigation. This is particularly useful for follow-up questions if a query is interested in a specific post mentioned.</li></ul><h4>The Multi Agent Structure</h4><p>Now that you understand how to create a simple agent and how to connect it to others or tools, the rest is about creating a bunch of these with the same principles and connecting the dots to solve our problem. After some thought, I came up with this structure, but feel free to simplify or expand upon it for your specific needs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l-bZWMIRicpOnd2xr7fH7w.png" /><figcaption>Agent &amp; Tool Structure. Created by author.</figcaption></figure><p>Here’s our simple multi-agent network. At the top, everything is connected to the root_agent. This root_agent knows about all five of our sub-agents, each with clear definitions, instructions, and tools, and has its own overarching instructions too. We’ve already mentioned some of our tools and (sub) sub-agents previously. I’ve tried to color-group them in the schema, but let me briefly explain each one:</p><ul><li><strong>subreddit_agent:</strong> A specialist agent that finds and confirms the most appropriate subreddits for user queries by leveraging analysis and search capabilities to return verified, relevant subreddit names.</li><li><strong>reddit_reader_agent:</strong> A content aggregation specialist that fetches, filters, and structures Reddit posts (including their top comments) from specified subreddits, providing comprehensive summaries and daily updates.</li><li><strong>single_post_agent:</strong> A detailed post analyzer that retrieves comprehensive information for specific Reddit posts using URLs, including metadata, content, and a configurable number of top comments.</li><li><strong>response_agent: </strong>A presentation specialist that transforms structured Reddit data into engaging, well-formatted Markdown responses with proper headings, links, and user-friendly layouts for optimal readability.</li><li><strong>single_post_response_agent:</strong> Similar to the response_agent, but specialized in formatting responses for individual Reddit posts.</li></ul><h3>ADK in Action</h3><p>Alright! Time to test our setup! Let’s use ADK’s simple web interface by running this command from your project’s root folder:</p><pre>adk web</pre><p>The interface is straightforward and easy to get started. Once the local server is up and you’ve navigated to the page in your browser, select the subfolder where we’ve built our agent.</p><p>At its core, our problem was getting summarized, up-to-date Reddit conversations about a given topic. I’ll ask a simple question. It might take a few seconds for the whole system to finish its final touches, but here are the results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*5CfyD1eBQTQvUWe0nW3S6g.gif" /><figcaption>Source: Author</figcaption></figure><p>Here it is! I asked for: “Give me a weekly summary of open-source large language models, please,” and as you can see, it returned mostly discussed, up-to-date topics in a nicely readable format — pretty cool!</p><p>The web interface also allows you to inspect each event and trace the requests and responses exchanged between agents.</p><p>For example, our subreddit_agent analyzed the query with Google Search support and returned this:</p><pre>name: &quot;reddit_subreddit_agent&quot;<br>response:<br>  result: &quot;Based on my analysis, here are some relevant subreddits for open source large language models:<br>    * **r/LocalLLaMA**: Highly relevant, specifically focused on the Llama model (685 active users).<br>    * **r/LocalLLM**: Relevant for locally run large language models (40 active users).<br>    * **r/MachineLearning**: A general but active community that discusses various machine learning topics, including LLMs (136 active users).<br>    * **r/ArtificialInteligence**: Another general but relevant community for AI discussions (130 active users).&quot;name: &quot;reddit_subreddit_agent&quot;<br>response:<br>result: &quot;Based on my analysis, here are some relevant subreddits for open source large language models: * **r/LocalLLaMA**: Highly relevant, specifically focused on the Llama model (685 active users). * **r/LocalLLM**: Relevant for locally run large language models (40 active users). * **r/MachineLearning**: A general but active community that discusses various machine learning topics, including LLMs (136 active users). * **r/ArtificialInteligence**: Another general but relevant community for AI discussions (130 active users).&quot;</pre><p>Then our root_agent took this information and passed it to the reddit_reader_agent in this format:</p><pre>request: &quot;Fetch top 25 posts from r/LocalLLaMA+r/LocalLLM for the last week, sorted by &#39;top&#39;, time_filter &#39;week&#39;. For the top 10-15 posts from this list, also fetch their top 3 comments.&quot;</pre><p>won’t paste the rest of the raw outputs here because they are quite verbose. But basically, the reddit_reader_agent returned results following this request and its own instructions, and then the response_agent took this response to format it for the final output.</p><p>Okay, let’s also test our single-post response capability within the agent system. I noticed ‘Gemma 3n Preview’ in the summarized content, so I asked: “Can you tell me more about the Gemma 3n Preview post, please?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/911/1*by_ss23WvCf8PIP1bFXpWg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/884/1*sJO2Lc9c8MHscQsr5uHDbQ.png" /><figcaption>Source: Author</figcaption></figure><p>Amazing! The follow-up query gave me a really detailed post summary — maybe even more than I needed! (This indicates a good opportunity for future prompt fine-tuning.)</p><p>I just animated the order of calls between the root and agents here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*uJfEmxAjowFYUbdfetKzZw.gif" /><figcaption>Source: Author</figcaption></figure><h3>Conclusion</h3><p>And there you have it! We’ve journeyed from a common problem — keeping up with the flood of AI news — to building our very own agentic newsletter, powered by the Agent Development Kit and Gemini models. We’ve seen how breaking down a complex task into manageable agents, each with clear instructions and specialized tools, can create a surprisingly powerful and personalized system.</p><p>Hopefully, this practical walkthrough has demystified the process of building agentic applications and sparked some ideas. The setup we’ve created for Reddit summaries is just a starting point. Imagine extending this to other news sources, different types of content, or entirely new personal productivity tools! For instance, with a few more specific prompts and perhaps an email integration tool, you could easily adapt this system to send you these curated summaries directly to your inbox on a weekly basis.</p><p>The world of AI agents is rapidly evolving, and frameworks like ADK are making it more accessible than ever to experiment and build. While my initial experiments with more complex setups were kindly supported by GCP credits from the AI GDE team on Vertex AI, the version we’ve built today is designed to be largely free to experiment with using Google AI Studio and its generous free tier. So, I encourage you to take these concepts, and see what unique solutions you can create to solve your own “small problems.”</p><p>Happy building!</p><p><strong>Link to the full code:</strong> <a href="https://github.com/ertugrul-dmr/reddit_curator">https://github.com/ertugrul-dmr/reddit_curator</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1864da937b77" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Flash Agents: Building Efficient and Effective AI with Gemini]]></title>
            <link>https://ertugruldemir.medium.com/flash-agents-building-efficient-and-effective-ai-with-gemini-7e440748dc8d?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e440748dc8d</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Mon, 17 Feb 2025 16:30:53 GMT</pubDate>
            <atom:updated>2025-02-22T13:26:19.670Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Guide to Agentic Systems with smolagents and Vertex AI</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GvYHN67NiJnJjk7S5Vy84g.jpeg" /><figcaption>An image of agentic wizard. Generated by the code at the last section</figcaption></figure><h4>The Rise of Agentic Systems</h4><p>In past couple months, we’re seeing incredible advancements in LLMs or let’s say AI in general, with models becoming significantly more capable, not just in benchmarks, but in their ability to perform complex tasks and interact with the real world. But what’s really exciting is the emergence of models specifically designed for building <strong>agentic experiences</strong> with more tool support and native agentic additions.</p><p>Google recently expanded its Flash model series with two new additions: <em>Gemini 2.0 Flash</em> and <em>Gemini 2.0 Flash Lite</em>. <em>Gemini Flash</em> is designed for <strong>exceptional speed and cost-effectiveness</strong>, while still offering strong capabilities in tool usage and agentic behavior. <em>Gemini Flash Lite</em>, meanwhile, is Google’s <strong>most cost-effective model yet</strong>, making agent development more accessible and feasible than ever before.</p><p>This is significant because Google has strongly emphasized an “<strong><em>agentic era</em></strong>” in the release of these models, signaling a major commitment to future <strong>agent-based development</strong>. Crucially, these <em>Flash</em> models are readily available through platforms designed for different stages of development. <em>Google AI Studio</em> provides a user-friendly environment for <strong>prototyping and building agents</strong>. For deployment at scale, <em>Vertex AI</em> offers the infrastructure and tools necessary to create <strong>robust, production-ready agentic systems</strong>. This opens up possibilities like building teams of <strong>specialized AI assistants</strong> that are not only intelligent, but also extremely fast and budget-friendly.</p><p>In this article, we’re going to dive into the world of these “<strong>Flash Agents</strong>.” We’ll start by exploring the recent advancements in AI that have paved the way for agentic systems. Then, we’ll get crystal clear on what exactly <strong>Agentic AI</strong> is and how these agents work in theory and practice. We’ll look at compelling use cases where speed and efficiency are paramount. We’ll also discuss how reasoning models play a crucial role in orchestrating agent workflows, and even touch upon the “<strong>Agent Leaderboard</strong>” to see who’s pushing the boundaries. Finally, we’ll bridge the gap to the technical side, briefly exploring some of the agentic frameworks and even provide some simple starter code examples to get you hands-on with building your own “<strong>Flash Agents</strong>.”</p><h4><strong>What Exactly <em>Is</em> Agentic AI?</strong></h4><p>Agentic AI, you might hear it thrown around a lot, and it can sound a bit broad, almost like a catch-all term for different AI ideas. So, let’s focus on the core idea: the “<strong>Agent</strong>” itself. “Agent” definitely sounds fancy, like digital helpers running around in the background. But in the world of <em>Agentic AI</em>, what does it really mean?</p><p>Think of an agent as something that has a goal and can autonomously work towards achieving that goal. It’s not just about executing pre-defined instructions. It can make decisions, interact with its environment, and importantly, choose the right tools for the job. For example, a simple function that scrapes website content or an API call to transfer data are tools, not agents themselves. These tools are passive; they execute a specific task when called upon. An agent, in contrast, might decide that it needs to scrape a website in order to fulfill its goal of, say, “find the latest price for product X.” It might then choose to use a data processing tool and then store the result in a database — all driven by its initial goal.</p><p>However, the capabilities of agents extend far beyond this simple example. This is where Large Language Models come into play. These sophisticated agents leverage the reasoning and language understanding capabilities of LLMs to handle much more complex tasks and goals. They can be designed to take on different roles, almost like having different personalities or skillsets, specializing in areas like code generation, image understanding (vision-language models), or complex data analysis. These complex, LLM-powered agents are not simply tool-users; they are capable of intricate planning, multi-step reasoning, and even managing a network of other, more specialized agents to achieve a larger objective. Think of a project manager AI that delegates tasks to different specialized agents (each of which might be using various tools) to get a complex project done. The LLM-powered agent is orchestrating the entire process, adapting to new information, and making high-level decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*deTpPFtOykuCz0KLsZs7BQ.png" /><figcaption><a href="https://arxiv.org/pdf/2210.03629">Source: React: Synergizing Reasoning and Acting in Language Models</a></figcaption></figure><p>Ultimately, <em>Agentic AI</em> is about building systems that can <strong>reason, act, and reflect</strong>. The “<strong>reason</strong>” part — the thinking or planning — is definitely key. It’s like the agent’s brain figuring out what needs to be done and how to do it. It’s the process of deciding which agent, or combination of agents, is the right tool for the job at hand. This planning stage is crucial for breaking down complex problems into manageable steps.</p><p>Now, the “<strong>act</strong>” part is where the agent actually <em>does</em> something. It’s the <strong>execution phase</strong>. Based on its reasoning, the agent takes action in its environment. This could be anything from making an API call to fetch data, writing some code to perform a calculation, sending a message to another agent, or even interacting with a user. Think of “act” as the agent performing the specific task it was designed for, based on the plan it formulated in the “reason” stage.</p><p>And finally, there’s the “<strong>reflect</strong>” part. This is where the agent examines the outcome of its action. It’s primarily about <strong>validation</strong>: Did the action achieve the intended goal? Was it successful? Did it produce errors or unexpected outcomes? The agent assesses the results against its expectations. For example, if the agent was supposed to fetch data from an API, the “reflect” step would check if the data was successfully retrieved, if it’s in the correct format, and if there were any API errors. This “reflect” step provides crucial feedback in case of failure, to retry or let the user know it fails.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t7H92jZrSZ3_tiiaNJMwrg.gif" /><figcaption><a href="https://huggingface.co/docs/smolagents/conceptual_guides/intro_agents">Source: Huggingface</a></figcaption></figure><p>Think of assembling a party in a fantasy role-playing game. The <strong>party leader</strong> character — often the wisest and most versatile — takes on the role of <strong>orchestrator</strong>. Each member of your party has <strong>specialized skills</strong>: the <em>wizard</em> excels at ranged magical attacks, the <em>warrior</em> is a melee combat expert, the <em>rogue</em> is adept at stealth and reconnaissance. The party leader understands these specializations. A good party leader wouldn’t send a wizard charging into a melee fight, but know exactly when to call upon the wizard’s fireballs to turn the tide of battle. Similarly, in <em>Agentic AI</em>, the “<strong>orchestrator</strong>” model understands the strengths and weaknesses of each tool and strategically deploys them to solve complex problems by breaking them down into manageable, well-defined tasks.</p><h4>When to Use Agents</h4><p>The agentic approach offers significant advantages in specific scenarios. While there are no strict rules, consider using agents when a task:</p><ul><li>Comprises <strong>multiple subtasks</strong>.</li><li>Involves <strong>repetitive actions</strong> with minimal changes.</li><li>Requires a <strong>non-sequential order of operations</strong>.</li><li>Spans <strong>different domains or environments</strong>.</li><li>Has a <strong>short feedback loop</strong> that can be validated <em>internally</em>.</li></ul><p>Conversely, an agentic approach may <em>not</em> be the most practical choice (at least currently) when:</p><ul><li>The task is <strong>singular</strong> or cannot be decomposed into smaller parts.</li><li><strong>Simpler automation methods</strong> are sufficient.</li><li>A <strong>specialized model</strong> within the specific domain can handle the task <em>independently</em>.</li><li>The outcome is <strong>highly subjective</strong> or difficult to validate <em>objectively</em>.</li></ul><h4>Reasoning Models vs Agentic Reasoning</h4><p>We are currently at the forefront of AI advancements, and as such, terminology can sometimes be broad or confusing. <strong>Reasoning models</strong> are a prominent topic, <a href="https://medium.com/@ertugruldemir/reasoning-models-at-home-760310732f6b"><strong>and you can find a brief overview in my previous article</strong></a><strong>.</strong></p><p>In the context of reasoning models, the focus is on <strong><em>inference-time thinking</em></strong>. This involves training models to be more verbose and gradually reach out the final answer by building upon it’s previous tokens (like thoughts), akin to “<strong>thinking aloud</strong>.” Research has shown that arriving at conclusions gradually within a larger context often yields better results than providing direct answers. You can observe this process in action with <em>Gemini 2.0 Flash Thinking</em> in <em>AI Studio</em> by expanding the “<strong>Thoughts</strong>” tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/364/1*IhZDeppk8Rsj6lajWzuIAQ.png" /><figcaption>An Example from Thought process of Gemini</figcaption></figure><p><strong><em>Agentic reasoning</em></strong>, or more accurately, <strong><em>planning</em></strong> within an <em>agentic design context</em>, refers to the process of models delegating tasks. While these two concepts share a fundamental similarity: <strong>problem-solving through decomposition</strong> — they are technically distinct. The key difference lies in <em>who</em> or <em>what</em> is doing the decomposition, and <em>how</em> it is done. In <strong>reasoning models</strong>, the model itself breaks down the problem and solves it <em>concurrently</em> during the <em>inference process</em>. In <strong>agentic reasoning</strong>, a model (often a <strong>larger, coordinating model</strong>) breaks down the problem and assigns subtasks to <em>other t</em>ools.</p><h4><strong>Why Not Always Use the Most Powerful Reasoning Models?</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*axf_tuuT0863lbkY0Jc6Cg.png" /><figcaption><a href="https://huggingface.co/spaces/galileo-ai/agent-leaderboard">Source</a></figcaption></figure><p>It’s a natural question: if we have incredibly powerful reasoning models, why not use them for <em>everything</em>? There’s nothing stopping you, of course, but there are a few key factors to consider that often make a multi-agent approach, especially with models like Gemini Flash, a more strategic choice:</p><ul><li><strong>Cost-Effectiveness:</strong> While large reasoning models excel at many tasks, their power comes at a price. You’re typically using them in one of two ways: either through API calls, which charge per token, or by investing heavily in dedicated cloud or local resources. Either way, costs can quickly escalate. Remember, reasoning models are inherently verbose — they “think out loud,” generating many tokens. In agentic systems, which often involve numerous back-and-forth interactions (loops), this verbosity translates directly to higher expenses.</li><li><strong>Speed and Iteration:</strong> Closely related to cost is the issue of speed. Agentic designs are inherently iterative; the number of steps required to complete a task is often unpredictable. In many situations, particularly during experimentation and development, it’s far more practical to have rapid, smaller iterations than to wait for a slower, more powerful model to complete each step. Faster feedback loops allow for quicker adjustments and a more agile development process.</li><li><strong>Specialists vs. Generalists:</strong> This is a crucial point. While large, generalist models are remarkably capable across a wide range of domains, sometimes a specialist is simply better. We’re increasingly seeing this in specific areas like coding or mathematical reasoning, where smaller, specialized models, often with far fewer parameters, outperform their larger counterparts <em>on relevant benchmarks</em>. This is particularly relevant in multi-agent systems. Why use a massive generalist to handle a task that a smaller, highly-tuned specialist can do more efficiently and accurately? Think back to our RPG party analogy — sometimes you need the rogue’s stealth and lockpicking skills, or the warrior’s combat prowess, rather than relying solely on the wizard’s broad magical abilities. <em>And, just like a wizard’s powerful spells might be limited by mana or daily uses, a large language model’s resources are valuable.</em> Each member brings unique strengths to the table, and efficient resource management is key.</li></ul><p>In essence, the core idea is about strategic resource allocation. Instead of relying solely on a single, all-powerful model, we can orchestrate a team of smaller, specialized models (whether they’re fine-tuned, carefully prompted, or purpose-built). This “team of experts” approach is, I believe, the heart of effective agentic systems, and it’s precisely where models like Gemini Flash can truly excel. They offer a balance of capability, speed, and cost-effectiveness that makes them ideal for building and deploying these kinds of agentic workflows.</p><h4>Evaluating Agents: Benchmarking Efficiency and Versatility</h4><p>One of my primary arguments for advocating Flash Agents centers on efficiency, speed, and versatility. However, as many in this community know, claims mean little without robust benchmarking. While numerous function-calling benchmarks exist, they often lack the broad scope needed to draw definitive conclusions about agentic capabilities. Fortunately, the Agent Leaderboard on Hugging Face provides a much-needed solution. This relatively new benchmark evaluates LLM agent capabilities across various dimensions in a robust, comprehensive setting.</p><p>We’ve already discussed that agents are not merely about calling tools; it’s about <em>when</em>, <em>how</em>, and in <em>what order</em> those tools are employed. Recognizing this, the creators of the Agent Leaderboard have curated a diverse set of scenarios from different domains and developed a key metric: <strong>Tool Selection Quality (TSQ)</strong>. TSQ determines whether an agent has selected the correct tool and, crucially, has provided the correct arguments for each tool.</p><p>This TSQ metric, coupled with the diverse dataset, allows us to assess multiple critical dimensions of agent performance. These include: basic tool usage (understanding documentation, parameters, and formatting); effective tool selection from multiple options; the ability to execute multiple tools in parallel; and efficient tool reuse for batch operations. Robust error handling is also a key area of evaluation, assessed through irrelevance detection and the ability to handle situations where tools are missing. Furthermore, LLMs are tested on context management in long interactions, multi-turn conversational abilities, handling complex, multi-step interactions, and the ability to manage missing parameters to ensure complete information is gathered. These evaluations cover a wide range of scenarios, from simple automation to complex workflows, thereby ensuring that LLMs can reliably and effectively integrate with external tools in real-world applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cePesSSW-sLv2BD1g_xtxQ.png" /><figcaption><a href="https://huggingface.co/spaces/galileo-ai/agent-leaderboard">Agent Leaderboard</a></figcaption></figure><p>The test results, as displayed on the leaderboard, are particularly interesting. They demonstrate that you don’t necessarily need extremely powerful, heavy reasoning models like OpenAI’s o1 or o3-mini. (The creators also opted out DeepSeek R1 due to its limited function-calling support.) The leaderboard results, in fact, reinforce the advantages of efficient, agentic models, such as Gemini Flash, which is the core argument of this article.</p><p>I was particularly pleased to discover this leaderboard, as I was unaware of its existence when I began writing this article. Its findings heavily support my initial arguments about the benefits of Flash Agents, making it a valuable resource for anyone exploring this rapidly evolving field.</p><h3>smolagents: smol meets flash</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5acTlkikP758TjJJOCusEw.png" /><figcaption><a href="https://huggingface.co/docs/smolagents/index">smolagents</a></figcaption></figure><p>When I started writing this, I initially envisioned sharing longer code snippets demonstrating agentic designs using Pydantic AI. However, after reviewing the previous sections, I realized I’ve already spent considerable time on the theoretical aspects. Leaving you without any code didn’t feel right, so here’s the “practical of practicals” (as one might say!).</p><p>A good approach is to start simple and iteratively add complexity, and getting the hang of agentic designs couldn’t be easier than with the smolagents library from Hugging Face. Combining the speed and efficiency of Gemini 2.0 Flash with the simplicity of smolagents feels like a winning combination. So, let’s get started!</p><p>First, prepare your environment by installing the necessary libraries:</p><pre>pip install --upgrade --user --quiet &quot;google-cloud-aiplatform&quot; &quot;openai&quot; &quot;smolagents&quot;</pre><p>For the Gemini series of models, you can leverage <strong>Vertex AI</strong>. While it’s also possible to use AI Studio, working with a production-ready cloud service like Vertex AI feels more suitable for agentic workflows, in my opinion. This way, your system won’t be constrained by usage limits or other api call restrictions. Make sure you have access to <strong>Vertex AI on Google Cloud</strong> and the necessary permissions for its services.</p><p>Speaking of permissions, ensure you have your Service Account keys for this project, obtainable from the Google Cloud Console.</p><pre>import os<br>import vertexai<br>import threading<br>import time<br><br>from IPython.display import display, Image<br>import google.auth<br>import google.auth.transport.requests<br>import openai<br>from smolagents import ChatMessage, CodeAgent, Model, DuckDuckGoSearchTool<br>from smolagents.agents import ActionStep<br>from smolagents.tools import Tool, tool<br><br>os.environ[&quot;GOOGLE_APPLICATION_CREDENTIALS&quot;] = &quot;your_service_account_key.json&quot;<br><br>PROJECT_ID = &quot;your_project_id&quot;<br>LOCATION = os.environ.get(&quot;GOOGLE_CLOUD_REGION&quot;, &quot;us-central1&quot;)<br><br>vertexai.init(project=PROJECT_ID, location=LOCATION)</pre><p>After initializing our credentials with Vertex AI services through Google Cloud, we need to address one final detail: creating a custom model class for compatibility.</p><pre>class VertexAIServerModel(Model):<br>    &quot;&quot;&quot;This model connects to a Vertex AI-compatible API server.&quot;&quot;&quot;<br><br>    def __init__(<br>        self, model_id: str, project_id: str, location: str, endpoint_id: str, **kwargs<br>    ):<br>        #  Try to import dependencies<br>        try:<br>            from google.auth import default<br>        except ModuleNotFoundError:<br>            raise ModuleNotFoundError(<br>                &quot;Please install &#39;openai, google-auth and requests&#39; extra to use VertexAIGeminiModel as described in the official documentation: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library&quot;<br>            ) from None<br><br>        # Initialize parent class with any additional keyword arguments<br>        super().__init__(**kwargs)<br>        self.model_id = model_id<br>        self.project_id = project_id<br>        self.location = location<br>        self.endpoint_id = endpoint_id<br>        self.kwargs = kwargs<br>        self._refresh_task = None<br><br>        # Initialize credentials and set up Google Cloud authentication with required permissions<br>        self.credentials, _ = default(<br>            scopes=[&quot;https://www.googleapis.com/auth/cloud-platform&quot;]<br>        )<br>        self._refresh_token()<br>        self._setup_client()<br>        self._start_refresh_loop()<br><br>    def __call__(<br>        self,<br>        messages: list[dict[str, str]],<br>        **kwargs,<br>    ) -&gt; ChatMessage:<br><br>        # Prepare the API call parameters<br>        completion_kwargs = self._prepare_completion_kwargs(<br>            messages=messages,<br>            model=self.model_id,<br>            **self.kwargs,<br>        )<br><br>        # Make the API call to Vertex AI<br>        response = self.client.chat.completions.create(**completion_kwargs)<br>        self.last_input_token_count = response.usage.prompt_tokens<br>        self.last_output_token_count = response.usage.completion_tokens<br><br>        # Convert API response to ChatMessage format<br>        message = ChatMessage.from_dict(<br>            response.choices[0].message.model_dump(<br>                include={&quot;role&quot;, &quot;content&quot;, &quot;tool_calls&quot;}<br>            )<br>        )<br>        return message<br><br>    def _refresh_token(self):<br>        &quot;&quot;&quot;Refresh the Google Cloud token&quot;&quot;&quot;<br>        try:<br>            self.credentials.refresh(google.auth.transport.requests.Request())<br>            self._setup_client()<br>        except Exception as e:<br>            print(f&quot;Token refresh failed: {e}&quot;)<br><br>    def _setup_client(self):<br>        &quot;&quot;&quot;Setup OpenAI client with current credentials&quot;&quot;&quot;<br>        self.client = openai.OpenAI(<br>            base_url=f&quot;https://{self.location}-aiplatform.googleapis.com/v1beta1/projects/{self.project_id}/locations/{self.location}/endpoints/{self.endpoint_id}&quot;,<br>            api_key=self.credentials.token,<br>        )<br><br>    def _start_refresh_loop(self):<br>        &quot;&quot;&quot;Start the token refresh loop&quot;&quot;&quot;<br><br>        def refresh_loop():<br>            while True:<br>                time.sleep(3600)<br>                self._refresh_token()<br><br>        self._refresh_thread = threading.Thread(target=refresh_loop, daemon=True)<br>        self._refresh_thread.start()</pre><p>The class above, <a href="https://github.com/GoogleCloudPlatform/generative-ai/blob/main/open-models/use-cases/vertex_ai_deepseek_smolagents.ipynb">taken from here</a>, is essential for ensuring compatibility between smolagents and Vertex AI. Think of this class as representing the Gemini text generation model, the core engine powering your agent.</p><pre>model = VertexAIServerModel(<br>    model_id=&quot;google/gemini-2.0-flash-001&quot;,<br>    endpoint_id=&quot;openapi&quot;,<br>    project_id=PROJECT_ID,<br>    location=LOCATION,<br>)</pre><p>We’re almost ready to go — exciting! Let’s test CodeAgent from the library without any additional tools:</p><pre>agent = CodeAgent(model=model, tools=[], add_base_tools=False)<br>response = agent.run(&quot;Hello! How are you?&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*eAapiO5Yp2Ga1s1i3TjusA.png" /></figure><p>It works! Although it took three steps for a simple response from an LLM, the magic happens within the CodeAgent loop. In each step, our agent executes its actions as code within an environment where it can validate its outputs using a code interpreter with persistent memory. For this simple task, structured JSON formatting would have been sufficient, but for more complex tasks, using code offers greater flexibility.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2hAFp6_BJtzJMZ7ESkHy_w.png" /><figcaption><a href="https://huggingface.co/docs/smolagents/v1.9.2/en/conceptual_guides/intro_agents">CodeAct</a></figcaption></figure><p>Now, let’s create our first callable tool. We’ll implement a character counter for a given word. Due to the token-based nature of LLMs, counting things as humans do can be challenging. A popular (though not practically robust) test involves counting the ‘r’s in “strawberry.” Let’s try that with the default reasoning Flash model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i1dqJvhIdSiq82nasIQZZQ.png" /></figure><p>As you can see, the model struggles. Now, let’s integrate our specialized tool:</p><pre>@tool<br>def count_characters(word: str, characters: str) -&gt; str:<br>    &quot;&quot;&quot;<br>    This tool counts the occurrences of specified characters in a given word.<br>    <br>    Args:<br>        word: The word in which to count characters.<br>        characters: A string containing the characters whose occurrences should be counted in the word.<br>    <br>    Returns:<br>        A message indicating the count of the specified characters in the word.<br>    &quot;&quot;&quot;<br>    count = 0<br>    for char in characters:<br>        count += word.count(char)<br>    return f&quot;There are {count} {characters}&#39;s in the word {word}.&quot;<br><br>agent = CodeAgent(model=model, tools=[count_characters], add_base_tools=False)<br>response = agent.run(&quot;how many r&#39;s in strawberry&quot;)</pre><p>And here’s the output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*XrgnKdx4NUTHn-JXcxLI7w.png" /></figure><p>That’s a considerable number of tokens for such a simple task, but at least we got the correct answer this time! As you can see, our agent decided which tool to use (the count_characters function), called that function in Python, and used its output as the final answer.</p><p>The interesting thing is, with CodeAgent, we might not have needed to write that tool at all. Since it reasons through code, perhaps we can support our agents in other ways, such as with search engines, and leave the rest to the Flash agent itself.</p><pre>agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=model)<br><br>response = agent.run(&quot;How many seconds would it take for a cheetah at full speed to run through boğaziçi köprüsü?&quot;)</pre><p>And voila:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*YDD7LCGH0UPTkDDdBHfagg.png" /></figure><p>Our agent managed to break down our question, call the search engine to retrieve the necessary variables, and provide the final answer. Okay, let’s increase the complexity slightly — but not <em>too</em> much.</p><p>We can implement our own search engines for specific use cases. Imagine a simplified version of OpenAI’s Deep Research. In our case, we’ll replace the web crawler with a simple Wikipedia browser tool to fetch summaries of relevant pages. Then, to make things a bit more entertaining, we’ll also add the FLUX.1-schnell Hugging Face Space as a callable tool to generate images.</p><pre>import requests<br>from urllib.parse import quote<br>from smolagents import Tool<br><br>class WikiBrowserTool(Tool):<br>    name = &quot;wiki_browser&quot;<br>    description = &quot;&quot;&quot;<br>    This tool fetches a Wikipedia page summary using Wikipedia&#39;s REST API.<br>    Provide the page title and optionally a language code to retrieve a formatted summary.<br>    &quot;&quot;&quot;<br>    inputs = {<br>        &quot;title&quot;: {<br>            &quot;type&quot;: &quot;string&quot;,<br>            &quot;description&quot;: &quot;The title of the Wikipedia page (e.g., &#39;Python (programming language)&#39;).&quot;<br>        },<br>        &quot;language&quot;: {<br>            &quot;type&quot;: &quot;string&quot;,<br>            &quot;nullable&quot;: True,<br>            &quot;description&quot;: &quot;The language code for Wikipedia (default is &#39;en&#39;).&quot;<br>        }<br>    }<br>    output_type = &quot;string&quot;<br><br>    def forward(self, title: str, language: str = &quot;en&quot;) -&gt; str:<br>        base_url = f&quot;https://{language}.wikipedia.org/api/rest_v1/page/summary/&quot;<br>        encoded_title = quote(title)<br>        url = base_url + encoded_title<br>        try:<br>            response = requests.get(url)<br>            response.raise_for_status()<br>            data = response.json()<br>            page_title = data.get(&quot;title&quot;, title)<br>            extract = data.get(&quot;extract&quot;, &quot;No summary available.&quot;)<br>            return f&quot;Title: {page_title}\n\n{extract}&quot;<br>        except Exception as e:<br>            return f&quot;Error fetching the page &#39;{title}&#39;: {e}&quot;<br><br>wiki_browser_tool = WikiBrowserTool()<br><br>image_generation_tool = Tool.from_space(<br>    &quot;black-forest-labs/FLUX.1-schnell&quot;,<br>    name=&quot;image_generator&quot;,<br>    description=&quot;Generate an image from a prompt&quot;<br>)<br><br>@tool<br>def image_display(image_path: str) -&gt; str:<br>    &quot;&quot;&quot;<br>    This tool displays a generated image from a given file path.<br>    It loads the image and renders it in the notebook using IPython.display.<br>    <br>    Args:<br>        image_path: The file path to the image that should be displayed.<br>    <br>    Returns:<br>        A confirmation message indicating that the image was displayed successfully.<br>    &quot;&quot;&quot;<br>    display(Image(filename=image_path))<br>    return &quot;Displayed generated image successfully.&quot;<br><br><br>agent = CodeAgent(<br>    tools=[wiki_browser_tool, image_generation_tool, image_display],<br>    model=model,<br>    add_base_tools=False,<br>    max_steps=10,<br>    planning_interval=3<br>)<br><br>response = agent.run(&quot;tell me about 2024 us elections, who is the current u.s president? how things have changed in 2025 since then. summarize it everything in a image and render it&quot;) in 2025 since then. summarize it everything in a image&quot;)</pre><p>Okay, now the results are <em>interesting</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/909/1*3YpGkybVg2nEKG1jstWWqw.png" /></figure><p>So, the agent decided when and where to use the provided tools. It then called the relevant Wikipedia pages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/911/1*nf3kMLRiCZ3AsOEYI-43KQ.png" /></figure><p>We can confirm these events occurred after the Gemini 2.0 Flash cutoff date (August 2024), so our Wiki search is working correctly. With this information, our agent managed to create a prompt for the Hugging Face Space and call the image_display tool to render the final generation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*uq_tJbrwqcZ0jGg5Gu_wRQ.png" /></figure><p>Oh well… I won’t comment, but that’s certainly an <em>interesting</em> generation. The quality can vary significantly depending on the image generation model you connect to your agent. It may be beneficial to experiment with different image generation models to achieve better results.</p><h3><strong>Conclusion: The Power of Efficient, Agentic Systems with Gemini Flash</strong></h3><p>Throughout this article, we’ve journeyed from the broad landscape of AI advancements to the specific, practical implementation of “Flash Agents” using Google’s Gemini Flash models and the smolagents library. The core argument has been that, while incredibly powerful generalist LLMs have their place, a strategic, multi-agent approach often provides superior efficiency, speed, and cost-effectiveness, especially for complex, iterative tasks.</p><p>We began by establishing the context: the rapid evolution of LLMs and the emergence of models specifically designed for agentic workflows. We clarified the essence of Agentic AI, emphasizing the crucial roles of reasoning, acting, and reflecting. An agent isn’t just a tool; it’s an autonomous entity capable of planning, choosing the right tools, executing tasks, and evaluating outcomes. We saw how Large Language Models amplify these capabilities, enabling agents to handle complex tasks, orchestrate other agents, and adapt to dynamic environments.</p><p>The distinction between <em>reasoning models</em> (focused on inference-time thinking) and <em>agentic reasoning</em> (focused on task delegation and orchestration) led to the crucial question: Why not always use the most powerful reasoning model? The answer, we explored, lies in resource optimization. Cost (both in terms of API calls and computational resources), speed (especially during development and experimentation), and the advantage of specialists over generalists all point towards the benefits of a multi-agent system.</p><p>The Agent Leaderboard on Hugging Face provided crucial empirical evidence supporting this argument. Its comprehensive evaluation, focusing on Tool Selection Quality (TSQ), demonstrated that highly capable models like Gemini Flash can achieve excellent performance in agentic scenarios, often rivaling or even surpassing much larger, more expensive models. This isn’t to say that larger models are obsolete; rather, it underscores the importance of choosing the <em>right</em> tool for the job.</p><p>Finally, we transitioned from theory to practice with the smolagents section. This hands-on exploration demonstrated the ease with which one can build and experiment with agentic systems, leveraging the speed and efficiency of Gemini Flash. We saw how even relatively simple agents, equipped with the right tools (like a character counter, a search engine, or an image generator), can tackle complex tasks that would be challenging for a single, generalist model. The examples, progressing from basic interactions to a multi-tool agent capable of fetching information and generating images, illustrated the power and flexibility of this approach.</p><p><strong>Key Takeaways:</strong></p><ul><li><strong>Agentic AI is about Orchestration:</strong> It’s not just about individual LLMs; it’s about building systems where models can plan, delegate, and collaborate.</li><li><strong>Efficiency Matters:</strong> Gemini Flash models offer a compelling balance of capability, speed, and cost-effectiveness, making them ideal for building and deploying agentic systems.</li><li><strong>Remember the Specialist Advantage:</strong> Smaller, focused models often provide the best performance for specific tasks within an agentic system.</li><li><strong>Benchmarking is Crucial:</strong> The Agent Leaderboard provides a valuable resource for evaluating the true capabilities of LLM agents.</li><li><strong>Practical Implementation is Accessible:</strong> Libraries like smolagents simplify the development of agentic systems, allowing for rapid prototyping and experimentation.</li><li><strong>Iterative approach is essential:</strong> Start with simpler tools and agents. As you gain familiarity increase the complexity. Don’t jump into building complex workflows.</li></ul><p>I’ve only scratched the surface of agent capabilities in this article, but I hope it provides a helpful starting point. These examples can be implemented in far more complex settings, with practical scenarios like data analysis, content creation, or automated workflow management. However, it’s crucial to note that, at their current level of development, I wouldn’t recommend using agents for customer-facing roles or critical decision-making processes, as their decision-making often involves multiple steps, adjustments, trial and errors. While a hotel booking agent might be a great idea in theory, complete trust in the agent’s final booking decision might be premature, at least for now. At the same time, repetitive or time-consuming tasks where the system can easily validate — such as coding, as in our example — are good starting points for agents.</p><p><em>Before finishing I’d like to thank google ML GDE and Google Cloud since credits are provided for this project as a part of the </em><strong><em>#VertexAISprint</em></strong><em>.</em></p><p>The combination of powerful, efficient models like Gemini Flash and accessible development frameworks like smolagents puts the power of agentic AI within reach. By embracing a strategic, multi-agent approach, developers can build systems that are not only intelligent but also practical, scalable, and cost-effective. The future of AI is not just about bigger models; it’s about smarter, more collaborative systems — and Flash Agents are poised to play a significant role in that future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e440748dc8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reasoning Models at Home]]></title>
            <link>https://ertugruldemir.medium.com/reasoning-models-at-home-760310732f6b?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/760310732f6b</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deepseek]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Sat, 08 Feb 2025 14:22:04 GMT</pubDate>
            <atom:updated>2025-02-10T05:34:29.192Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>How to Train Your Own Specialized Reasoning Model at Home</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*5EBaGphvbsLei58Z0TGQuA.jpeg" /><figcaption>Reasoning Process in a Finetuned Small LLM — Source: Author</figcaption></figure><p>Reasoning models are not entirely new, but the release of DeepSeek’s R1 model sparked significant excitement within the open-source community. After reading the <strong>DeepSeek-R1</strong> paper, I became particularly interested in their <strong>post-training</strong> technique, which reportedly leads to substantial performance improvements.</p><p>The traditional post-training method is <strong>Supervised Fine-Tuning (SFT)</strong>. In SFT, the model is provided with prompts and desired outputs to learn user preferences, structural alignment, safety protocols, and more. Essentially, SFT demonstrates what developers or users expect from the model, guiding it to align with these examples.</p><p>Since post-training has been mentioned, it’s worth briefly discussing <strong>pre-training</strong> before delving further into R1’s post-training approach, as the two are related. One of the initial steps in training Large Language Models (LLMs) is<strong> Causal Language Modeling</strong>. In this stage, the model learns to predict the next token in a sequence by attending to the preceding tokens. This process yields “<strong>Base</strong>” models, which are trained on vast amounts of public text data in a semi-supervised manner, without explicit labels, to predict subsequent tokens. Speaking of public text data, we are nearing its depletion (to some extent, disregarding other modalities in this context). This scarcity is one of the primary drivers for researchers to concentrate on the post-training phase of LLMs.</p><p>Returning to post-training, numerous techniques exist, such as <strong>SFT, PPO, and DPO</strong>, among others. While some methods perform better in specific domains, they often limit creative freedom for instruction alignment or become prohibitively expensive for extended runs. Conversely, base models lack these limitations but are unpredictable, difficult to instruct, and often produce outputs that are barely comprehensible to humans.</p><p>The authors of R1 discovered a method to achieve the best of both worlds: aligning raw models to instructions while preserving a degree of creative freedom. I won’t detail the entire process here, but fundamentally, they allow the base model to <strong>“reason”</strong> with considerable freedom, while setting simple objectives focused on accuracy and format. To achieve this, they employed <strong>Reinforcement Learning</strong>, implementing an optimization technique called <strong>“GRPO” (Group Relative Policy Optimization)</strong>. GRPO eliminates the need for a critic model — traditionally the same size as the policy model and used in methods like PPO/DPO<strong> </strong>— and instead calculates a baseline using aggregated group scores.</p><p>Applying this method directly to base models resulted in strong reasoning capabilities, leading to the first version of the R1 model: <strong>DeepSeek-R1-Zero</strong>. Although this model improved the reasoning capabilities of base models, its outputs were often difficult for humans to read or interpret. As mentioned, SFT methods help models adhere to human preferences and readability. However, we also discussed how SFT can stifle creative freedom and generalization, as demonstrated in the paper: “<strong>SFT Memorizes, RL Generalizes</strong>: A Comparative Study of Foundation Model Post-training.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hZq8vVwGfwdkm2Ik_3MmNA.jpeg" /><figcaption>From: <a href="https://arxiv.org/abs/2501.17161v1">https://arxiv.org/abs/2501.17161v1</a></figcaption></figure><p>Therefore, the authors utilized a small, high-quality dataset to “<strong>cold start</strong>” their models before the <strong>Reinforcement Learning</strong> step. This approach was instrumental in the creation of the renowned DeepSeek-R1 model. In the next chapter, we will explore the implementation of some of these techniques to achieve similar results.</p><h4>Force to Reason</h4><p>In their report, DeepSeek mentioned a new set of <strong>distilled models</strong>. These significantly smaller models were trained using a distillation technique with SFT, leveraging the outputs of DeepSeek-R1. Notably, these distilled models did not undergo any Reinforcement Learning steps, which is an interesting point and an area worth exploring for us. Therefore, instead of distilling models, <strong>what if we implement GRPO directly on the original models?</strong></p><p>We could proceed with either base models or instruct models, assuming the instruct models have already been “cold started.” Given that this is an experiment conducted on my local gaming rig equipped with an RTX 3090, I have chosen the Qwen2.5–0.5B-Instruct model to begin with. It’s a model with a very small number of parameters for an LLM, and while its limitations are uncertain, it’s worth experimenting with.</p><h4>Setting Up the Rewards</h4><p>In the original paper, the authors utilized a large <strong>Mixture of Experts (MoE)</strong> model with <strong>671B parameters</strong>. This substantial model size allowed them to provide the model with considerable latitude, fostering freedom, robustness, and creativity. However, in my case, working with a 0.5B parameter model (yes, you read that right!), I felt it necessary to implement <strong>stricter rules</strong> to guide the model effectively. The original DeepSeek-R1 model employed two primary rewards for reward modeling: <strong>Accuracy and Format</strong>. These two rewards were designed to assess the correctness of the final response and whether the formatting incorporated reasoning steps enclosed within &lt;think&gt; and &lt;/think&gt; tags.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TMLedvq5v-_OG7UbYyczqQ.png" /><figcaption>Train Rewards Chart — Source: Author</figcaption></figure><p>In my more constrained scenario, I incorporated a broader set of rules to reward or penalize the model:</p><ul><li>Is the answer correct?</li><li>Does it follow the suggested format?</li><li>Does it include reasoning tags?</li><li>If reasoning tags are present, what is length within the thinking process?</li><li>Does it include validation tags?</li><li>If validation tags are present, what is the length of self-critique?</li><li>Does it utilize an “Aha!” moment marker?</li></ul><p>I recognize that some of these rewards are subjective and may not generalize broadly. However, as mentioned, I believed that several of these were essential for a smaller model and to facilitate faster training. Prepare yourself for the real arbitrary reward points in the code! Furthermore, it’s important to consider the dataset we are employing, as several of these rules are tailored to its specific characteristics. The dataset is <strong>GSM8K</strong> <strong>(Grade School Math 8K)</strong>, which comprises linguistically diverse grade school math word problems. This dataset was specifically created to support question answering tasks on basic mathematical problems that necessitate multi-step reasoning. Consequently, some of these rules are task-specific to this dataset and <strong>may not generalize well</strong>. Nevertheless, allow me to explain the rationale behind some of these reward choices.</p><ul><li>For accuracy reward we’re going to extract final answer between <strong>&lt;answer&gt;&lt;/answer&gt;</strong> tags as predictions and check the accuracy of them with ground truth answers.</li><li>The formatting part is checking for <strong>tags for reasoning, criticizing and answer</strong> parts to follow general structure.</li></ul><p>These two are similar to original paper but rest is just up to your task and creativity.</p><ul><li>To encourage longer thinking processes I <strong>count the number of words</strong> between reason and validate tags and add that as a reward in a scale up to some extent. (To prevent model exploiting useless long answers)</li></ul><p>The original work mentioned <strong>“Aha!”</strong> moment happened by itself while reasoning. Although it’s possible for bigger models with longer training procedures, in my scenario I set a cheesy game rule to force this moment.</p><p>In a recent paper <strong>s1: Simple test-time scaling</strong>, authors inserted a “<strong>Wait</strong>” token at the point where models preparing to end their predicted sequence, to force them rethink. It sound similar to “Aha!” moment so I just <strong>took this idea and inserted into my reward function</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*d092iErAl53pWN88rUx50Q.jpeg" /><figcaption>Source: <a href="https://arxiv.org/abs/2501.19393">https://arxiv.org/abs/2501.19393</a></figcaption></figure><ul><li>Basically if there’s an “Aha!” moment, such as “<strong>Wait, but or rethink</strong>” and followed by minimum of characters I set huge bonus for reward to encourage this kind of thinking, and also set that into system prompt.</li></ul><h4>Spoiler Alert!</h4><p><strong><em>Here’s an example from the training set predictions:</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/1*cIeibT7JLSGxFWBvLiaqoA.jpeg" /><figcaption>An example from train set predictions. Source: Author</figcaption></figure><p>Well, looking at this output, it seems that the model is attempting to follow the rules I established. Even <strong>though it exploits some aspects</strong> (which is expected), it appears to be a promising reasoning model, complete with a self-initiated <strong>“Wait, let’s rethink” moment.</strong></p><h3>Code Time</h3><p>Okay, I think we’ve covered enough of the theoretical background. Now, let’s dive into the actual code implementation. We’ll begin by creating utility functions for several key components: setting up the system prompt, extracting answers from the dataset, extracting content between specified tags, and loading the <strong>GSM8K </strong>dataset itself.</p><pre>import re<br>import torch<br>from datasets import load_dataset, Dataset<br>from trl import GRPOConfig, GRPOTrainer<br><br>from nltk.tokenize import word_tokenize<br><br>SYSTEM_PROMPT = &quot;&quot;&quot;<br>Respond in the following format:<br>&lt;reasoning&gt;<br>[Break and reason your answer here]<br>&lt;/reasoning&gt;<br>&lt;validate&gt;<br>[Criticize your reason and think about it. If you see somethin start with &#39;wait&#39;]<br>&lt;/validate&gt;<br>&lt;answer&gt;<br>[Final integer answer justified by validate]<br>&lt;/answer&gt;<br>&quot;&quot;&quot;<br><br>def extract_hash_answer(text: str) -&gt; str | None:<br>    if &quot;####&quot; not in text:<br>        return None<br>    return text.split(&quot;####&quot;)[1].strip()<br><br>def extract_xml_answer(response, tag=&quot;answer&quot;):<br>    import re<br>    match = re.search(rf&#39;&lt;{tag}&gt;(.*?)&lt;/{tag}&gt;&#39;, response, re.DOTALL)<br>    if match:<br>        return match.group(1).strip()<br>    return &quot;&quot;<br><br>def get_gsm8k_questions(split = &quot;train&quot;) -&gt; Dataset:<br>    data = load_dataset(&#39;openai/gsm8k&#39;, &#39;main&#39;)[split]<br>    data = data.map(lambda x: {<br>        &#39;prompt&#39;: [<br>            {&#39;role&#39;: &#39;system&#39;, &#39;content&#39;: SYSTEM_PROMPT},<br>            {&#39;role&#39;: &#39;user&#39;, &#39;content&#39;: x[&#39;question&#39;]}<br>        ],<br>        &#39;answer&#39;: extract_hash_answer(x[&#39;answer&#39;])<br>    })<br>    return data<br><br>dataset = get_gsm8k_questions()</pre><p>Next, we’ll implement one of the most important pieces: the <strong>Reward Function</strong>. I created this based on the points I mentioned in the previous chapter, but feel free to tinker or write your own.</p><pre>def custom_reward_func(prompts, completions, answer, min_reasoning_length=10, **kwargs) -&gt; list[float]:<br>    responses = [completion[0][&#39;content&#39;] for completion in completions]<br>    q = prompts[0][-1][&#39;content&#39;]<br>    extracted_responses_answer = [extract_xml_answer(r, tag=&quot;answer&quot;) for r in responses]<br>    extracted_responses_reasoning = [extract_xml_answer(r, tag=&quot;reasoning&quot;) for r in responses]<br>    extracted_responses_validate = [extract_xml_answer(r, tag=&quot;validate&quot;) for r in responses]<br><br>    rewards = []<br>    for original_response, extracted_answer, extracted_reasoning, extracted_validate in zip(<br>        responses, extracted_responses_answer, extracted_responses_reasoning, extracted_responses_validate<br>    ):<br>        is_correct = (extracted_answer == answer[0])<br>        is_int = extracted_answer.isdigit()<br>        has_answer_tags = &quot;&lt;answer&gt;&quot; in original_response and &quot;&lt;/answer&gt;&quot; in original_response<br>        has_reasoning_tags = &quot;&lt;reasoning&gt;&quot; in original_response and &quot;&lt;/reasoning&gt;&quot; in original_response<br>        has_validate_tags = &quot;&lt;validate&gt;&quot; in original_response and &quot;&lt;/validate&gt;&quot; in original_response<br>        reasoning_length = len(word_tokenize(extracted_reasoning.lower()))<br>        validate_length = len(word_tokenize(extracted_validate.lower()))<br><br>        reward = 0.0<br>        reasoning_reward = 0.0<br><br>        if is_correct:<br>            reward += 5.0<br>        if is_int:<br>            reward += 0.5<br><br>        if has_validate_tags:<br>            reward *= 1.25<br>            if validate_length &gt;= 5:<br>                min_validate_length = 5<br>                max_validate_length = 256<br>                max_validate_bonus = 3.0<br>                if validate_length &gt;= min_validate_length:<br>                    if validate_length &gt;= max_validate_length:<br>                        validate_bonus = max_validate_bonus<br>                    else:<br>                        validate_bonus = ((validate_length - min_validate_length) / (max_validate_length - min_validate_length)) * max_validate_bonus<br>                else:<br>                    validate_bonus = 0.0<br>            else:<br>                validate_bonus = 0.0<br>        else:<br>            validate_bonus = 0.0<br><br>        if has_reasoning_tags:<br>            reward *= 1.25<br>            if reasoning_length &gt;= 5:<br>                min_scaling_length = 5<br>                max_scaling_length = 1024<br>                max_scaling_bonus = 10<br>                if reasoning_length &lt;= min_scaling_length:<br>                    reasoning_reward = 0.0<br>                elif reasoning_length &gt;= max_scaling_length:<br>                    reasoning_reward = 5.0<br>                else:<br>                    reasoning_reward = ((reasoning_length - min_scaling_length) / (max_scaling_length - min_scaling_length)) * max_scaling_bonus<br>            else:<br>                reasoning_reward = 0.0<br>        else:<br>            reasoning_reward = 0.0<br><br>        total_reward = reward + reasoning_reward + validate_bonus<br><br>        if has_validate_tags:<br>            validate_lower = extracted_validate.lower()<br>            if re.search(r&quot;(wait|but|rethink)(?=.{20,})&quot;, validate_lower, re.DOTALL):<br>                total_reward *= 10.0<br><br>        rewards.append(total_reward)<br><br>    return rewards</pre><p>Okay, we’re done with utility functions and the reward function. Now, let’s get into the training script. For efficient and faster training with limited compute power, we’re going to use some libraries and techniques such as <strong>Unsloth, trl, vLLM, and LoRA</strong>, etc.</p><p>I am not going into deep details about these in this article but will mention their role in this pipeline.</p><p>Basically Unsloth allows optimized models to train with a smaller memory footprint and faster iterations.</p><pre>from unsloth import FastLanguageModel, PatchFastRL<br>PatchFastRL(&quot;GRPO&quot;, FastLanguageModel)<br><br><br>max_seq_length = 1024<br>lora_rank = 32<br><br>model, tokenizer = FastLanguageModel.from_pretrained(<br>    model_name = &quot;unsloth/Qwen2.5-0.5B-Instruct-unsloth-bnb-4bit&quot;,<br>    max_seq_length = max_seq_length,<br>    load_in_4bit = False,<br>    fast_inference = True, # Enable vLLM fast inference<br>    max_lora_rank = lora_rank,<br>    gpu_memory_utilization = 0.8,<br>)<br><br>model = FastLanguageModel.get_peft_model(<br>    model,<br>    r = lora_rank,<br>    target_modules = [&quot;q_proj&quot;, &quot;k_proj&quot;, &quot;v_proj&quot;, &quot;o_proj&quot;,<br>                      &quot;gate_proj&quot;, &quot;up_proj&quot;, &quot;down_proj&quot;],<br>    lora_alpha = lora_rank*2,<br>    use_gradient_checkpointing = &quot;unsloth&quot;,<br>    random_state = 1907,<br>)</pre><p>By applying <strong>LoRA we are training a small subset of model parameters</strong>. Then we set<strong> trl</strong> training arguments and the trainer itself.</p><pre>output_dir=&quot;outputs/Qwen-.5B-GRPO&quot;<br><br>training_args = GRPOConfig(<br>    output_dir=output_dir,<br>    run_name=run_name,<br>    learning_rate=3e-5,<br>    adam_beta1 = 0.9,<br>    adam_beta2 = 0.99,<br>    weight_decay = 0.1,<br>    warmup_steps=5,<br>    lr_scheduler_type=&#39;cosine&#39;,<br>    logging_steps=1,<br>    bf16=True,<br>    optim=&quot;paged_adamw_8bit&quot;,<br>    per_device_train_batch_size=1,<br>    gradient_accumulation_steps=32,<br>    num_generations=2,<br>    max_prompt_length=256,<br>    max_completion_length=512,<br>    num_train_epochs=1,<br>    save_steps=100,<br>    max_grad_norm=1.0,<br>    report_to=&quot;none&quot;,<br>    log_on_each_node=False,<br>    use_vllm=True,<br>)<br><br>trainer = GRPOTrainer(<br>    model=model,<br>    processing_class=tokenizer,<br>    reward_funcs=[custom_reward_func],<br>    args=training_args,<br>    train_dataset=dataset,<br>)<br>trainer.train()</pre><p>And let the magic happen! These hyperparameters are probably far from optimal (the same goes for other components too), but I believe the open-source community will find better approaches in no time. (My current setup is built upon <a href="https://gist.github.com/willccbb/4676755236bb08cab5f4e54a0475d6fb">GRPO Llama-1B</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*fMeysbXSUHGZ7Kw1X4CrJg.png" /><figcaption>Completion Length Chart — Source: Author</figcaption></figure><p>As you can see above, the model consistently increased its output by lengthening its reasoning and validation steps. At some point, it stopped increasing though, and there are a couple of reasons for that:</p><ul><li><strong>Lack of max_completion_length limit:</strong> As the reasoning steps get longer, the model starts to cut out the final answer, therefore not getting the important accuracy reward. (The limit I set in trainer)</li><li><strong>Lack of model capability:</strong> This might sound like an excuse, but for a 0.5B parameter model, it really matters.</li><li><strong>Reward function upper limits</strong> for reasoning lengths. (Remember we set limit to max gain from length…)</li></ul><p><strong>Evaluation Time!</strong></p><p>If you thought I was going to draw conclusions without evaluating the results on a test set, you’re wrong, haha! Luckily, GSM8K includes a test split, allowing us to evaluate our results and make comparisons. First, let’s use the same system prompt on the original instruct model to establish baseline results. I am setting sampling to False to ensure deterministic results for both sets of predictions.</p><blockquote><strong>Accuracy Score of Original Model: 0.0887035633055345</strong></blockquote><p>Well… That’s not great… However, it’s important to consider that the model is not explicitly trained to provide only integer answers, other than what’s implied by the system prompt. Therefore, <strong>sometimes the correct answer might be present within the prediction, even if not perfectly formatted</strong>. Still, we can observe that the model attempts to follow instructions, but doesn’t fully grasp their meaning or simply repeats the one-shot example in the prompt…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dHut9FFumHSMZfisWlXFjw.png" /><figcaption>Original Model Predictions</figcaption></figure><p><strong>Now, let’s test the GRPO-tuned model using the same settings:</strong></p><blockquote><strong>Accuracy Score of Tuned Model: 0.6527672479150872</strong></blockquote><p>Wow! That’s better than I expected for such a small model. I even had to double-check for potential data leakage. Even if there were no leaks, we might have fine-tuned this model to overfit this specific type of task. So, instead of a generalized model, we essentially have a <strong>specialized </strong>GSM8K calculator model. But still, it’s a significant gain and opens the door for many specialized model training endeavors (or for generalist big models), which is another topic I am very excited about…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fLw0_ENl8vl9CY7tNTd6Fg.png" /><figcaption>Finetuned Model Predictions</figcaption></figure><h3>Conclusion: Reasoning at Home — Just the Beginning!</h3><p>So, there you have it! We took a tiny 0.5B parameter model, threw some GRPO at it in our home lab (or, you know, gaming rig!), and managed to teach it some pretty decent reasoning skills, at least for math problems. Who would’ve thought, right? While our little Qwen-GRPO isn’t going to be replacing DeepSeek-R1 anytime soon, the jump in accuracy we saw is genuinely exciting. It proves, even with limited resources, we can start to play with these advanced techniques and push the boundaries of what smaller models can do.</p><p>This whole experiment really underscores a key point: <strong>reasoning models aren’t just for massive labs with endless compute</strong>. You can tinker with this stuff at home, learn a ton, and even get surprisingly good results. Sure, there’s still a long way to go — generalization, more robust rewards, maybe even figuring out how to make that “Aha!” moment less cheesy — but this is just the beginning. The open-source community is buzzing with ideas, and I can’t wait to see what amazing reasoning models we can all build together, brick by brick, or maybe, line of code by line of code! The future of reasoning at home? It’s looking surprisingly bright, and I, for one, am strapped in and ready for the ride!</p><h4>Resources:</h4><ul><li><a href="https://arxiv.org/abs/2501.12948">DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning</a></li><li><a href="https://arxiv.org/abs/2402.03300">DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models</a></li><li><a href="https://arxiv.org/abs/2501.19393">s1: Simple test-time scaling</a></li><li><a href="https://arxiv.org/abs/2501.17161v1">SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training</a></li><li><a href="https://huggingface.co/docs/trl/main/en/grpo_trainer">GRPO Trainer</a></li><li><a href="https://unsloth.ai/blog/r1-reasoning">Train your own R1 reasoning model with Unsloth</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=760310732f6b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fact Check with DataGemma]]></title>
            <link>https://ertugruldemir.medium.com/fact-check-with-datagemma-b83da70cdb96?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/b83da70cdb96</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Fri, 13 Sep 2024 11:03:48 GMT</pubDate>
            <atom:updated>2024-09-13T11:03:48.357Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to get numbers out of a LLM?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*SeHadRYHsbwSWnSyGzLWMQ.png" /></figure><p>Almost every person who has interacted with large language models (LLMs) has encountered the term “hallucinations” or experienced them without knowing the term. Hallucinations occur frequently, even with the most advanced LLMs today, due to the probabilistic nature of generative models. These models are especially prone to generating factually incorrect information when responding to queries involving numerical or statistical data.</p><p>If a request falls outside the scope of an LLM’s pretraining or fine-tuning, the model may produce factually incorrect, nonsensical, unrelated answers or refuses to answer at all. Current solutions to this problem involve significant human intervention, such as providing context (using techniques like Retrieval-Augmented Generation) or restricting models through prompting techniques.</p><p><a href="https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems">Gödel’s First Incompleteness Theorem</a> offers an interesting parallel to this challenge. The theorem states that in any consistent mathematical system powerful enough to describe basic arithmetic, there will always be true statements that cannot be proven within that system. In terms of LLMs, this suggests that no system can be both complete (able to prove every truth) and consistent (without contradictions) if it includes basic arithmetic. Gödel proved that no matter how well-designed the rules of a system are, there will always be some statements about numbers that are true but cannot be proven using the available rules.</p><p>Consequently, one cannot be certain of the factual accuracy of information from an LLM (which can be viewed as a set of rules in the context of Gödel’s theorem). However, we can minimize the effects of this phenomenon by increasing the available information, effectively defining a larger system that encompasses the original one.</p><p>To express this more technically in terms of Gödel’s theorem, we can define a larger system F’ that contains the whole of the original system F. In mathematical logic, F might represent our initial formal system (analogous to an LLM’s initial training), while F’ represents an expanded system with additional axioms or rules (analogous to augmenting an LLM with additional context or data). By expanding to F’, we can potentially prove statements that were unprovable in F alone. However, it’s crucial to note that according to Gödel’s theorem, even in F’ there will still be true but unprovable statements. This parallels the idea that while we can improve an LLM’s performance by providing more information.</p><p>So how do we expand F’? Do we have a vast knowledge base to use in our systems? <strong>Fortunately, we do:</strong></p><h3>Data Commons</h3><p>Data Commons, a Google-led open-source initiative, aims to centralize and democratize access to global public datasets. With over 250 billion data points from hundreds of sources, it covers a wide range of statistical information from various public entities worldwide. The project’s key innovations include normalizing diverse datasets into a unified Knowledge Graph using Schema.org, and implementing a natural language interface for user queries. This interface allows users to explore the vast database using everyday language, making the wealth of public data more accessible and easier to utilize.</p><p>Think of Data Commons as a constantly expanding repository of reliable, public information on a wide range of topics, from health and economics to demographics and the environment. Users can interact with this wealth of data using everyday language. This makes Data Commons an accessible and powerful tool for exploring and utilizing extensive public data resources, opening up new possibilities for research, analysis, and decision-making across various fields.</p><h3>Leveraging Data Commons with LLMs: Introducing DataGemma</h3><p>To leverage this pool of factual information using LLMs, we need a specific model to navigate this vast jungle of data. This is where <strong>DataGemma</strong> comes in.<strong> DataGemma </strong>is designed to bridge the gap between the extensive factual information in Data Commons and the powerful language processing capabilities of LLMs. By integrating DataGemma with LLMs, we can potentially create a system that combines the vast knowledge base of Data Commons with the natural language understanding of LLMs, resulting in more accurate and reliable responses to queries involving numerical and statistical data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*KzRZs_hGZNV5AAD7Y-10Xg.png" /><figcaption>Source: <a href="https://docs.datacommons.org/papers/DataGemma-FullPaper.pdf">Knowing When to Ask — Bridging Large Language<br>Models and Data</a></figcaption></figure><p>So how does DataGemma leverage the Data Commons? It employs well-established techniques: Retrieval-Augmented Generation (RAG) and Retrieval-Infused Generation (RIG). You might be thinking, “There’s nothing new here; these are already widely used!” While that’s true, DataGemma is specifically fine-tuned for three crucial tasks:</p><ul><li>Determining when to use external sources</li><li>Identifying which external sources should be used</li><li>Constructing appropriate queries to fetch the required data</li></ul><p>This specialized fine-tuning of Gemma models makes it particularly effective for working with Data Commons and LLMs.</p><h3><strong>Practice Time</strong></h3><p>Great! We’ve covered the theoretical aspects, and now it’s time to dive into the practical application. If you’re interested in learning more about the underlying concepts, I’ll be adding references to the original paper at the end of this article. Let’s get started with the actual use case! The best part? You can all use this for free, as it’s completely open source!</p><p>Before we jump into the code, let’s quickly go over what you’ll need to get started. You’ll want to have a machine with at least 16 GB of VRAM to run this smoothly. As for setup, make sure you’ve got a Python environment set up with torch, transformers, and <a href="https://github.com/datacommonsorg/llm-tools">data_gemma</a> installed. Don’t forget to set up your token for downloading from <a href="https://huggingface.co/models">Hugging Face</a>, and you’ll also need <a href="https://apikeys.datacommons.org/">a Data Commons API key</a>. Once you’ve got all that sorted out and your environment is good to go with the right packages and API access, you’ll be all set to dive into the code. It might sound like a bit to handle, but trust me, it’s pretty straightforward once you get into it. Ready to get coding?</p><h4>DataGemma RIG</h4><p>Along with these essential imports, we’ll also be including a custom helper function. This function is designed to stylize the output from our LLM. It’s a neat little trick that will make our results more visually appealing and easier to read.</p><pre>#!pip install -q git+https://github.com/datacommonsorg/llm-tools<br>#!pip install -q bitsandbytes accelerate<br><br>from IPython.display import Markdown<br>import textwrap<br>from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig<br>import torch<br>import data_gemma as dg<br>from IPython.display import Markdown<br>import textwrap<br><br>HF_TOKEN = &quot;YOUR_HF_TOKEN_HERE&quot;<br>DC_API_KEY = &quot;YOUR_DATACOMMONS_KEY_HERE&quot;<br>dc = dg.DataCommons(api_key=DC_API_KEY)<br><br>def display_chat(prompt, text):<br>    formatted_prompt = &quot;&lt;font size=&#39;+1&#39; color=&#39;brown&#39;&gt;🙋‍♂️&lt;blockquote&gt;&quot; + prompt + &quot;&lt;/blockquote&gt;&lt;/font&gt;&quot;<br>    text = text.replace(&#39;•&#39;, &#39;  *&#39;)<br>    text = textwrap.indent(text, &#39;&gt; &#39;, predicate=lambda _: True)<br>    formatted_text = &quot;&lt;font size=&#39;+1&#39; color=&#39;teal&#39;&gt;🤖\n\n&quot; + text + &quot;\n&lt;/font&gt;&quot;<br>    return Markdown(formatted_prompt + formatted_text)</pre><p>Next, we’ll set up our quantization configuration using BitsAndBytes. Quantization is a technique that reduces the precision of the model’s weights, allowing us to run larger models with limited computational resources. Since we’re working with limited VRAM, we’ll use 4-bit quantization, which significantly reduces memory usage while maintaining reasonable model performance.</p><pre>nf4_config = BitsAndBytesConfig(<br>   load_in_4bit=True,<br>   bnb_4bit_quant_type=&#39;nf4&#39;,<br>   bnb_4bit_compute_dtype=torch.bfloat16,<br>)<br><br>model_id = &#39;google/datagemma-rig-27b-it&#39;<br>tokenizer = AutoTokenizer.from_pretrained(model_id, token=HF_TOKEN)<br>model = AutoModelForCausalLM.from_pretrained(<br>    model_id,<br>    device_map=&#39;auto&#39;,<br>    quantization_config=nf4_config,<br>    torch_dtype=torch.bfloat16,<br>    token=HF_TOKEN<br>)</pre><p>After configuring the quantization, we’ll load the <a href="https://huggingface.co/google/datagemma-rig-27b-it">datagemma-rig</a> model specifically for the RIG (Retrieval-Infused Generation) task. RIG is a technique that combines information retrieval with text generation, allowing the model to access and incorporate external knowledge when generating responses.</p><p>It’s worth noting that if you plan to feed your outputs into models with longer context windows, such as Gemini, you might consider using <a href="https://huggingface.co/google/datagemma-rag-27b-it">datagemma-rag</a> model instead. RAG is similar to RIG but is particularly well-suited for models that can handle more extensive context.</p><p>Next we set our query, datagemma wrapper for our model and RIGFlow for final answer.</p><pre>input_text = &quot;YOUR_QUERY_GOES_HERE&quot;<br><br>datagemma_model_wrapper = dg.HFBasic(model, tokenizer)<br><br>ans = dg.RIGFlow(llm=datagemma_model_wrapper, data_fetcher=dc, verbose=False).query(query=input_text)<br>Markdown(textwrap.indent(ans.answer(), &#39;&gt; &#39;, predicate=lambda _: True))<br><br>display_chat(input_text, ans.answer())</pre><p>For test purposes I asked about Turkey’s last 10 year gdp/c statistics. Which is a good example to test fact capabilities of DataGemma model. And here’s the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0mkNBK6jzesqHbPJpRBosg.png" /><figcaption>DataGemma output.</figcaption></figure><p>As you can see it gave pretty detailed and factual answers even with footnotes! Model used multiple sources and queried their answers before generating the final output.</p><p>I didn’t want to copy paste whole answer as a text but you can test it yourself with the given code examples. Now let’s compare it to same model (gemma-2–27b) without the extra capabilities of DataGemma.</p><h4>Standard Gemma:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ARiyuOlP1g8Y592ozIOiGg.png" /><figcaption>Standard Gemma 2 27b Result</figcaption></figure><p>Well… The standard Gemma model refuses to give specific answer. Which is still better than hallucinating but at the same time it’s not that useful.</p><p>In contrary to standard Gemma, the DataGemma model used multiple related queries using it’s capabilities and great data sources it has from Data Commons. Here you can see some of the queries it used:</p><pre>... calling HF Pipeline API &quot;how is the turkish(türkiye) gdp per capita from 20...&quot;<br>... calling DC with &quot;what was the gdp per capita of turkey in 2014?&quot;<br>... calling DC with &quot;what was the gdp per capita of turkey in 2023?&quot;<br>... calling DC with &quot;what is the gdp per capita of turkey in 2024?&quot;<br>... calling DC with &quot;how many jobs were created in turkey between 2014 and 2024?&quot;<br>... calling DC with &quot;what is the poverty rate in turkey?&quot;</pre><p>As you can see it breaks down the task with smaller queries and catches the related data for final answer.</p><p><strong>Eval Results:</strong></p><p>The paper claims:</p><ul><li>DataGemma’s Retrieval Information Generation (RIG) system improved factual accuracy from 5–17% to 58%</li><li>Additionally, DataGemma’s Retrieval-Augmented Generation (RAG) system achieved 99% accuracy for statistical claims when citing directly from the data table.</li></ul><h3>Conclusions</h3><p>The challenge of hallucinations in LLMs has been a persistent issue in the field of artificial intelligence. This article has introduced DataGemma as a promising solution to mitigate this problem, particularly when dealing with numerical and statistical data. By leveraging the vast repository of public information available through Data Commons, DataGemma demonstrates a significant improvement in the factual accuracy of LLM responses.</p><p>The key advantages of DataGemma lie in its ability to:</p><ol><li>Determine when to use external sources</li><li>Identify which external sources to use</li><li>Construct appropriate queries to fetch the required data</li></ol><p>These capabilities, combined with the extensive dataset provided by Data Commons, allow DataGemma to produce more reliable and factually accurate responses compared to standard LLMs.</p><p>In our example you can see the difference easily. While the standard model refused to provide specific information, DataGemma was able to break down the query into smaller, manageable parts and retrieve relevant data to construct a comprehensive and accurate response.</p><p>This advancement has promising implications for the use of LLMs in various fields, particularly those requiring precise numerical or statistical information.</p><p>Future developments may focus on expanding the range of data sources, improving the efficiency of data retrieval and integration, and fine-tuning the models for even more specialized applications.</p><p>As we continue to push the boundaries of AI capabilities, solutions like DataGemma represent a important step towards creating more trustworthy and dependable artificial intelligence systems.</p><h3>Resources:</h3><ul><li>DataGemma Blog:<a href="https://blog.google/technology/ai/google-datagemma-ai-llm/">https://blog.google/technology/ai/google-datagemma-ai-llm/</a></li><li>DataGemma Paper: <a href="https://docs.datacommons.org/papers/DataGemma-FullPaper.pdf">https://docs.datacommons.org/papers/DataGemma-FullPaper.pdf</a></li><li>Client Library: <a href="https://github.com/datacommonsorg/llm-tools">https://github.com/datacommonsorg/llm-tools</a></li><li>RIG Model: <a href="https://huggingface.co/google/datagemma-rig-27b-it">https://huggingface.co/google/datagemma-rig-27b-it</a></li><li>RAG Model: <a href="https://huggingface.co/google/datagemma-rag-27b-it">https://huggingface.co/google/datagemma-rag-27b-it</a></li><li>RIG Colab Notebook: <a href="https://colab.research.google.com/github/datacommonsorg/llm-tools/blob/master/notebooks/datagemma_rig.ipynb#scrollTo=hB-DKl0BxlN7">https://colab.research.google.com/github/datacommonsorg/llm-tools/blob/master/notebooks/datagemma_rig.ipynb#scrollTo=hB-DKl0BxlN7</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b83da70cdb96" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Analyzing Financial Reports with Gemini 1.5]]></title>
            <link>https://ertugruldemir.medium.com/analyzing-financial-reports-with-gemini-1-5-8d8695578a79?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d8695578a79</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Thu, 22 Feb 2024 10:20:01 GMT</pubDate>
            <atom:updated>2024-02-22T10:20:01.714Z</atom:updated>
            <content:encoded><![CDATA[<p>A potential RAG alternative?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bquPlLApkwn1LAaZbl5MOA.jpeg" /><figcaption>Image generated by Author</figcaption></figure><p>The Gemini 1.5 Pro represents the latest advancement in the Gemini series of LLMs (Large Language Models), developed by Google DeepMind. This model stands out for its exceptional computational efficiency and its multimodal mixture-of-experts framework. With the ability to recall and reason through fine-grained information from a context comprising <strong>millions of tokens</strong>, Gemini 1.5 Pro excels in processing and comprehending extensive documents, video content, and audio recordings, as well as their various combinations.</p><p>In this article, I will demonstrate practical applications of Gemini 1.5’s new features and explain what makes it uniquely effective in multimodal settings with extended context.</p><h4>Limitations of Current Large Language Models</h4><p>State-of-the-art LLMs currently face challenges with context length; as input grows, it becomes increasingly difficult for these models to retain all necessary information before generating an answer. This limitation is primarily due to the nature of the attention mechanism within transformer models, but for now, we will not delve into its technical intricacies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*dbr4-6o5alo_2TFesgDcTQ.png" /><figcaption>Current SOTA Contex Lengths — Source: Google Blog</figcaption></figure><p>What sets Gemini 1.5 apart is its ability to perform near-perfect retrieval of information<strong> up to at least 10 million tokens</strong>. Remarkably, it maintains levels of performance on par with its predecessor, Gemini 1.0 Ultra, even as it handles larger context windows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*ysuCh4XJIfHtM2oDDDKxkA.png" /><figcaption>Gemini 1.5 Pro compared to Gemini 1.0 — Source: Gemini 1.5 Report</figcaption></figure><h4>The Needle in the Haystack</h4><p>In many practical scenarios, we do not require the entire context; instead, we need to extract specific points of interest from the given context. With the limited context lengths of most models, the preferred solution has been to use Retrieval Augmented Generation (RAG) techniques. RAG enables developers to divide and manage their inputs according to context limitations and identify the most relevant sections through text similarity searches. Locating the most relevant segment within a vast array of information is a formidable challenge, akin to the age-old proverbial search for a needle in a haystack.</p><p>To address this challenge, the <a href="https://github.com/gkamradt/LLMTest_NeedleInAHaystack">NeedleInAHaystack</a> test was developed. It involves inserting random facts or statements at various depths within a long context and then measuring the retrieval performance. This test helps assess the ability of a model to accurately identify and extract pertinent information, even when it is embedded within a larger and more complex context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hGm6i9e-4uRalzO59vN5Wg.jpeg" /><figcaption>GPT-4 and Claude-2.1 Haystack Tests-Source:Project Page</figcaption></figure><p>The performance of models such as GPT-4 and Claude-2.1 tends to diminish as the context length grows. In contrast, Google reports that Gemini 1.5 Pro achieves near-perfect recall of the “needle” <strong>(&gt;99.7%) with up to 1 million tokens</strong>, which is a notable achievement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*DIAHtjtHnTvzx6bUV2O7QA.png" /><figcaption>Haystack Tests for Gemini 1.5 — Source: Gemini 1.5 Report</figcaption></figure><p>Moreover, the report indicates that Gemini 1.5 Pro maintains its recall performance across different modalities, even with increased token lengths. This consistency is an important factor when dealing with multimodal long context tasks.</p><p>While there are many technical aspects to explore, such as the mixture of experts and faster training speeds, let us now focus on a practical use case for Gemini 1.5 Pro.</p><h3>A Practical Use Case for RAG and Gemini 1.5 Pro</h3><p>Imagine you are an amateur investor excited about the latest advancements in the AI sector. As a forward-thinking investor, you have purchased stocks from a high-end graphics processing unit (GPU) manufacturer. At the beginning of 2024, you want to evaluate some of your investments, starting with their yearly financial reports.</p><p>After struggling for a few minutes to read the financial reports and tables, you realize that you lack both the time and financial expertise to analyze these lengthy documents filled with complex tables and jargon. You decide to use a large language model (LLM) to help you, but you soon discover that as the reports grow longer, the model begins to miss important information and sometimes hallucinates.</p><p>To solve this issue, you opt for a RAG approach. By splitting the documents into readable chunks and retrieving the most relevant pieces, you can feed your LLM with the necessary information to generate accurate answers. However, you may find that retrievers sometimes fail to provide relevant content, especially with documents containing custom table formats.</p><p>This is where Gemini 1.5 comes in. With its impressive 1 million context length and support for diverse modalities, including PDFs, it seems tailor-made for your task.</p><p>As an added challenge, consider that NVIDIA’s latest fiscal report,<strong> released on February 21st, 2024</strong>, falls outside the training data of current LLM models. Moreover, you plan to compare this almost 100-page document with its ~170-page predecessor from the previous year for in-depth analysis. Fortunately, Gemini 1.5 makes the process surprisingly simple — <strong>just drag and drop</strong> these files into your AI Studio chat, and it automatically handles the parsing and provides token counts for each document.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O6P4XR0GuZOhQZm44_QJDA.png" /><figcaption><em>From Google AI Studio</em></figcaption></figure><p>As you can see, both documents combined contain<strong> more than 220,000 tokens</strong>, surpassing the context limits of most LLM models. Now that you have uploaded both documents, you ask the most important question on your mind: “How is the company doing compared to the previous year, and how does that affect shareholder incomes?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*ljJdGluCDqL6Eqotk3ZYkw.png" /><figcaption><em>From Google AI Studio</em></figcaption></figure><p>And just like that, Gemini 1.5 provides a clear and concise analysis, breaking down the information for you. The analysis reveals that the company is doing well and even offers insights into the reasons behind its success. Although you may not be an expert in financial analysis, you can still understand that the company’s performance is strong based on the information provided.</p><p>Feeling more confident in Gemini 1.5’s capabilities, you decide to ask another question: “What are the key factors that contributed to the company’s growth compared to the previous year?”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/863/1*Y8OpXtZP8l_BkZQu8OlrNQ.png" /><figcaption><em>From Google AI Studio</em></figcaption></figure><p>Impressive, isn’t it? It seems artificial intelligence plays a significant role — quite the coincidence!</p><p>Even better, the extended context length allows for further exploration through additional questions, adding reports from the same company or others, or incorporating financial news sources, videos, and graphics to enrich your analysis. Remarkably, even with these lengthy reports, we’ve <strong>only utilized about a quarter of Gemini 1.5’s context capacity</strong>. This opens the door for extended and in-depth conversations, enabling you to swiftly grasp complex topics.</p><h3><strong>Conclusion</strong></h3><p>Gemini 1.5, with its expanded context length and multi-modality capabilities, effortlessly fits into your daily tasks, eliminating the need for complex RAG pipelines or dealing with “hallucinations.”</p><p>However, this does not diminish the value of RAGs. They continue to be helpful in situations where feeding the entire context to the LLM involves unknown costs or when information loss through chunking is minimal. Additionally, there are really long contexts that may still require chunking. In such cases, Gemini 1.5’s longer context length allows for feeding larger chunks, which is another advantage for more complex RAG systems.</p><p>The immense context length of Gemini 1.5 is truly promising, and I am excited to see how longer context models with MoE’s develop in the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d8695578a79" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Arrival of SDXL 1.0]]></title>
            <link>https://medium.com/data-science/the-arrival-of-sdxl-1-0-4e739d5cc6c7?source=rss-c2ba54d15a24------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e739d5cc6c7</guid>
            <category><![CDATA[diffusion-models]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[thoughts-and-theory]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ertuğrul Demir]]></dc:creator>
            <pubDate>Wed, 02 Aug 2023 18:18:21 GMT</pubDate>
            <atom:updated>2023-08-03T10:53:55.511Z</atom:updated>
            <content:encoded><![CDATA[<h4>Introducing SDXL 1.0: Understanding the Diffusion Models</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mD9zQJdsh78tugkA_ICOkw.png" /><figcaption>A cute little robot learning how to paint — Created by Using SDXL 1.0</figcaption></figure><p>In the rapidly evolving world of machine learning, where new models and technologies flood our feeds almost daily, staying updated and making informed choices becomes a daunting task. Today, we direct our focus towards SDXL 1.0, a text to image generative model which has undoubtedly garnered considerable interest within the field.</p><p>SDXL 1.0, short for “Stable Diffusion XL,” is touted as a latent text-to-image diffusion model that supposedly surpasses its predecessors with a range of promising enhancements. In the upcoming chapters, we will closely examine these claims and take a closer look at the new enhancements.</p><p>Most notably, SDXL is an open-source model that addresses a major concern in the domain of generative models. While black-box models have gained recognition as state-of-the-art, their architecture’s opacity hinders a comprehensive assessment and validation of their performance, limiting broader community involvement.</p><p>In this article, we embark on a detailed exploration of this promising model, inspecting its capabilities, building blocks, and drawing insightful comparisons with previous stable diffusion models. My aim is to provide a clear understanding without delving too deeply into technical complexities, making this an engaging and accessible read for all. Let’s get started!</p><h3>Understanding Stable Diffusion: Unraveling the Magic of Text-to-Image Generation</h3><blockquote>If you feel confident about how Stable Diffusion works, or don’t get involved in technical parts feel free to skip this chapter.</blockquote><p>Stable Diffusion, the groundbreaking deep learning text-to-image model, sent shockwaves through the AI world upon its release in 2022, harnessing the power of cutting-edge diffusion techniques.</p><p>This significant development represents a notable advancement in AI image generation, potentially expanding access to high-performance models for a broader audience. The intriguing capability to transform plain text descriptions into intricate visual outputs has captured the attention of those who have experienced it. Stable Diffusion exhibits proficiency in producing high-quality images while also demonstrating noteworthy speed and efficiency, thereby increasing the accessibility of AI-generated art creation.</p><p>Stable Diffusion’s training involved large public datasets like LAION-5B, leveraging a wide array of captioned images to refine its artistic abilities. However, a key aspect contributing to its progress lies in the active participation of the community, offering valuable feedback that drives the model’s ongoing development and enhances its capabilities over time.</p><h4><strong>How does it work?</strong></h4><p>Let’s start with main building blocks of a Stable Diffusion model and how one can train and make predictions with these models individually.</p><p><strong>U-Net and the Essence of Diffusion Process:</strong></p><p>To generate images using computer vision models, we venture beyond the conventional approach of relying on labeled data, such as classification, detection, or segmentation. In the realm of Stable Diffusion, the goal is to enable models to learn the complex details of images themselves, capturing complex contexts with an innovative approach known as “Diffusion.”</p><p>The diffusion process unfolds in two distinctive phases:</p><ul><li>In the first part, we take an image and introduce a controlled amount of random noise. This step is referred to as <strong>forward diffusion</strong>.</li><li>In the second part, we aim to denoise the image and reconstruct the original content. This process is known as <strong>reverse diffusion</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y5Cr0YF_sNQCZAhop5Y0kw.png" /><figcaption>Noise Addition by steps. Source: <a href="https://arxiv.org/abs/2112.07804">Tackling the Generative Learning Trilemma with Denoising Diffusion GANs</a></figcaption></figure><p>The first part, which involves adding Gaussian noise to the input image at each time step <strong>t</strong>, is relatively straightforward. However, the second stage poses a challenge, as directly computing the original image is not feasible. To overcome this obstacle, we employ a neural network, and this is where the ingenious <strong>U-Net </strong>comes into play.</p><p>Leveraging <strong>U-Net</strong>, we train our model to predict noise from a given randomly noised image at time step t and calculate the loss between the predicted and actual noise. With a sufficiently large dataset and multiple noise steps, the model gains the ability to make educated predictions on noise patterns. This trained U-Net model also proves invaluable for generating approximate reconstructions of images from given noise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*80sKE188DKNHkN4BKUco0g.png" /><figcaption>Reverse Diffusion. Source: <a href="https://arxiv.org/abs/2006.11239">Denoising Diffusion Probabilistic Models</a></figcaption></figure><p>If you are familiar with basic probability and computer vision models, this process is relatively straightforward. However, there is one more issue worth noting. Training millions of noise-added images and rebuilding them would be extremely time-consuming and would deplete computing power. To address this challenge, researchers have revisited a well-known architecture: <strong>Autoencoders</strong>. As we have already utilized a similar approach with U-Net, incorporating transpose convolutions and residual blocks, some of these elements will also prove vital role in autoencoders too.</p><p>With autoencoders, one can “<strong>encode</strong>” the data into a much smaller “<strong>latent</strong>” space and “<strong>decode</strong>” it back into the original space. Actually that’s why the original Stable Diffusion paper is called <strong>Latent Diffusion.</strong> This allows us to effectively compress large images into lower dimensions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/1*NIcwIxy-JdutHq2_jUk0zw.jpeg" /><figcaption>Simple AutoEncoder representation. Image by author.</figcaption></figure><p>The forward and reverse diffusion operations will now occur within significantly smaller latent spaces, resulting in reduced memory requirements and substantially faster processing.</p><p>We are nearly finished with the renowned “Stable Diffusion” architecture; the only remaining part is the <strong>conditioning</strong>. Typically, this aspect is achieved using <strong>Text Encoders</strong>, though other methods using images as conditioning, such as <strong>ControlNet</strong>, exist, though it falls outside the scope of this article. <strong>Text conditioning</strong> plays a pivotal role in generating images based on text prompts, where the true magic of the Stable Diffusion model lies.</p><p>To achieve this, we can train a text embedding model like <strong>BERT</strong> or <strong>CLIP</strong> using images with captions and add token embedding vectors as conditioning inputs. Employing a <strong>cross-attention</strong> mechanism (queries, keys and values), we can map the conditional text embeddings into U-Net residual blocks. Consequently, we can incorporate image captions alongside the images themselves during the training process and effectively condition image generations based on the provided text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y-tZocQ8RoArpiGT.png" /><figcaption>Latent Diffusion Model — Source: <a href="https://arxiv.org/pdf/2112.10752.pdf">Paper</a></figcaption></figure><p>Now that you know all about the building blocks of a Stable Diffusion model, armed with this knowledge, we can readily compare the previous Stable Diffusion models and make a more informed assessment of their strengths and limitations.</p><h3>What’s New in SDXL?</h3><p>Now that we have grasped the fundamentals of SD models, let’s delve into the SDXL paper to uncover the transformative changes introduced in this novel model. In summary, SDXL presents the following advancements:</p><ul><li><strong>Increased Number of U-Net Parameters:</strong> SDXL enhances its model capacity by incorporating a larger number of U-Net parameters, allowing for more sophisticated image generation.</li><li><strong>Heterogeneous Distribution of Transformer Blocks:</strong> Departing from the uniform distribution of transformer blocks in previous models ([1,1,1,1]), SDXL adopts a heterogeneous distribution ([0,2,4]), introducing optimized and improved learning capabilities.</li><li><strong>Enhanced Text Conditioning Encoder:</strong> SDXL leverages a bigger text conditioning encoder, OpenCLIP ViT-bigG, to effectively incorporate textual information into the image generation process.</li><li><strong>Additional Text Encoder:</strong> The model employs an additional text encoder, CLIP ViT-L, which concatenates its output, enriching the conditioning process with complementary textual features.</li><li><strong>Introducing “Size-Conditioning”:</strong> A novel conditioner called “Size-Conditioning” takes the original training image’s width and height as conditional input, enabling the model to adapt its image generation based on size-related cues.</li><li><strong>“Crop-Conditioning” Parameter:</strong> SDXL introduces the “Crop-Conditioning” parameter, incorporating image cropping coordinates as conditional input.</li><li><strong>“Multi-Aspect Conditioning” Parameter:</strong> By incorporating the bucket size for conditioning, the “Multi-Aspect Conditioning” parameter enables SDXL to cater to various aspect ratios.</li><li><strong>Specialized Refiner Model:</strong> SDXL introduces a second SD model specialized in handling high-quality, high-resolution data; essentially, it is an img2img model that effectively captures intricate local details.</li></ul><p>Now, let’s take a closer look at how some of these additions compare to previous stable diffusion models.</p><p><strong>Stability.ai’s Official Comparison:</strong></p><p>Let’s begin by examining stability.ai’s official comparison, as presented by the authors. This comparison offers valuable insights into user preferences between SDXL and Stable Diffusion. However, we must approach the findings with a cautious eye…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*LtwH_eFKaSdmWu2QG7JhSw.png" /><figcaption>Comparing user preferences between SDXL and previous models. Source: <a href="https://arxiv.org/abs/2307.01952">Paper</a></figcaption></figure><p>This study demonstrates that participants chose SDXL models over the previous SD 1.5 and 2.1 models. In particular, the SDXL model with the Refiner addition achieved a win rate of 48.44%. It is important to note that while this result is statistically significant, we must also take into account the inherent biases introduced by the human element and the inherent randomness of generative models.</p><p><strong>Performance Against State-of-the-Art Black-Box Models:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HimRbY_AktPkm9iKGKdi0Q.png" /><figcaption>Source: <a href="https://arxiv.org/abs/2307.01952">Paper</a></figcaption></figure><p>Currently, Midjourney is highly popular among users, with some considering it to currently be a state-of-the-art solution. According to the official survey, SDXL exhibits a higher preference rate in categories such as “Food &amp; Beverage” and “Animals.” However, in other categories like “Illustrations” and “Abstract,” users still prefer Midjourney V5.1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*azetBdooN43yupuCjTpx9A.png" /><figcaption>Source: SDXL: <a href="https://arxiv.org/abs/2307.01952">Improving Latent Diffusion Models for High-Resolution Image Synthesis</a></figcaption></figure><p>Once again, we observe a similar pattern in the case of complex prompts. The paper asserts that they are preferred in 7 out of 10 complex subjects. However, without knowledge of the specific prompts used, it becomes challenging to draw a conclusion. Additionally, the lack of information regarding the prompt encoder in Midjourney further complicates matters, and only time will reveal the true preferences.</p><p><strong>Number of U-Net Parameters</strong></p><p>As mentioned earlier, U-Net models play a vital role in Stable Diffusion, facilitating the reconstruction of images from given noise. In SDXL, the authors have made a noteworthy improvement by incorporating a considerably larger U-Net model compared to previous versions of SD in total of 2.6B U-Net parameters compared to ~860M for its predecessors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*f3JHoIQ28GFi_nXR8tGVUA.png" /><figcaption>Image by Author.</figcaption></figure><p>Although having more parameters may seem promising initially, it is essential to consider the tradeoff between complexity and quality. As the number of parameters increases, so do the system requirements for both training and generating. While it is still premature to arrive at definitive conclusions regarding the product’s quality, the tradeoff between complexity and quality is already evident.</p><p><strong>Number of Text Encoder Parameters</strong></p><p>Indeed, when examining the total number of text encoder parameter numbers, we observe a notable increase in SDXL 1.0 compared to its predecessors. The introduction of two text conditioners in SDXL, as opposed to a single one in previous versions, accounts for this significant growth in the text encoder’s parameter count. This expansion empowers SDXL to leverage a larger volume of textual information.</p><p>SDXL utilizes the OpenCLIP G/14 text encoder with 694.7 million parameters, compared to CLIP L/14 with 123.65 million parameters, resulting in a total of over 800 million parameters. This represents a substantial leap from its predecessors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*ilZJH_B8ImqWDbF3iezu5Q.png" /><figcaption>Image by Author.</figcaption></figure><p>Once more, employing larger and multiple text encoders may appear appealing initially, but it introduces added complexity that could prove detrimental. Consider a scenario where you are fine-tuning a SDXL model with your own data. In such cases, determining optimal parameters becomes more challenging than previous SD models because finding the “Sweet Spot” of hyperparameters for both encoders becomes more elusive.</p><p><strong>The Workflow</strong></p><p>Indeed, the term “XL” in SDXL is indicative of its expanded scale and increased complexity when compared to previous SD models. SDXL surpasses its predecessors in various aspects, boasting a larger number of parameters, including two text encoders and two U-Net models — the base model and the refiner, which essentially functions as an image-to-image model. Naturally this heightened complexity of SDXL pipelines:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Di_iM6_V0P2gxoOVC3eTOg.jpeg" /><figcaption>Usual SD 1.5 generation pipeline. Image by Author</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bJlZzO_opBpLz5KN49dlXw.jpeg" /><figcaption>SDXL generation pipeline. Image by Author</figcaption></figure><h4><strong>SDXL in Practice</strong></h4><p>The model weights of SDXL have been officially released and are freely accessible for use as Python scripts, thanks to the diffusers library from Hugging Face. Additionally, there is a user-friendly GUI option available known as ComfyUI. This GUI provides a highly customizable, node-based interface, allowing users to intuitively place building blocks of the Stable Diffusion model and visually connect them.</p><p>With this readily available implementations, users can seamlessly integrate SDXL into their projects, enabling them to harness the power of this cutting-edge latent text-to-image diffusion model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/846e32384fb372d96c965a1e52e0daf6/href">https://medium.com/media/846e32384fb372d96c965a1e52e0daf6/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T2jiNUMiK1M9u41rsmmIBw.png" /><figcaption>An Example of ComfyUI workflow pipeline. Image by author.</figcaption></figure><h3><strong>Current State of SDXL and Personal Experiences</strong></h3><p>While the new features and additions in SDXL appear promising, some fine-tuned SD 1.5 models are still delivering better results. This outcome is primarily attributed to the great support from the thriving community — an advantage that stems from the open-source approach.</p><p>At its initial model stage, SDXL exhibits improvements over 1.5, and I am confident that with continued community support, its performance will only grow stronger in the future. However, it is essential to acknowledge that as models become more complex, using and fine-tuning them computationally demands greater resources. But there’s no need for concern yet…</p><p><strong>LoRA</strong>s (Locally Rank-Adaptive Decompositions) has gained popularity for <strong>fine-tuning Large Language Models</strong>. This approach involves adding pairs of rank-decomposition weight matrices to existing weights and training only these newly added weights. As a result, training becomes faster and computationally more efficient. The incorporation of LoRAs is expected to pave the way for the community to create even better custom versions in the near future. Notably, SDXL already fully supports LoRAs.</p><p>Despite the positive developments, it’s worth noting that SDXL still grapples with some of the usual Stable Diffusion shortcomings, as officially acknowledged by the authors:</p><ul><li>The model does not achieve perfect photorealism</li><li>The model cannot render legible text</li><li>The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”</li><li>Faces and people in general may not be generated properly.</li><li>The autoencoding part of the model is lossy.</li></ul><p><strong>Personal Observations</strong></p><p>Personally, while experimenting with the SDXL model, I still find myself favoring the previous SD 1.5 community checkpoints in certain cases. With months of community support, it is relatively easy to find the right fine-tuned model that suits specific needs, such as photorealism or more cartoonish styles. However, it is currently challenging to find specific fine-tuned models for SDXL due to the high computing power requirements. Nevertheless, the base model of SDXL appears to perform better than the base models of SD 1.5 or 2.1 in terms of image quality and resolution, and with further optimizations and time, this might change in the near future.</p><p>It’s worth noting that with the increased model size, some users have reported difficulties running the model on their everyday laptops or PCs, which is unfortunate. I am hopeful that the quantization techniques commonly used in Large Language Models may find their place in this field too.</p><p>Additionally, with the change in text encoders, my usual prompts no longer yield satisfactory results on SDXL. Although the developers of SDXL claim that prompting is easier with SDXL, I have yet to find the right approach myself. It may take some time to adjust to the new prompt style, especially for those coming from previous versions.</p><h3><strong>Conclusion</strong></h3><p>In our article, we discovered the capabilities of Stable Diffusion XL, a model with the ability to transform plain text descriptions into intricate visual representations. We found that SDXL’s open-source nature and its approach to addressing concerns related to black-box models have contributed to its widespread appeal, allowing it to reach a broader audience.</p><p>With its increased number of parameters and extra features SDXL has proven itself as an “XL” model, boasting heightened complexity compared to its predecessors.</p><p>Implementing SDXL in practice has been made easier with the official release of its model weights, freely available as Python scripts from huggingface, along with the user-friendly ComfyUI GUI option.</p><p>While SDXL shows great promise, the journey towards perfection is still ongoing. Some finely-tuned SD 1.5 models continue to outperform SDXL in certain scenarios, thanks to the vibrant community support. However, with active engagement and support, I can see that SDXL will continue to evolve and improve over time.</p><p>Nonetheless, it is important to acknowledge that like its predecessors, SDXL does have some limitations. Achieving perfect photorealism, rendering legible text, handling compositionality challenges, and accurately generating faces and people are among the areas that the model is needs to be improved upon.</p><p>In conclusion, SDXL 1.0 represents a significant leap forward in text-to-image generation, unleashing the creative potential of AI and pushing the boundaries of what’s possible. As the AI community continues to collaborate and innovate, we can look forward to witnessing even more astonishing developments in the fascinating world of SD models and beyond.</p><h3>References:</h3><ul><li><a href="https://arxiv.org/abs/2006.11239">Denoising Diffusion Probabilistic Models — Jonathan Ho, Ajay Jain, Pieter Abbeel, 2020</a></li><li><a href="https://arxiv.org/abs/2112.10752">High-Resolution Image Synthesis with Latent Diffusion Models — Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer, 2021</a></li><li><a href="https://arxiv.org/abs/2307.01952">SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis — Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, Robin Rombach, 2023</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e739d5cc6c7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-arrival-of-sdxl-1-0-4e739d5cc6c7">The Arrival of SDXL 1.0</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>