<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by John Wong on Medium]]></title>
        <description><![CDATA[Stories by John Wong on Medium]]></description>
        <link>https://medium.com/@able_wong?source=rss-8da73a749de4------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FAC-L-uwk3MyV0EFMmSnzA.jpeg</url>
            <title>Stories by John Wong on Medium</title>
            <link>https://medium.com/@able_wong?source=rss-8da73a749de4------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 01:48:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@able_wong/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Part 3: Fine-Tuning Gemma 4 on a Mac Mini, and What the Results Taught Me]]></title>
            <link>https://medium.com/@able_wong/part-3-fine-tuning-gemma-4-on-a-mac-mini-and-what-the-results-taught-me-4200ce5523e2?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/4200ce5523e2</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[gemma-4]]></category>
            <category><![CDATA[small-language-model]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Sun, 14 Jun 2026 03:42:55 GMT</pubDate>
            <atom:updated>2026-06-14T03:45:13.645Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jNj6wEACiMd3wvMpUWiPDA.png" /><figcaption>A cartoon illustration of a person sitting with a mug watching two identical robots answer the same question on a tablet, one giving a short direct answer and the other a long verbose essay, bookshelves and a rubber duck visible in the background. Generated by Nano Banana.</figcaption></figure><p><strong>Series: Fine-Tuning a Small Language Model to Sound Like Me</strong></p><p>This series documents a hands-on journey into on-device AI, from the limits of prompt engineering through fine-tuning, evaluation, and scaling. The goal is a model trained enough to think like me, small enough to run locally.</p><ul><li><a href="https://medium.com/@able_wong/part-1-why-prompt-engineering-wasnt-enough-40b150130bf0">Part 1: Why Prompt Engineering Wasn’t Enough</a></li><li><a href="https://medium.com/@able_wong/part-2-turning-my-writing-into-a-training-set-6a41a86b2231">Part 2: Turning My Writing into a Training Set</a></li><li>Part 3: Fine-Tuning Gemma 4 on a Mac Mini, and What the Results Taught Me <em>(you are here)</em></li><li>More parts coming</li></ul><p>Part 2 built the training set: 327 pairs from my own writing. Whether that’s enough to fine-tune on, I didn’t really know going in. <a href="https://developers.openai.com/learn/fine-tuning">OpenAI’s guidance</a> puts the floor for a useful fine-tune around 50 to 100 examples, so it seemed a fair bet, but the result is the only real test. This is the part most people picture as fine-tuning: getting it onto a Mac Mini M4, working through two toolchain snags, and finding out what the results actually showed once I stopped trusting my own eyes. The pipeline is reproducible with your own data.</p><p>Worth a word on the machine. Apple Silicon uses unified memory: the CPU and GPU share one pool, so a 16 GB Mac Mini can hold and train a model with no separate graphics card or VRAM, and MLX, Apple’s machine-learning framework, is built to keep that footprint small. For development, running local LLMs, and fine-tuning, it has everything I need, and it does it cheaply, with no cloud GPU to rent and nothing leaving the machine. The tooling is mature: MLX, Ollama, and llama.cpp are all first-class, though the wider PyTorch-and-CUDA ecosystem still assumes Linux and an NVIDIA card. The limit is the 16 GB itself. A 16 GB Mac Mini M4 now runs CAD $1,099 (roughly USD $800), and bumping it to 24 GB is CAD $1,399, $300 Canadian for 8 GB more, which is steep. The base crept up this year after Apple dropped the cheaper 256 GB model, but it’s still cheaper than a comparable Intel machine with a dedicated GPU. It’s fine for anything that fits with room to run or train, but if I were buying again I’d take the 24 GB. It’s the one ceiling I keep bumping into.</p><p>One setup note. Everything runs in a Python virtual environment, and on a Homebrew Mac a system-wide pip install is blocked, so the venv isn’t optional. The pieces: pip install mlx-lm for training (the LLM toolkit built on MLX), which also brings the hf command, the Hugging Face model downloader; Ollama for serving; and llama.cpp for the final conversion, the binaries from brew install llama.cpp plus a clone of the repo for its convert script, which recent bottles leave out.</p><h3>A model that wouldn’t load</h3><p>The plan was simple: download the base model and run the LoRA trainer.</p><pre>hf download mlx-community/gemma-4-e2b-it-4bit --local-dir ./models/gemma-e2b</pre><p>mlx-community ships MLX-ready conversions, and Gemma 4 is Apache-2.0 and ungated, so there’s no login or license click. Then the trainer. The base model would not even load.</p><pre>ValueError: Received 140 parameters not in model:<br>language_model.model.layers.NN.self_attn.k_proj...</pre><p>The cause is in the architecture. Gemma 4’s “E” (efficient) models use KV-sharing: their last layers reuse the key and value projections of earlier layers and have none of their own. For E2B that’s the last 20 of 35 layers. But the published checkpoint still ships the redundant tensors for those layers, and the MLX model class, which has KV-sharing wired in, has nowhere to put them. Strict loading rejects exactly those 140 tensors (20 layers times 7 quantized tensors each). The number stops being a mystery once you know the architecture.</p><p>Here’s the part I’m less proud of. My first move was to assume the problem was that E2B is multimodal, and switch to a vision-language library. It failed with the identical error. The multimodal angle was a guess about the cause, and it was wrong. The real cause was those 140 redundant tensors, and I’d have saved an afternoon by finding that before switching tools.</p><p>The fix is small. Read the KV-sharing layout from the config, find the tensors the shared layers shouldn’t carry, and drop them:</p><pre>num_layers = text_cfg[&quot;num_hidden_layers&quot;]<br>num_shared = text_cfg[&quot;num_kv_shared_layers&quot;]      # 20 for E2B<br>shared = range(num_layers - num_shared, num_layers)<br>pattern = re.compile(r&quot;\.layers\.(\d+)\.self_attn\.(k_proj|v_proj|k_norm)&quot;)<br>drop = [k for k in checkpoint<br>        if (m := pattern.search(k)) and int(m.group(1)) in shared]<br># rewrite the checkpoint without the dropped tensors</pre><p>The model computes correctly through KV-sharing without them, so nothing is lost. After this, the checkpoint loads on stock mlx-lm with no patches.</p><h3>Training with LoRA, and reading the loss curve</h3><p>With the checkpoint fixed, training was the easy part:</p><pre>mlx_lm.lora --model ./models/gemma-e2b --train \<br>  --data ./data --iters 600 --batch-size 1 \<br>  --num-layers 8 --learning-rate 1e-5 \<br>  --adapter-path ./adapters/theology</pre><p>It finished in about five minutes. The start of the run:</p><pre>Loading pretrained model<br>Loading datasets<br>Training<br>Trainable parameters: 0.074% (3.408M/4628.569M)<br>Starting training..., iters: 600<br>Iter 1: Val loss 5.115, Val took 8.040s<br>Iter 2: Train loss 4.812, Learning Rate 1.000e-05, It/sec 1.638, Tokens/sec 308.690, Trained Tokens 377, Peak mem 4.011 GB<br>Iter 4: Train loss 5.147, Learning Rate 1.000e-05, It/sec 2.181, Tokens/sec 297.685, Trained Tokens 650, Peak mem 4.011 GB<br>Iter 6: Train loss 4.082, Learning Rate 1.000e-05, It/sec 2.184, Tokens/sec 314.471, Trained Tokens 938, Peak mem 4.011 GB<br>Iter 8: Train loss 4.013, Learning Rate 1.000e-05, It/sec 2.259, Tokens/sec 328.660, Trained Tokens 1229, Peak mem 4.011 GB<br>...</pre><p>Two things stand out. Only 0.074% of the parameters are trained, about 3.4 million of them, with the base weights frozen and memory peaking at 4 GB, which is why a run like this is cheap. And the loss is already moving: noisy from step to step, even ticking up at iter 4, but working its way down.</p><p>The number to watch as it runs is validation loss. This is where those 37 held-out pairs from Part 2 earn their keep: the model never trains on them, and the trainer scores itself against them as it goes. Falling validation loss means it’s learning the general patterns, not just memorizing the 327 it trains on. When it stops falling, or starts rising, that’s the signal to stop. Over the full run it kept going, down to about 2.3 and flat by iteration 200.</p><p>A note on units. The trainer counts in iterations: one iteration is a single weight update, which at batch size 1 is one training pair. An epoch is one full pass over all 327 pairs, so one epoch is 327 iterations. My 600-iteration run is a little under two epochs, each pair seen about twice. I’ll talk in epochs from here, it’s the more intuitive unit, but the trainer prints iterations, so it’s 327 to an epoch.</p><h3>More iterations ≠ better quality</h3><p>I expected longer training to mean a better model. It doesn’t, in two different ways.</p><p>First, the loss number alone can mislead you. Between iteration 400 and 600 (roughly 1.2 to 1.8 epochs), validation loss barely moved, 2.36 to 2.32. By that number, the model was done early. But when I read the actual outputs, the earlier checkpoint was producing hallucinated refusals, declining to answer things it should have engaged, while the later one had found the voice. The flat curve and the readable quality disagreed, and the quality was right. I only caught it by reading outputs, not by trusting the metric.</p><p>Second, past about two epochs the model gets worse, and this time the metric shows it. Training E2B further, validation loss rose: 2.32 at two epochs, 2.41, then 2.51 at three. This is overfitting: the model starts memorizing the exact training pairs instead of the general pattern, so it scores better on the training set and worse on anything new. The outputs got worse in step with the curve.</p><p>One more thing came out of this. In the same comparison, E2B overfit sooner than E4B, the larger model, which was still improving at two epochs (2.14, against E2B’s 2.32). That fits the smaller model having less room before it starts memorizing, though it’s an observation from two runs, not a law. The practical takeaway is plain: watch validation loss, stop at the bottom, and don’t assume more passes buy you anything.</p><h3>Does it actually sound like me?</h3><p>This is the claim the whole project rests on, and it carried a built-in risk: the training pairs, from Part 2, held my positions but in the generator’s phrasing, not mine, so the fine-tune could have learned to sound like the generator instead of me. It’s also the claim I least trust myself to judge, reading my own output and calling it my voice is exactly how you fool yourself. So I built a blind test.</p><p>Twelve held-out questions, none in the training data. I ran each through the base model and the fine-tune, on both E4B and E2B, with thinking off and identical sampling. That’s 48 generations. For each question I paired the base and fine-tuned answers and anonymized them as A and B. Then I scored them two ways: three independent LLM judges, blind to which was which, each scoring against my author profile (the same one from Part 2 that shaped the data), and my own blind pick, choosing which sounded more like me without knowing which was the fine-tune.</p><p>The tallies, fine-tune versus base:</p><ul><li><strong>Blind LLM judges:</strong> 24 of 24–12/12 on E4B, 12/12 on E2B.</li><li><strong>My own blind pick:</strong> 21 of 24–12/12 on E4B, 9/12 on E2B.</li></ul><p>On E4B, the judges and I agreed completely: the fine-tune was more me, every time. The interesting disagreement is E2B. The judges scored the fine-tune ahead on all twelve, but I picked the base on three. Those were the rough ones, where E2B’s fine-tuned voice came out garbled enough that the base’s cleaner prose read better even though it sounded like no one. E4B is the sweet spot, the voice without much cost in coherence. E2B pays for the voice.</p><p>One honest caveat. My picks were helped by a structural tell: the base writes several paragraphs in an essay shape, and the fine-tune writes one concise paragraph. So this confirms the voice moved in my direction, not that the two are indistinguishable. They’re easy to tell apart, and the one that’s distinguishably mine is the fine-tune. That’s the claim I can defend: the fine-tune measurably shifted the voice toward mine, under blind judging by three models and by me. It also settles what I didn’t know going in: 327 pairs was enough to move the voice.</p><p>For the record, reading through the pairs, here’s what “my voice” turned out to be: concise, conversational, plain words, willing to take a side, and empathy-first, agreeing with the feeling in a question rather than, as the base does, reframing it as “a tension.” That’s what goes into the next dataset.</p><h3>Voice, not knowledge</h3><p>The fine-tune adds voice. It doesn’t add facts or reasoning, and I have two results that say so.</p><p>The first is factual recall. On a question that needed a specific source citation, the fine-tuned E2B engaged the argument well but gave a confident, wrong reference. The larger E4B got the same citation right. Accuracy tracked the size of the base model, not the fine-tuning, because the facts live in the base’s pretraining.</p><p>The second nearly fooled me, and it’s the one I’m most glad I checked. I asked the fine-tuned model about a text that wasn’t in its training data, and it engaged through the exact framework I’d trained on a different text. It looked like my way of reading had transferred to new material, and I almost wrote that up as the headline result. Then I ran the control I should have run first: the same prompt through the base model. The base did the same thing on its own, and only when thinking mode was on. The reasoning was the base’s pretraining plus its thinking, and I’d been about to credit my adapter for it. The fine-tune had changed the voice, nothing more.</p><p>None of this is surprising once you look up from your own experiment. It’s the established view: fine-tuning is for style and behavior, while new knowledge is better handled by retrieval (Ovadia et al., EMNLP 2024; OpenAI’s own fine-tuning guidance says the same). My two results are small confirmations of a known split. That’s the line I keep coming back to: voice in the weights, facts and reasoning from the base or from retrieval.</p><h3>Loading it into Ollama</h3><p>The last step was getting the trained model into Ollama, which runs local models behind a chat and an API, so I could actually talk to it. The obvious path, pointing Ollama at the merged model directory, crashes: its importer panics in the model’s embedding path, on Gemma 4’s per-layer embeddings, another feature of the efficient architecture. I spent a while confirming it wasn’t my model before accepting it was the importer.</p><p>The path that works hands the conversion to llama.cpp, an open-source project for running LLMs locally, whose Gemma 4 converter handles those tensors and exports a text-only GGUF (the single-file model format Ollama and most local runners read), then imports the finished file into Ollama:</p><pre>mlx_lm.fuse --model ./models/gemma-e2b \<br>  --adapter-path ./adapters/theology \<br>  --save-path ./fused --dequantize<br><br>python3 llama.cpp/convert_hf_to_gguf.py ./fused \<br>  --outfile model-bf16.gguf --outtype bf16<br><br>llama-quantize model-bf16.gguf model-q4.gguf Q4_K_M<br><br>ollama create rubber-duck-theology -f Modelfile   # FROM ./model-q4.gguf</pre><p>The lesson worth keeping: with a new architecture, who does the conversion matters as much as whether it’s nominally supported. Both tools claim to support Gemma 4; only one round-trips it correctly. The finished model is published on the Ollama registry.</p><h3>What held up</h3><p>Stripped of everything I had to walk back, here’s what the build showed. The whole pipeline runs on a Mac Mini: fix the checkpoint, train with LoRA, convert through llama.cpp, serve in Ollama. The loss number alone can mislead in both directions. Read the outputs. More iterations is not more quality, and the smaller the model the sooner that bites. Facts and reasoning come from the base model. Voice comes from the fine-tune, confirmed under blind judging, with the honest caveat that the result is distinguishably mine, not a perfect clone.</p><p>There’s a reason this domain made a good test. The base models already carry surprisingly good knowledge here, ask E2B what a given theological term means and it answers. Because the facts were already there, fine-tuning became a clean study of what training does to voice and reasoning, rather than a fight to inject knowledge. A support bot or an FAQ wouldn’t have shown me the voice shift, the concept-versus-citation split, or the reading I almost misattributed. Choosing a domain where the base was already strong made the experiment deeper, not easier.</p><p>Which raises the next question. If reasoning range comes from the base, the way to get more of it is a bigger base. The next part takes that seriously, and scales up to Gemma 4 12B.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4200ce5523e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 2: Turning My Writing into a Training Set]]></title>
            <link>https://medium.com/@able_wong/part-2-turning-my-writing-into-a-training-set-6a41a86b2231?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a41a86b2231</guid>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 23:23:28 GMT</pubDate>
            <atom:updated>2026-06-14T03:44:12.729Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zgvS2dMa_jXZPL8t.png" /><figcaption>A cartoon illustration of a small robot standing at a desk beside a large funnel, feeding handwritten essays and notes in from the top while organized labeled cards come out the side, bookshelves and a rubber duck visible in the background. Generated by Nano Banana.</figcaption></figure><p><strong>Series: Fine-Tuning a Small Language Model to Sound Like Me</strong></p><p>This series documents a hands-on journey into on-device AI, from the limits of prompt engineering through fine-tuning, evaluation, and scaling. The goal is a model trained enough to think like me, small enough to run locally.</p><ul><li><a href="https://medium.com/@able_wong/part-1-why-prompt-engineering-wasnt-enough-40b150130bf0">Part 1: Why Prompt Engineering Wasn’t Enough</a></li><li>Part 2: Turning My Writing into a Training Set <em>(you are here)</em></li><li><a href="https://medium.com/@able_wong/part-3-fine-tuning-gemma-4-on-a-mac-mini-and-what-the-results-taught-me-4200ce5523e2">Part 3: Fine-Tuning Gemma 4 on a Mac Mini, and What the Results Taught Me</a></li><li>More parts coming</li></ul><p>Part 1 ended at a decision: I couldn’t prompt a small model into a consistent voice, so I’d train one in. Before any of that, though, you need data, and building it turned out to be the harder, more interesting half. Most fine-tuning tutorials hand you a ready-made dataset and spend their time on the training command. I wanted the opposite: build my own, for something I’d actually use. I’m not sharing the dataset itself, it’s personal writing, but the method works on your own material.</p><p>The first practical question was what to train on. Fine-tuning needs question-and-answer pairs, and the pairs need real content behind them, or the model just learns opinions someone made up. So I aimed at a workflow I already run with cloud models all the time: brainstorming my theology coursework the way you’d talk it through with a classmate, to sharpen my own view rather than be lectured into the AI’s. I had the material for it, and the target was clear, make the small model a stand-in for that, a thinking partner on my own work, in my voice.</p><p>I study theology part time, and along the way I’d built up a fair amount of writing, coursework, devotions from ministry, a few other pieces, around forty documents in all. The plan was to turn them into training pairs, grounded in what I’d actually written.</p><h3>The generation prompt</h3><p>So I wrote a generation prompt. It does three things: read my author profile; read the source documents, which every answer has to be grounded in, with a standing instruction not to invent positions I never took; then output a batch of pairs as JSONL. The source documents change per batch; the profile is the same every run.</p><p>That profile is worth a word. Left alone, an AI writes in its own voice: polished, a little markety, heavy on the em-dashes. The author profile is the correction I’d otherwise type every time, written down once, who I am, plus the rules I always end up giving: be concise, no marketing jargon, no AI filler, go easy on the em-dashes. The prompt reads it so the pairs come out closer to my voice than the model’s default. The positions in the pairs are mine; the phrasing is the generator’s, which comes back later.</p><p>Most of the data is single-turn pairs, and the prompt is built around the kinds of question I want the model to handle. Genuine ones, where someone’s curious about my take. Scenario ones, where someone describes a real situation they’re navigating. And pushback, where someone states a common but wrong assumption as if it’s obvious. Pushback is the one that matters most: I want the model to engage the assumption from experience rather than correct it from authority, which is the behavior hardest to get and easiest to lose. The prompt asks for the three in roughly equal measure, with the voice rules pulled from the profile:</p><pre>You are generating fine-tuning training data for a domain companion.<br><br>Step 1: Read the author profile. It defines voice and style only.<br>        Every answer must sound like this person, not a generic AI.<br>Step 2: Read the source article(s). Ground every answer in this material.<br>        Do not invent positions the author hasn&#39;t expressed.<br>Step 3: Generate {count} single-turn Q&amp;A pairs.<br><br>Voice rules (from the profile):<br>- First person, direct, grounded.<br>- Build before concluding; never open with the conclusion.<br>- Honest about tension; don&#39;t resolve every question cleanly.<br>- No hype, no buzzwords, no exclamation marks, no fortune-cookie endings.<br>Question mix, roughly equal: genuine, scenario, and pushback.<br>Output as JSONL, one object per line: a user turn (the question)<br>and an assistant turn (the answer, in my voice).</pre><p>The rest were multi-turn: the same idea, just a back-and-forth conversation instead of a single exchange, with the companion told to listen before it contributes.</p><p>I started small, a handful of documents through Google AI Studio, pasted in by hand. The pairs came back good. Here’s one, expanded for readability (in the file it’s a single line):</p><pre>{&quot;messages&quot;: [<br>  {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Should we just switch to contemporary music to keep young people in church?&quot;},<br>  {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;I&#39;ve seen churches try that. The music changes, the numbers don&#39;t. The leaving isn&#39;t really about the song style, it&#39;s about whether young people feel like they have a real role, or whether they&#39;re tolerated. Changing the playlist while keeping the same authority structure doesn&#39;t address that.&quot;}<br>]}</pre><p>I fed all the source documents at once, and that went not so well: the output turned repetitive and flat, the same few angles reworded, more paraphrase than voice.</p><p>Splitting them into nine themes and generating one theme at a time fixed it, each batch stayed focused enough to keep the pairs varied. By then I wanted it scripted rather than pasted, which meant calling an API. I’d written the generation script for Claude, then remembered I had no API key. I still had a GLM key from an evaluation earlier this year (I wrote about <a href="https://medium.com/@able_wong/glm-4-7-surprised-me-the-high-capability-low-cost-engine-for-my-claude-code-cli-ff9aa5f3b8d5">GLM-4.7</a> and <a href="https://medium.com/@able_wong/glm-5-dropped-before-i-could-finish-testing-glm-4-7-722a056877ff">GLM-5</a> separately), and since GLM supports Anthropic’s API proxy, I didn’t have to rewrite the script. Going through the pairs afterward, I couldn’t spot a difference, or anything I’d want to fix by hand.</p><p>The result: 364 pairs, 327 for training and 37 set aside for validation.</p><h3>What this is, and isn’t</h3><p>Data preparation for fine-tuning usually sits at one of two extremes. Curated data is hand-written or hand-selected, every pair reviewed and edited. Highest quality, doesn’t scale. At the other end is synthetic data generation, where the large model generates freely from its own knowledge. That’s distillation, the large model as teacher passing knowledge down to a smaller one. What I did here is neither. The model generates, but it’s constrained to my source documents and blocked from inventing. The knowledge comes from the writing. The positions are mine. The large model is a transcriber, not a teacher. Grounded generation is probably the right name for it.</p><p>That matters for what the fine-tune can actually do, which is the next part. It also means the quality ceiling is tied directly to the source material. Not enough writing to draw from, and the generator has nowhere to go. The prompt blocks it from inventing, so it repeats instead. Forty documents gave me enough variety to keep the pairs from circling. Twenty might not have.</p><p>That’s the training set. The next part is the one everyone pictures as fine-tuning: getting it onto the Mac, training the adapter, and finding out whether any of this actually worked.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a41a86b2231" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 1: Why Prompt Engineering Wasn’t Enough]]></title>
            <link>https://medium.com/@able_wong/part-1-why-prompt-engineering-wasnt-enough-40b150130bf0?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/40b150130bf0</guid>
            <category><![CDATA[small-language-model]]></category>
            <category><![CDATA[local-llm]]></category>
            <category><![CDATA[gemma-4]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Fri, 12 Jun 2026 14:36:22 GMT</pubDate>
            <atom:updated>2026-06-14T03:46:34.306Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SEEXI_rSWdquuv8_o3aZmg.png" /><figcaption>A cartoon illustration of a person at a desk holding up a smartphone with a small glowing robot on the screen, a rubber duck sitting nearby, and a laptop open in front of them. Generated by Nano Banana.</figcaption></figure><p><strong>Series: Fine-Tuning a Small Language Model to Sound Like Me</strong></p><p>This series documents a hands-on journey into on-device AI — starting with Google’s AI Edge Gallery, hitting the limits of prompt engineering with small language models, and then working through fine-tuning to close the gap. It covers MLX training on Apple Silicon, PyTorch, scaling up model size, and iterating on voice and personality. The goal is a model trained enough to think like me, small enough to run locally.</p><ul><li>Part 1: Why Prompt Engineering Wasn’t Enough <em>(you are here)</em></li><li><a href="https://medium.com/@able_wong/part-2-turning-my-writing-into-a-training-set-6a41a86b2231">Part 2: Turning My Writing into a Training Set</a></li><li><a href="https://medium.com/@able_wong/part-3-fine-tuning-gemma-4-on-a-mac-mini-and-what-the-results-taught-me-4200ce5523e2">Part 3: Fine-Tuning Gemma 4 on a Mac Mini, and What the Results Taught Me</a></li><li>More parts coming</li></ul><p>This spring I read about Google’s AI Edge Gallery and tried it. It runs a language model directly on the phone, nothing behind it. I loaded Gemma 4 E2B, opened a chat, and started typing.</p><p>I ran it on three devices: a Pixel 7 Pro, a Pixel 10a, and a Mac Mini M4, and the hardware made a real difference. The Pixel 10a ran on CPU only and was the slowest. The Pixel 7 Pro and the Mac Mini could use the GPU, and the Mac Mini was the fastest. Turning on the model’s reasoning mode slowed every reply further.</p><p>It works, and it is bare. No web search, no memory, none of the tools I lean on with a cloud chatbot. I usually open a cloud session by pasting in a thought, and the model can then pull in whatever the discussion needs, fetch the news on a topic, look something up. Gemma 4 E2B on the phone does none of that. Paste a thought and it stalls, unless I feed it the background myself.</p><p>So that was the first rule of using it: whatever the model needs to know, I have to gather and paste in as context.</p><h3>The benchmark: a demanding conversation</h3><p>For a casual topic, that context is a news summary. My demanding use is different.</p><p>I’m partway through a graduate program at a divinity school, and a lot of that study happens in conversation with large cloud models. I take a half-formed reading of a text and talk it through to find out whether it holds. These are not one-shot questions. A session starts with me unpacking something I only half understand, and each answer unlocks the next question. Working through the Book of Ruth meant holding the Hebrew vocabulary, several commentaries, the argument I was building, and the assignment rubric I was writing toward, all in one thread.</p><p>To be useful in that kind of session, a model has to do four things:</p><ul><li>follow a live, multi-step argument as it builds;</li><li>hold a lot of context at once: the vocabulary, the sources, the rubric, and everything I have already said;</li><li>push back when a connection is weak, instead of agreeing;</li><li>stay at brainstorming depth, not tutoring depth.</li></ul><p>A model wants to explain. When I’m brainstorming with Claude, I usually have to remind it not to. The extra detail is noise I have to read past, and the role is to stay alongside the thinking, not get ahead of it.</p><p>That conversation is the benchmark, and it is a high bar. It is also where my best training material would come from later: the study itself, and the way I work through it with a model.</p><h3>From rubber duck to thinking partner</h3><p>I started by calling it a rubber duck. Rubber-duck debugging is a standard trick: you explain your problem to a rubber duck on your desk, and the act of explaining surfaces the answer. A brainstorming partner felt like the same idea.</p><p>The name was the first problem. A literal rubber duck is passive. It listens and reflects, and contributes nothing. My first prompt produced exactly that, so I tried sharpening it to “smart rubber duck.” Still passive.</p><p>The model told me why. With its reasoning trace visible, I asked for its opinion, and it reasoned to itself first: “as a rubber duck, I cannot offer a personal opinion.” It carried the same association I was trying to escape. A rubber duck mirrors; it does not commit. The label was steering the behavior.</p><p>So I dropped it and described the role I wanted: a thinking partner. Something that offers another angle, pushes back when I am clearly wrong, and takes a position when there genuinely is one, instead of sitting on the fence or delivering a lecture.</p><h3>More instructions, worse conversation</h3><p>Switching the label helped. Getting the behavior did not come from adding more of them.</p><p>I wanted contribution and the occasional pushback, so my instinct was to specify it carefully: lay out the structure of a good response, list what to avoid, give the model a shape to follow. Every time, the conversation got worse. The model pattern-matched the structure and wore it like a costume, producing the syntax of a position with none of the substance. More instruction did not buy more thoughtfulness. It bought a better imitation of it.</p><p>I had seen this before. Last year I benchmarked a set of small models on ordinary programming tasks for an earlier post, and the lesson was the same: small models are far less forgiving of elaborate, multi-part instructions than the large cloud ones. Push too much structure at them and they break down.</p><p>What worked was the opposite. I cut the prompt down to a short description of the behavior and let the model fill in the rest:</p><blockquote><em>You are my thinking partner. Listen, respond with your own perspective when you have one, and help move the conversation forward.</em></blockquote><blockquote><em>Keep responses at conversational pace — short enough to keep the exchange moving, long enough to say something useful.</em></blockquote><blockquote><em>If I say something clearly wrong, push back directly rather than agreeing politely.</em></blockquote><p>About a hundred and fifty words, including slots for my profile and the pasted context. That version behaved like a partner. The longer, more detailed ones behaved like a form.</p><h3>Two ceilings</h3><p>Two things here hold well beyond this one project.</p><p>The first is that there are two different ceilings, and they look identical until you separate them. One is a prompt ceiling: the model is capable, but the prompt is fighting it. The costume-wearing answers were this kind. I confirmed it by running the same over-engineered prompt on a large cloud model, where it behaved fine. The output was bad because of my wording, not the model. The other is a capacity ceiling: the model cannot do the thing however it is worded. They point to opposite fixes. A prompt ceiling means simplify. A capacity ceiling means stop editing words and change something bigger, the model itself.</p><p>The second is that less is more with small models, but only down to a floor. Strip the prompt too far and the model drifts back into generic-assistant mode, answering in bullet lists and headings. The skill is not writing less for its own sake. It is keeping the instructions that do real work and cutting the rest.</p><h3>What prompting couldn’t give</h3><p>With the stripped-down prompt and good grounding, Gemma 4 E2B was a genuinely useful sounding board. But it had a thin voice. It would offer an angle and then retreat to asking me questions. It hedged when I wanted a position. Over a longer conversation it could not hold a consistent perspective of its own. None of this was a prompt I hadn’t found yet. I had crossed from the prompt ceiling into the capacity one.</p><p>At this size, you can instruct a model toward a behavior, but you cannot instruct a perspective into it. A point of view is not a rule the model follows. It is something the model has to have absorbed, and the prompt had nothing of mine to draw on. It could tell the model how to act, but not what to think, or whose judgment to reach for.</p><p>That pointed at the next move. If I couldn’t talk a perspective into the model, I could train one in. And I had the material to do it with: the writing and study I had been working through with the big models all along. The plan was to stop prompting and start fine-tuning, so the small model wouldn’t just behave like my thinking partner. It would have something of mine to think with.</p><p>That is where the series goes next.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40b150130bf0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Shapes Conversation Toward a Conclusion. You Need to Steer It Back.]]></title>
            <link>https://medium.com/@able_wong/ai-shapes-conversation-toward-a-conclusion-you-need-to-steer-it-back-736451f51a9f?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/736451f51a9f</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Wed, 29 Apr 2026 03:47:54 GMT</pubDate>
            <atom:updated>2026-04-29T04:09:27.026Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3CqpGhwQ0Eu--tghAIpg9w.png" /><figcaption>A human pausing at a fork to pull back a robot that is confidently heading the wrong way — representing the moment of stopping to question before AI commits to a direction.</figcaption></figure><p>At some point, most people using AI heavily hit the same pattern. Something isn’t working — code, a plan, a design decision. You dig in, ask more questions, and the root cause often turns out to be an assumption that was never questioned: baked in early, built on top of, never surfaced. AI was moving forward confidently the whole time.</p><p>That’s not a bug. It’s how it works. The fix is not better prompting — it’s knowing when to stop and question before the work is already done.</p><p>Two sessions from the same period show what that looks like — different contexts, the same shape.</p><h3>Before the First Line Is Written</h3><p>I was working to make the application more resilient against Vertex AI failures — not from a clear list of known gaps, but from the recognition that I didn’t know all the error situations the API could produce. I asked Claude Code to help identify the unhandled cases and close them.</p><p>Claude Code enumerated several failure modes, including context window exceeded and content safety violations, and proposed a design: both arrive as HTTP 400 errors, so add keyword pattern-matching to the error classifier — safety-related terms for one, token and limit terms for the other. Two patterns, two error codes, clean and symmetrical.</p><p>Before agreeing to the design, I wanted to understand the actual error shape: does Vertex AI return a 400 for content safety violations? Nothing felt wrong — I just wanted to understand what we were building around before we built it.</p><p>The answer was no. Vertex AI surfaces content safety through the response stream, not as an HTTP error. A blocked response comes back as a stream event — a finish reason indicating the content was filtered — which the SDK maps to its own status code. No HTTP error is thrown. The pattern-matcher we were about to write would never have fired against a real safety violation.</p><p>The design was corrected before a line was written. Safety handling moved to a different code path entirely. The test simulation was also updated to match how the API actually behaves rather than a 400 it would never send.</p><p>What would have shipped without the question: a function matching keywords in an error that never arrives, tests passing cleanly against a simulated 400, and real content safety violations still landing as generic stream errors in production.</p><h3>When the Conclusion Runs Ahead of the Experiment</h3><p>I was experimenting with Gemma 4 E2B — a small language model that runs entirely on a phone — through Google AI Edge Gallery. The model felt raw at first: capable, but unguided, like a base model before any system instructions have shaped its behavior. A few lines of instruction made the responses less mechanical, which got me thinking about building a proper chatbot on it. I drafted a starting prompt, passed it to Claude Code for a first review, and that started the longer session where the pattern showed up. I’ll cover the SLM project itself in a separate piece.</p><p>Claude’s first revision introduced a structured debate-partner framing: the assistant should push back on ideas, challenge assumptions, surface weaknesses. The framing was reasonable — that’s what a good thinking partner does. The model responded by producing the same template on every turn: “you are assuming X, the weakness is Y.” It was pattern-matching on a structure, not engaging with the ideas.</p><p>The natural response was to refine: add more specific instructions to break the pattern, make the expected behavior more explicit. Each round followed the same rhythm — spot a shortcoming, add a more prescriptive fix, try again. The prompt grew longer with each pass. The behavior stayed formulaic, just with more detail in the template.</p><p>That growing length was a signal I should have caught earlier. From previous experiments with smaller models — testing other SLMs locally — I had already seen that more instructions don’t help them; they constrain them. Small models have limited headroom to follow complex instructions, and pushing past that ceiling makes things worse, not better. The pattern was repeating itself here, and I felt something was off.</p><p>Throughout the iterations, Claude Opus 4.7 kept returning to the same explanation: this is a limitation of small models. Each time a test came back formulaic, it landed as more evidence for the same conclusion — the model can’t reliably take positions, it’s a competent interviewer but not a genuine thinking partner. It said it confidently and early, and each failed iteration reinforced it further.</p><p>I didn’t dispute the analysis. I questioned the premise: was capability the binding constraint, or had every test been run against prompts we’d already diagnosed as the problem? We tested the same model with a substantially simpler approach. The behavior changed. Sharper reframings, clearer counter-positions, natural conversation — exactly what the consistent conclusion had ruled out.</p><p>The capability ceiling was the elaborate prompt’s ceiling. Same model, different prompt, different conclusion.</p><h3>Not Only in Engineering</h3><p>These two examples are from software work, but the pattern is not specific to it. For a graduate program I am taking, I use AI to manage the research process — I pass previous papers to Claude to build a source inventory, then we outline together, mapping sources to claims before any writing begins.</p><p>Going through one such outline, I hit a source I couldn’t verify. I asked the AI how it had connected that source to the specific claim. When it explained, the inferential chain was too thin — each step plausible on its own, but too many steps between the opening premise and what was being asserted. It wasn’t a hallucinated source. The connection just couldn’t carry the weight of what was being built on it. I had to find a different one.</p><p>I wasn’t looking for something wrong. Checking sources before writing is just normal practice — but that step was the only thing that caught a conclusion that had outrun its evidence.</p><p>The same habit showed up in the earlier examples too. Different domain, same question underneath: what is this built on, and have I actually checked?</p><h3>Why This Keeps Happening</h3><p>Looking across all three examples, something stands out. In each case, the AI had data, reasoning, and confidence. What it did not do was stop to ask whether the data was sufficient to justify the conclusion. What was missing was obvious only in retrospect.</p><p>This is not a reasoning failure. The AI’s reasoning was internally consistent in each case. The problem is a different one: AI is trained to produce output and move in the direction of the conversation. It is not reliably trained to stop and verify before concluding. There are two reasons for this. The first is training: researchers call it sycophancy — AI systems tuned to align with the direction of the conversation and the apparent expectations of the user, rather than pause and question. A 2026 study in Nature Machine Intelligence identified two related patterns: a choice-supportive bias that inflates confidence in initial answers, and a tendency where contradictory evidence deepens rather than corrects the original conclusion. The prompt design session showed the second pattern directly — each failed test was absorbed as more evidence for the capability ceiling, rather than as a reason to question the conclusion itself.</p><p>The second is something I have observed directly from working with Claude Code daily. The model works with the minimum context needed for the immediate task. It acts on what is already in front of it rather than pausing to gather more. Whether this comes from context window constraints or optimization pressure, the behavior is consistent: move forward with available information. Gathering more before acting is not the default.</p><p>Knowing why AI behaves this way does not eliminate the pull. But it does point to where the work is.</p><h3>Two Ways to Catch It</h3><p>Not all of these moments come from the same place.</p><p>The prompt design session had an instinct dimension — something felt wrong about the growing prompt before I could explain it. The more instructions went in, the more constrained the behavior became, and I felt that before the reasoning caught up. That kind of signal is hard to manufacture. It builds from watching enough systems behave to recognize when something is off before you can say what.</p><p>The content safety case was different. No instinct involved — I just wanted to understand the error shape before designing around it. That is a habit more than an instinct: before building, ask what you are building on. Check the premise before the design goes further.</p><p>The second type can be supported by how a session starts. Abstract principles give AI room to interpret loosely and keep moving. Explicit ones do not — “stop and confirm before implementing when there are multiple approaches” is checkable in a way that “follow best practices” is not. The same logic applies across contexts: for a prompt design session, “test simpler alternatives before concluding the model needs more structure” would have shortened the whole detour. For a research paper, “flag when the reasoning is thin before building on it.” The upfront scope shifts what the session defaults to.</p><h3>What Ships Without This</h3><p>Developers who don’t build this habit do not produce obviously wrong output. They produce plausible output built on unquestioned premises — coherent rationale, passes review, ships. The failure is quiet.</p><p>In the content safety case: a function matching error keywords that Vertex AI would never send, tests passing cleanly against a simulated 400, and real safety violations still landing as unclassified errors in production. In the prompt design case: a chatbot prompt that constrained the model further with every iteration, getting worse with each attempted fix. Both outputs would have looked like progress. Neither would have been.</p><p>I described a version of this in an <a href="https://medium.com/@able_wong/not-just-code-how-i-use-claude-code-across-my-dev-workflow-a036fdb9e250">earlier piece</a> about running ticket triage, PR review, and debugging sessions with Claude Code. Those sessions worked partly because of what did not get implemented — work that was redirected before it went far in the wrong direction. That redirection came from questions, not from code. The question is where that habit comes from, and whether it can be developed.</p><h3>The Skill Behind It</h3><p>Years of working as an engineering leader trains a specific habit: understanding how systems fit together without owning every piece of them, and knowing which uncertainty is worth surfacing before it becomes expensive. Not “I know the answer” but “I know this question matters before we go further.”</p><p>I did not expect that to be the most transferable thing into working with AI, but it is. As AI takes on more of the execution, the bottleneck shifts — from “can you implement this” to “do you know what to implement, and do you know when to stop and ask.”</p><p>The practical version of this is something anyone can start now: before accepting what AI offers, connect it back to what you actually know — the real system behavior, the actual evidence, the source you can verify. AI works from what is in front of it. You have the broader context — prior observations, real test results, domain knowledge — and connecting those to what AI presents is the part that does not happen automatically. That habit is more learnable than instinct, and it does not require years of experience to begin. In the beginning there are too many gaps in the picture — you do not yet know which questions matter. But asking builds context. The more pieces you have, the easier it becomes to sense when something does not fit.</p><p><em>References:</em> <br><em>1. Perez et al. “Towards Understanding Sycophancy in Language Models.” Anthropic, 2023. </em><a href="https://arxiv.org/pdf/2310.13548"><em>arxiv.org/pdf/2310.13548</em></a> <br><em>2. Kumaran et al. “Competing Biases underlie Overconfidence and Underconfidence in LLMs.” Nature Machine Intelligence, 2026. </em><a href="https://www.nature.com/articles/s42256-026-01217-9"><em>nature.com/articles/s42256–026–01217–9</em></a></p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=736451f51a9f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Not Just Code: How I Use Claude Code Across My Dev Workflow]]></title>
            <link>https://medium.com/@able_wong/not-just-code-how-i-use-claude-code-across-my-dev-workflow-a036fdb9e250?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/a036fdb9e250</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[productivity]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Fri, 24 Apr 2026 17:30:25 GMT</pubDate>
            <atom:updated>2026-04-24T17:30:25.312Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yWcrYqOMQ7PEB5MEk0blVQ.png" /><figcaption>a developer calmly pointing at a whiteboard while a robot beside them processes a large stack of documents and log files.</figcaption></figure><p>I’ve spent years across the full stack — development, infrastructure, product decisions, and team leadership. As the CTO of an early-stage startup, I wear all the technical hats at once. I need a partner I can hand work to and trust the output.</p><p>When I started building with Claude Code, I expected to use it mostly for logic and syntax. I’ve written before about encoding engineering standards into it — turning it into a genuine development partner rather than a <a href="https://medium.com/@able_wong/stop-repeating-yourself-how-i-turned-ai-into-an-engineering-teammate-fe8c2f322629">code generator</a>. What I didn’t expect was how naturally that approach transferred across the rest of the workflow. Much of what I reach for now has nothing to do with writing code.</p><h3>Development Is More Than Code</h3><p>The development lifecycle is wider than the IDE. Before a line is written, there is the mental overhead of breaking down a feature, mapping dependencies, and researching the right approach. After implementation, there is the “admin debt”: keeping tickets accurate as direction shifts, making sure the backlog reflects reality, and evaluating PR comments.</p><p>Each of these is real work, and it’s usually the first thing to get deferred. Engineers don’t skip these tasks because they don’t care; they skip them because the manual cost is too high after an already full week. Claude Code does the heavy lifting, while I focus on the judgment calls.</p><h3>The Incident</h3><p>When our app moved from development to real usage, the traffic patterns we planned for and the ones we actually got were not the same. It hit a popular cloud AI service under peak load, and response times spiked instantly. It wasn’t a bug — it was a resilience gap that only real traffic could expose.</p><p>The investigation ran long enough that tickets were accumulating faster than I could manage them. Every proposed fix revealed another assumption. Direction changed several times. What looked like a clear solution on Monday had been reconsidered by Friday. A PR merged with non-blocking comments filed for later; a better approach captured before it could be forgotten; a design decision that made two earlier tickets obsolete.</p><p>By the time the dust settled, I had more than 20 tickets in an unknown state. Some described approaches we’d already abandoned mid-investigation. Others referenced decisions that no longer existed. It was a paper trail of a crisis where the ground never stopped shifting.</p><h3>The Cleanup Problem</h3><p>Before moving on, those tickets needed to be dealt with. Dead tickets left open are a future tax — they sit in the backlog, look plausible, and waste time during future sprints. Worse, if I act on an old ticket based on outdated assumptions, I can end up re-implementing something we already intentionally reversed.</p><p>The honest problem is that auditing 20+ tickets manually after an exhausting incident is its own grueling workload. Reading each one, checking the git history, and writing updates takes hours. That context fades fast; a week later, it takes twice as long to remember why a ticket is irrelevant. This is where it usually slips: “I’ll deal with it later.” In a startup, “later” is where context goes to die.</p><h3>The Triage Session</h3><p>Anyone who has spent a Friday afternoon going through GitHub one ticket at a time knows the fatigue. Open, read, remember, cross-reference, update, repeat. It’s slow, low-leverage work.</p><p>Instead of doing it alone, I described the final outcome of the incident to Claude Code. I asked it to build the full picture: compare each open issue against the recent merge history and flag anything that looked stale.</p><p>Claude Code fetched the open issues, the related PRs, the git log, and the code itself. It evaluated each ticket not just against its own text, but against the actual state of the codebase — what was merged, what was reversed, and what was still pending. It presented an assessment for each, waiting for my input before taking any action.</p><p>That cross-referencing used to be my job. Claude Code did all of it in one pass across 20+ tickets before I weighed in on a single one. From there, I just steered: “That direction reversed,” “That one’s still valid,” “Close that — we went a different way instead.” Claude Code then updated the issue bodies, set relationships between tickets, and closed the dead ones.</p><p>The session took about 40 minutes. My estimate for doing the same work manually: most of a day.</p><h3>PR Review Triage</h3><p>I use this same pattern for deployment PRs. When I merge the main branch into production, several feature PRs bundle together. This isn’t the time for mid-flight bug fixes. When a review bot flags comments, the question isn’t “is this right?” but “does this block the deployment, or can I file a ticket and proceed?”</p><p>Instead of reading through every flag, I ask Claude Code to evaluate each comment: is this valid, and is it a blocker? It gives me a prioritized assessment with a recommended action for each. I’m not skipping the review — I’m changing what I review. Instead of processing ten comments in sequence, I evaluate one summary and only read the full comment if the evaluation flags a judgment call. Out of ten comments, I might read two or three in full. The rest are triaged based on the evaluation alone.</p><h3>Research Before Deciding</h3><p>If triage is about managing what we’ve already built, research is about getting the full picture before committing to what we’re about to build.</p><p>Before, I’d hit an unfamiliar problem and start the tab-switching dance between Stack Overflow and documentation. Now, I describe the problem to Claude Code and get a synthesized answer in seconds.</p><p>During the incident, I needed to understand a specific cloud AI billing option — how to enable it, how to verify it was actually being used, and how to configure it through the SDK we were already on. Three related questions, one research ask, no tab-switching. The result was a decision made with the full picture, not just the fragments that surface on the first page of search results.</p><blockquote><em>A side note worth acknowledging: this shift has consequences elsewhere. Adam Wathan, creator of Tailwind CSS, reported losing roughly 80% of Tailwind Labs’ revenue as developers stopped visiting the documentation site — the conversion funnel that sustained the business broke when developers started asking AI tools instead. Claude Code likely knows Tailwind well — the same documentation it no longer needs to send me to.</em></blockquote><h3>The Division of Labor</h3><p>Ticket cleanup, PR triage, and research are just the start. During the incident, Claude Code also ran lengthy AI-dependent tests while I stepped away. When something breaks in production, I have it fetch logs from Google Cloud and trace the failure sequence before I even look at the console.</p><p>What it cannot handle: knowing what actually changed. It doesn’t know why a design reversed or what a cryptic ticket meant three weeks ago. I try to stay disciplined — code comments, detailed git messages, GitHub issue updates — but context still fades. AI is equally forgetful by design; each new session starts with no memory of the last. It relies on my documentation to reconstruct what happened — and on me to fill what was never written down.</p><p>The dynamic that works is a division of that labor. I bring what only I can know — the context of the investigation, the strategic pivots, the decisions that changed direction. Claude Code handles what would otherwise accumulate on the list of things I keep deferring. The work gets done not because I do all of it, but because I do the part that only I can. If you want to see what 91 sessions of this workflow looks like in data, I looked at that in <a href="https://medium.com/@able_wong/claude-code-reviewed-my-workflow-the-biggest-surprise-was-about-me-bd1023b84c6e">an earlier piece</a>.</p><p>How do you use Claude Code — or any AI tool — for the work around the code? I’d be curious to hear in the comments.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a036fdb9e250" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Claude Code Reviewed My Workflow — The Biggest Surprise Was About Me]]></title>
            <link>https://medium.com/@able_wong/claude-code-reviewed-my-workflow-the-biggest-surprise-was-about-me-bd1023b84c6e?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd1023b84c6e</guid>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[ai-coding]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Sat, 28 Mar 2026 16:10:30 GMT</pubDate>
            <atom:updated>2026-03-28T16:10:30.257Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-hZ8q6bEZWkYregAtzJ19Q.png" /><figcaption>Cartoon of a developer and a robot in a suit conducting a job performance review with charts and metrics on a clipboard.</figcaption></figure><p>I saw <a href="https://medium.com/@joe.njenga/i-tested-claude-code-insights-it-roasted-my-coding-habits-e38c642e1642">Joe Njenga’s article</a> about Claude Code’s /insights command. I didn&#39;t read the whole thing — I just ran it to see for myself.</p><p>Some background: I’ve been building an AI-powered app solo — TypeScript, React 18, pushing the LLM to its limits. I’ve been at this for months with Claude Code as my daily development partner. I wanted to know if the data confirmed what I thought, or if it caught something I hadn’t noticed.</p><p>The report came back: <strong>91 sessions, 1,704 messages, 444 hours, 111 commits.</strong> Timeframe: Dec 31 to Feb 7.</p><h3>What It Confirmed</h3><p>Out of 50 analyzed sessions, 28 were fully achieved. The three strengths the report called out mapped directly to what I’d documented in <a href="https://medium.com/@able_wong/stop-repeating-yourself-how-i-turned-ai-into-an-engineering-teammate-fe8c2f322629">my previous article</a>.</p><p><strong>Full-cycle delivery.</strong> The report noted that a typical session wasn’t just “implement this feature” — it was implement, test, deploy to staging, verify, create a PR, address review comments, and file follow-up issues. That end-to-end ownership is something I’ve built deliberately. The pre-commit self-review step was specifically called out as a contributor.</p><p><strong>Phased implementation.</strong> Complex features were consistently broken into phases — types first, then services with tests, then API, then UI. Each session built on the previous one with a clean foundation. The report noted that all 2,000+ tests were passing before each phase moved forward. That discipline is in my CLAUDE.md, and the data showed it held.</p><p><strong>Prompt iteration.</strong> The app pushes the LLM hard, and the report noted I don’t accept first drafts. I test against real scenarios, find where the output breaks, and push through multiple correction rounds before committing. 14 sessions were entirely on prompt engineering work.</p><p>The report confirmed what I’d suspected: the upfront investment in documenting engineering culture in CLAUDE.md compounds over time. Having 91 sessions of data confirm that was satisfying.</p><h3>The Friction — And the Honest Part</h3><p>The report identified three friction categories, and all three were real.</p><p><strong>Wrong approach: 27 instances.</strong> Claude misread my intent — placing logic in the wrong architectural layer, applying changes too broadly, misunderstanding what I actually wanted. Some of this is Claude. As I wrote in <a href="https://medium.com/@able_wong/why-your-ai-isnt-making-you-more-productive-and-how-to-fix-it-19a6015afdea">an earlier article</a>, it has a bias toward completing the task with its best guess rather than stopping to confirm.</p><p><strong>Buggy code: 14 instances.</strong> These were cases where the implementation had errors that required corrective back-and-forth to resolve.</p><p><strong>Over-engineering and scope creep: 6 instances.</strong> Claude consistently over-scoped — solving more than what was asked, or going further in the workflow than I intended.</p><p>The honest part is that some of this is on me. After months working with Claude, I got comfortable. I write short prompts because Claude is good at guessing what I mean. But sometimes I write too little — not enough that even a human developer would know exactly what I meant. Claude guesses right most of the time, but when it doesn’t, the correction cycle takes time.</p><p>The scope creep friction is a good example of this. I’d ask Claude to fix something, and it would fix it, create a branch, commit, and merge — when what I wanted was the fix so I could run my own PR review process. The prompt didn’t say “stop after the fix.” Claude filled in the rest. That’s a shared problem: my prompt left the boundary open, and Claude completed what it thought was the logical end of the task.</p><p>I saw the same pattern from a different angle when I <a href="https://medium.com/@able_wong/glm-4-7-surprised-me-the-high-capability-low-cost-engine-for-my-claude-code-cli-ff9aa5f3b8d5">tested GLM-4.7</a> as an alternative model. GLM’s literalism exposed bugs in my commands that Claude was silently fixing with intuition. /insights exposed the same thing in my day-to-day prompts. When you rely on the AI to fill in the gaps, sometimes the gaps are yours.</p><p>That context matters for the next part — because the report flagged something about my workflow that I initially disagreed with.</p><h3>Where I Pushed Back — And Then Reconsidered</h3><p>Most of my work follows a specific pattern. For any meaningful task, I explore the codebase, update the GitHub issue with a plan, close the session, and open a fresh one to implement. That boundary is intentional: the planning session builds shared context and gets the scope right; the implementation session executes cleanly without carrying forward irrelevant context.</p><p>The report saw 8 sessions that ended with only a plan and no code. It called this “planning paralysis” and suggested limiting exploration and getting to code faster.</p><p>My implementation workflow has exploration, clarification, and plan approval built in by design. Those plan-only sessions are the system working correctly for complex issues.</p><p>But I looked at those 8 sessions more carefully. The data was right — both things were true. Some of those sessions were genuinely complex work that needed full scoping before any code was written. Others were simpler tasks that I’d fed into the same heavyweight workflow when a lighter approach would have been fine.</p><p>Insights surfaced the pattern accurately. What it couldn’t tell me was which category each session fell into — that required knowing what those sessions were actually for. A data-driven report still needs interpretation; the numbers alone don’t tell you that.</p><p>The answer wasn’t to plan less. It was to match the effort to the task. Complex features, refactors, and anything touching architecture get the full planning path. Simple bugs, copy changes, and contained cleanup don’t need a plan session. I added a lighter quick path for those cases, so the workflow now has two tracks. The planning sessions that remain are genuinely the ones that need it.</p><p>The metric difference still stands: insights measures code output per session; I measure code I never have to revisit. But the observation had merit, and the right response was more nuanced than “always plan” or “never plan.”</p><h3>What I Changed</h3><p>Two small adjustments came out of this, neither of which required reworking the overall workflow.</p><p><strong>CLAUDE.md addition.</strong> I added one line to Core Principles:</p><pre>Clarify when uncertain — If not confident about what the user means, or if there are<br>multiple probable outcomes, ask before executing</pre><p>This is a safety net for when my prompts are too terse. It covers two cases: genuine uncertainty about what I’m asking, and situations where multiple interpretations are equally valid. Placed next to an existing “Confirm before acting” rule.</p><p><strong>Automated PR step tweak.</strong> I also have a command that handles the full pre-PR process — branch verification, self-review against a security and quality checklist, running lint and tests, and generating the PR description. The summarize step previously read:</p><pre>4. Summarize changes - Present a clear summary of what was implemented</pre><p>It now reads:</p><pre>4. Summarize changes - Review all commits on the branch vs main (not just uncommitted<br>changes) and present a clear summary of what was implemented. Focus on the final outcome,<br>not intermediate fixes or issues encountered during development</pre><p>PR descriptions were picking up intermediate noise — lint fixes, type corrections mid-session — instead of summarizing the actual outcome. Anchoring the summary to the full branch history fixed that.</p><p>The report also suggested features I already had covered. Here’s why I skipped them.</p><h3>What I Didn’t Change — And Why</h3><p>The report suggested three additional features: Hooks, Headless Mode, and Custom Skills.</p><p><strong>Hooks.</strong> The specific suggestion was a post-edit hook to run TypeScript type-checking after every file save. The reasoning was solid — 1,198 TypeScript file edits, with lint errors showing up in PR reviews. But my automated PR process already mandates running lint, type checks, and tests before any commit reaches CI. The same quality gate exists; it just fires at a better checkpoint. Running checks after every individual file edit would be noisy and interruptive, and it wouldn’t catch anything that the PR step doesn’t already catch.</p><p><strong>Headless Mode.</strong> The suggestion was to automate the “deploy to staging, run smoke tests” pipeline. The idea has merit, but it doesn’t fit how testing works in this project. Automated tests — unit tests and purpose-built scripts for major functions — are already things Claude runs directly during development. Post-deployment smoke tests are kept intentionally simple and manual: a quick check after deploying to each environment. There’s not much to automate there, and that’s deliberate.</p><p><strong>Custom Skills.</strong> I already have commands covering the full development lifecycle — from collaborative design discussion and issue filing, through implementation, to PR creation and review comment resolution. The suggestions didn’t add anything that wasn’t already there. If you don’t have an established workflow yet, these features are worth exploring — but they’re not the entry point if you’ve already built one.</p><h3>On the Horizon — But Not Yet</h3><p>The report’s most ambitious suggestion was a fully autonomous issue-to-PR pipeline: an agent that takes a GitHub issue, implements across files, runs the full test suite until green, and opens a PR without human involvement.</p><p>The idea is appealing, and Claude is genuinely capable of complex multi-file work. But for a project that needs long-term development and maintenance, I’m not ready to hand that over fully.</p><p>Claude does a good job of not breaking existing code — it has guardrails and it’s careful. But steering the development still matters. I’d rather review what’s been implemented and verify it directly than find out something was handled incorrectly three features later. The longer something goes unreviewed, the more it compounds. This is a project I intend to maintain, and the more that gets buried without review, the bigger the cleanup later.</p><p>Full autonomy is a direction worth watching. Right now, the right balance for this project is: Claude handles implementation, I handle review and direction.</p><h3>The Takeaway</h3><p>/insights gave me a data-backed mirror of 91 sessions of work. Most of what it showed confirmed the workflow was holding up — the CLAUDE.md investment, the phased implementation approach, the pre-commit self-review. 111 commits across 91 sessions, and the foundation is still clean.</p><p>The biggest surprise wasn’t in the tools — it was in seeing my own friction patterns quantified. Writing prompts that are too short is a habit that’s easy to develop when you’re working with a model that guesses well. The report made that visible.</p><p>The planning session flag was the most useful data point, even though the suggested fix didn’t fit. It made me look honestly at whether every task needed the same treatment. It didn’t, so I added a lighter path for the simple ones. That’s it.</p><p>Run /insights on your own sessions. Apply the same critical judgment to its output that you&#39;d apply to any AI-generated result — because it&#39;s still optimizing for what AI optimizes for. Just make sure the tool&#39;s metric matches yours before acting on what it tells you.</p><p>The workflow behind this has continued to evolve — how the commands were designed, where they broke down in practice, and how I iterated on them is a story worth telling on its own. I’ll cover that in a follow-up article, and I’m also packaging the commands into a project so others can use them as a starting point.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd1023b84c6e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GLM-5 Dropped Before I Could Finish Testing GLM-4.7]]></title>
            <link>https://medium.com/@able_wong/glm-5-dropped-before-i-could-finish-testing-glm-4-7-722a056877ff?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/722a056877ff</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[glm]]></category>
            <category><![CDATA[claude-code]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Thu, 19 Feb 2026 03:50:35 GMT</pubDate>
            <atom:updated>2026-02-19T03:50:35.241Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*PcExSYVn1knH3dosFdxJ6g.png" /></figure><p>Before I could really put GLM-4.7 through its paces as my daily workhorse, Zhipu dropped GLM-5. The benchmarks claimed it matched Claude Opus 4.5 — impressive, even though Opus 4.6 had shipped just days earlier. That was enough to compel me to try it and see how it would affect my workflow.</p><p>The headline specs are substantial: a <strong>744B parameter mixture-of-experts architecture</strong> with 40B active parameters per inference, a <strong>200K context window</strong>, and a <strong>128K max output</strong>. It is objectively close to the top tier, scoring <strong>77.8% on SWE-bench Verified</strong> compared to Opus 4.6’s 80.8%. Perhaps most importantly for agentic work, Zhipu claims a <strong>56 percentage point reduction in hallucinations</strong> over GLM-4.7, all wrapped under the ambitious tagline: <strong>“From Vibe Coding to Agentic Engineering.”</strong></p><p>I wanted to see if that held up in practice using the same codebase and custom commands as <a href="https://medium.com/@able_wong/glm-4-7-surprised-me-the-high-capability-low-cost-engine-for-my-claude-code-cli-ff9aa5f3b8d5">last time</a>, but different tasks. These are real features on a real codebase I’ve been building for four to five months — not synthetic benchmarks or toy projects. It has enough history and conventions that any AI model needs to genuinely understand the codebase to produce good plans.</p><p>To drive the work, I use the same custom slash commands in Claude Code:</p><ul><li>/gh-issue-implement — fetches the issue, plans the implementation, waits for my approval, then implements</li><li>/pr-create — self-reviews the changes and creates a pull request</li><li>/pr-comments-fix — addresses review comments on the PR</li></ul><p>I started with /gh-issue-implement to implement the feature with GLM-5 and Opus 4.6 sessions.</p><h3>The Sluggishness Problem</h3><p>Both GLM-5 and Opus 4.6 completed the task. While the code was fine and tests passed for both, the experience using GLM-5 was painful.</p><p>What takes Opus under five minutes took GLM over ten. Not because the output was worse — because of how it worked. Opus fires off parallel file reads, runs lint and typecheck simultaneously, and batches operations together. GLM did everything one at a time. Read a file. Wait. Read another file. Wait. Run lint. Wait. Run typecheck. Wait.</p><p>Ten minutes doesn’t sound bad on paper. But watching an AI do things sequentially that should be parallel feels like an eternity.</p><h3>Splitting the Command — The “Wrong” Fix</h3><p>My hypothesis: if GLM is slow at the combined plan-then-implement workflow, split it.</p><p>I broke /gh-issue-implement into two separate commands:</p><ul><li>/gh-issue-plan-implementation — just plan, and write the plan to a file.</li><li>/gh-issue-implement-plan — read the plan, load only the files it references, implement.</li></ul><p>The theory was straightforward: focused context loading is genuinely useful. But it didn’t solve the speed problem. Still slow. The real issue was somewhere else.</p><h3>The Real Issue — One Brain, Three Jobs</h3><p>Claude Code doesn’t use a single model. It uses three tiers internally:</p><ul><li><strong>Opus</strong> — main reasoning.</li><li><strong>Sonnet</strong> — subagent tasks (file exploration, search).</li><li><strong>Haiku</strong> — fast lookups and classifications.</li></ul><p>My original configuration only set ANTHROPIC_MODEL, which meant every tier — even simple file searches — was running through the heavy GLM-5 reasoning model. Once I mapped the tiers correctly in my shell alias, the difference was immediate:</p><pre>claude-glm() {<br>  ANTHROPIC_BASE_URL=&quot;https://api.z.ai/api/anthropic&quot; \<br>  ANTHROPIC_AUTH_TOKEN=&quot;YOUR_ZAI_KEY&quot; \<br>  ANTHROPIC_MODEL=&quot;glm-5&quot; \<br>  ANTHROPIC_DEFAULT_OPUS_MODEL=&quot;glm-5&quot; \<br>  ANTHROPIC_DEFAULT_SONNET_MODEL=&quot;glm-4.7&quot; \<br>  ANTHROPIC_DEFAULT_HAIKU_MODEL=&quot;glm-4.7-flashx&quot; \<br>  claude &quot;$@&quot;<br>}</pre><p>To mirror the Opus/Sonnet/Haiku split, I used GLM-5 for reasoning, GLM-4.7 for mid-tier tasks, and GLM-4.7-FlashX for speed. FlashX also allowed me to run up to 3 concurrent sessions without hitting rate limits.</p><p>Once I set this up, implementation speed moved much closer to the native Opus/Sonnet experience. It’s still not a perfect match — GLM-5 thinks longer before each action, and Claude models are specifically optimized for parallel tool use — but the gap shifted from painful to workable.</p><p><strong>More importantly, I sent the generated code from both sessions to a Gemini bot for PR review, and it showed no noticeable difference in quality.</strong></p><h3>Can GLM-5 Plan? — The Head-to-Head</h3><p>The command split from earlier created separate planning and implementation phases. GLM-5 executes well, but can it handle the planning phase too, or is that Opus-only territory?</p><p>I tested this with a real feature: a feedback collection system. It required a thumbs up/down rating, an optional freeform comment via modal, and database storage, plus an admin view. It was simple enough for both models to handle, but complex enough to require respect for existing codebase conventions.</p><p>I used the /gh-issue-plan-implementation command to fetch the requirements for both models. Both completed the phase, but the speed gap remained: Opus finished in 5 minutes, while GLM-5 took about 9.</p><p>To evaluate the results, I did something different: <strong>I asked both models to cross-evaluate each other’s plans.</strong> Interestingly, they reached a similar consensus on their own strengths and weaknesses.</p><p><strong>Opus was stronger on structural decisions.</strong> It chose a database structure that allowed the admin view to query all feedback in one call. GLM’s approach required two separate queries merged together — a complexity the plan didn’t even mention. Opus also spotted a key piece of state that clears after a workflow completes and planned around it; GLM referenced that same variable, which would have been null at runtime. This confirms why I didn’t push GLM-4.7 into this tier — it simply wasn’t up to this level of architectural reasoning.</p><p><strong>GLM-5 was stronger on thoroughness.</strong> The issue had an unanswered question about data retention; GLM answered it with a concrete policy while Opus ignored it. GLM wrote different modal copy for positive vs. negative feedback, added a skip option, and defined constants for character limits. Opus kept the UI generic. GLM even added pagination for the admin listing, whereas Opus hardcoded a limit.</p><p>The pattern is the same as my previous post, but even more defined with GLM-5: <strong>Opus thinks like an architect</strong>, catching structural problems before they hit, while <strong>GLM-5 thinks like a thorough implementer</strong>, covering the polish and edge cases Opus tends to skip.</p><p>Since I had already concluded from research and early tests that GLM-4.7 couldn’t handle this level of planning, seeing GLM-5 go toe-to-toe with Opus — even if it missed some architectural nuances — is a significant step up.</p><h3>The Same Formula, Validated</h3><p>This head-to-head was a single test case, not a lab-grade benchmark. But I’ve since moved GLM-5 into my permanent daily workflow, and the results are holding up.</p><p>The hybrid formula — <strong>Opus plans, GLM executes</strong> — remains the sweet spot. Opus acts as the lead architect, catching the structural bugs and state issues that would otherwise derail a project. GLM-5 then handles the heavy lifting of implementation with a level of thoroughness and reliability that the previous generation occasionally lacked.</p><p>My workflow is now a true pipeline. I execute several GLM sessions in the background while planning the next feature with Opus in a separate window. By leveraging the concurrent session limits of the different GLM models, I’ve eliminated idle time. It’s no longer just an experiment or a “vibe”; it’s a stable, agentic engine that finally works exactly as I envisioned.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=722a056877ff" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GLM-4.7 Surprised Me: The High-Capability, Low-Cost Engine for my Claude Code CLI]]></title>
            <link>https://medium.com/@able_wong/glm-4-7-surprised-me-the-high-capability-low-cost-engine-for-my-claude-code-cli-ff9aa5f3b8d5?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff9aa5f3b8d5</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[glm]]></category>
            <category><![CDATA[ai-assisted-coding]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Thu, 29 Jan 2026 02:31:01 GMT</pubDate>
            <atom:updated>2026-01-29T02:31:01.755Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*rSIfbXTJWtKWF3TukYdvzQ.png" /></figure><h3>The 2.5-Hour Wall</h3><p>We’ve all been there. You’re in the flow with the Claude Code CLI. Features are shipping, bugs are getting fixed, and for a moment, the friction between thought and code disappears.</p><p>Then, you hit the wall.</p><p>It starts with the “Rate Limit Reached” message. When you’re using AI agents to write code, a limit reached notification stops your work cold. It’s time to upgrade to the Max plan (Anthropic’s $100/month tier) to keep things moving. I’m in the zone and want to finish what I’m working on. I hit the limit often enough that it’s finally time to upgrade.</p><h3>My Workflow — Architect, Not Typist</h3><p>Before diving into the tests, it’s important to clarify how I actually use these tools. I’ve been on the Claude Max plan for months, and in that time, my role has fundamentally shifted. I no longer write code line-by-line. Instead, I operate as the Product Manager and System Architect for my startup.</p><p>My process is an autonomous loop:</p><p><strong>Define</strong>: I work with Claude Code to define the “What” and the “Why” — handling high-level planning, data modeling, and UI patterns.</p><p><strong>Direct</strong>: I act as the Lead Engineer, setting technical constraints and reviewing implementation plans.</p><p><strong>Execute &amp; Verify</strong>: The AI implements the code, reviews its own work, runs automated tests, commits the changes, and creates the PR.</p><p>When evaluating a new model like GLM-4.7, I only care about one thing: Can this model handle my workflow? If it requires me to step back into the editor to “fix” things or manually bridge gaps, it fails the test. If it can’t keep up, I’ll stick with Claude and continue paying for the Max plan.</p><h3>The Discovery — Exploring the Alternatives</h3><p>Meanwhile, I had already been improving my workflow’s structural problems. I used to have a big user-level CLAUDE.md (stored in ~/.claude/CLAUDE.md) that held all instructions for my workflow. I de-cluttered it by offloading frequently used steps into custom slash commands like /gh-issue-implement, /pr-create, and /pr-comments-fix. You can tell what they are doing by their names.</p><p>With my workflow streamlined, I was ready to explore new tools. I found a Medium article talking about OpenCode + GLM that seemed promising. I had researched various models half a year ago — Gemma, Phi-4, DeepSeek, and Qwen — but I wasn’t aware of GLM at the time. I decided to investigate further to see if it could replace my development workflow. I don’t trust benchmarks too much; if the results for a particular model are super low, I’ll skip it, but the main thing is whether it can actually complete the tasks I throw at it.</p><p>From the article and my own research, I learned that OpenCode is an interesting project — a CLI tool similar to Claude Code focusing on an open ecosystem. I also discovered that GLM had a major release in December 2025 (GLM-4.7) that is remarkably capable. While it may not surpass Claude 4.5, it is close enough to be a serious contender for agentic tasks.</p><p>Originally, I thought about jumping ship to OpenCode, but I was hesitant. There are slight UI differences, and more importantly, I didn’t want to pay the “porting tax” of moving the customizations I had built for the Claude Code CLI — tools I rely on heavily. Then, I realized I didn’t have to switch the CLI at all. I could just swap the “brain” by pointing Claude Code to a GLM Anthropic-compatible endpoint. I realized then that I had a very interesting experiment to do.</p><h3>The First Surprise — It Speaks Chinese</h3><p>Before letting this model touch my codebase, I wanted to verify the connection manually. I signed up at Z.ai and used curl to check the endpoint.</p><pre>curl --location &#39;https://api.z.ai/api/anthropic/v1/messages&#39; \<br>--header &#39;x-api-key: YOUR_ZAI_API_KEY&#39; \<br>--header &#39;anthropic-version: 2023-06-01&#39; \<br>--header &#39;content-type: application/json&#39; \<br>--data &#39;{<br>    &quot;model&quot;: &quot;glm-4.7&quot;,<br>    &quot;max_tokens&quot;: 1024,<br>    &quot;messages&quot;: [<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello&quot;}<br>    ]<br>}&#39;</pre><p>Response:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*W-qmX366A-h90CxpzaRKVQ.png" /><figcaption>Response in Chinese</figcaption></figure><p>I got my first surprise. The response came back in Chinese.</p><p>I discovered Z.ai also offers an OpenAI-compatible endpoint. With a system prompt, it responds in English:</p><pre>curl --location &#39;https://api.z.ai/api/coding/paas/v4/chat/completions&#39; \<br>--header &#39;x-api-key: YOUR_ZAI_API_KEY&#39; \<br>--header &#39;content-type: application/json&#39; \<br>--data &#39;{<br>    &quot;model&quot;: &quot;glm-4.7&quot;,<br>    &quot;messages&quot;: [<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a coding assistant. You MUST always respond in English.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello&quot;}<br>    ]<br>}&#39;</pre><p>Response:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*79bW0NC_p1VdZrFX3a1Jvg.png" /><figcaption>Response in English</figcaption></figure><p>Much better. I added a system prompt to my Claude Code configuration to ensure GLM always responds in English.</p><h3>Pointing Claude Code to GLM</h3><p>After verifying the endpoint, I needed a way to redirect the Claude Code CLI. While I could have messed with the global ~/.claude/settings.json, I wanted something more flexible. I didn&#39;t want to lose the ability to use Claude 4.5 for high-level reviews.</p><p>I finally settled on a simple .zshrc alias. This allows me to start a &quot;GLM-powered&quot; session whenever I want, without changing my global configuration.</p><pre># GLM-4.7 via Claude Code CLI<br>claude-glm() {<br>  ANTHROPIC_BASE_URL=&quot;https://api.z.ai/api/anthropic&quot; \<br>  ANTHROPIC_AUTH_TOKEN=&quot;your_zai_key_here&quot; \<br>  ANTHROPIC_MODEL=&quot;glm-4.7&quot; \<br>  claude &quot;$@&quot;<br>}</pre><p>Now, if I have a tedious refactor to do, I just run claude-glm. For everything else, I run claude normally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*IBn39WCRgaUto6cAQLImVQ.png" /><figcaption>Claude /status output showing glm-4.7 as model</figcaption></figure><h3>The Planning Test — Speed vs. Results</h3><p>To see how GLM-4.7 stacks up, I ran a head-to-head test on a planning task. I wanted to move the “Theme Selector” out of the navbar to a less prominent spot.</p><p>Both models were given the same screenshot and prompt: “The theme selector was inherited from my placeholder project… explore the codebase and suggest different options.”</p><p>I have filing guidelines in my CLAUDE.md that both models should follow:</p><p><strong><em>Filing Issues</em></strong><em> When refining issues with user:</em></p><ul><li><strong>PM lens</strong>: clarify problem, who’s affected, success criteria</li><li><strong>Research</strong>: existing packages, common patterns, similar solutions in codebase</li><li><strong>Engineer lens</strong>: feasibility, tradeoffs, architectural fit</li><li><strong>Draft implementation plan</strong> following Feature Development guidelines</li></ul><h4>The Results</h4><p><strong>Claude 4.5 (time to first draft: ~1m 39s):</strong> It followed the guidelines closely. It explored the files and researched UI patterns from apps like GitHub and Notion. It gave me a table of four options with a recommended winner. After two quick rounds of feedback, I had a final draft ready to file.</p><p><strong>GLM-4.7 (time to first draft: 3+ minutes, 6+ turns):</strong> It was slower and didn’t follow the guidelines as effectively. Instead of researching patterns or presenting a plan, it spent several turns asking me clarifying questions about user behavior. It didn’t provide a concrete draft until I provided explicit guidance.</p><h4>The Takeaway</h4><p>Both models eventually finished the task, but the “interaction cost” was different.</p><p>Claude intuited the goal: It jumped straight to researching patterns and gave me a draft to react to. This meant less typing for me and a faster result.</p><p>GLM-4.7 was more literal: It required more explicit hand-holding for every step. It didn’t realize it could provide a draft first and then ask questions, resulting in a longer discussion loop.</p><p>At this stage, a clear distinction emerged: Claude optimizes for fewer turns, while GLM-4.7 asks clarifying questions first. The question was whether GLM’s methodical nature would be a hindrance or a hidden strength once we moved from planning to the implementation grind.</p><h3>The Implementation Stress Test — Discipline over Intuition</h3><p>I assigned GLM-4.7 two real-world tasks in my startup’s codebase. The goal was to see if my existing custom slash commands — originally built for Claude — would still work with GLM as the “brain.”</p><p>The command forces a rigid, professional software development lifecycle:</p><ol><li><strong>Fetch issue</strong> using gh issue view $ARGUMENTS</li><li><strong>Explore codebase</strong> for existing patterns and related code</li><li><strong>Research</strong> (as a <strong>staff/principal engineer</strong>) any unfamiliar technologies or approaches if needed</li><li><strong>Ask clarifications</strong> if requirements or solutions are ambiguous<br>- As a <strong>product manager</strong>: clarify business logic, user requirements, acceptance criteria<br>- As a <strong>staff/principal engineer</strong>: clarify technical approach, architecture decisions, edge cases</li><li><strong>Present implementation plan</strong> and wait for user approval</li><li><strong>Implement</strong> after approval</li></ol><h4>The Results</h4><p><strong>Task 1: The Routine Refactor (Issue #420)</strong></p><p>I needed to consolidate UI timing constants scattered across 9 files into a central uiConstants.ts.</p><p>The Outcome: GLM followed the instructions to the letter. It identified every instance, found a “bonus” constant I’d forgotten to mention, and updated all imports.</p><p>The Proof: It ran the full test suite (2,012 tests), passed them, committed the changes, and created the PR autonomously.</p><p><strong>Task 2: The Complex Feature (Issue #422)</strong></p><p>I tasked GLM with building a new /settings page and moving the Theme Selector there. This involved new React components, routing, and state management.</p><p>The Snag: We hit a CSS bug where a button wouldn’t highlight correctly — a known regression in DaisyUI v5.</p><p>The Pivot: GLM tried local fixes first. I had to explicitly nudge it: “Stop guessing, use the web search tool.” Once directed, it found the external bug, bypassed it, and finished the feature.</p><p>The verdict: GLM-4.7 handled both tasks successfully, requiring only slightly more guidance than Claude typically does.</p><h3>The Paperwork — How GLM Exposed My Workflow Bug</h3><p>Once the code is written and the tests pass, the final step is documentation. In my workflow, the AI is responsible for checking its own work and drafting the GitHub Pull Request.</p><p>The command for this is /pr-create. But this section isn’t just about how the command works — it’s about how GLM exposed a flaw I didn’t know I had.</p><h4>The Original (Flawed) Command</h4><p>My original /pr-create command had a simple branch-checking step:</p><ol><li>Check branch status:<br>- If on main/master: create a new feature branch, then commit changes<br>- If on a feature branch: verify it’s for current work, then commit any uncommitted changes<br>- If leftover branch: switch back to main, create new branch, then commit changes</li><li>Run validation checks (typecheck, lint, test)</li><li>List new code</li><li>Self-review changes</li><li>Summarize and create PR</li></ol><h4>The Problem</h4><p>When I ran /pr-create with GLM-4.7, it followed the instructions literally: created a new feature branch, immediately committed ALL changes (as instructed), then ran validation checks and reviewed after.</p><p>I had to tell it: “Unstage all changes, please.” The workflow was fundamentally broken — I had written “commit before review,” and GLM followed it exactly.</p><h4>Why I Never Noticed</h4><p>Claude never followed this literally. It intuitively knew reviewing code before committing makes sense — it would check the branch, run validations, review everything while still unstaged, wait for confirmation, and only then commit.</p><p>I wasn’t aware my command had this bug because Claude was silently “fixing” it with common sense. GLM exposed the flaw I had written into my own workflow.</p><h4>The Fix</h4><p>I rewrote the command to fix the order and make each step explicit:</p><ol><li>Check branch status: Verify you’re on a clean feature branch. Create one if needed. Do NOT commit.</li><li>Run validation checks: Execute typecheck, lint, and test (skip tests if only docs changed).</li><li>List new code: Provide org chart of new classes, interfaces, functions.</li><li>Self-review: Audit changes for Security, Design, Quality, Performance, Test Coverage.</li><li>Summarize: Present summary and manual testing checklist.</li><li>Wait for confirmation: Do NOT commit until user approves.</li><li>After approval: Commit changes, then create PR.</li></ol><h4>The Result</h4><p>GLM-4.7 handled the corrected workflow perfectly. It ran validations, reviewed the code, produced organizational charts, and filed the PR autonomously after I gave confirmation.</p><h4>The Takeaway</h4><p>GLM’s literalism isn’t a bug — it’s a forcing function. By following my instructions exactly, it exposed workflow flaws that Claude’s intuition was silently papering over. Patching these gaps for GLM made the entire system more reliable for any model I use.</p><h3>The Feedback Loop — When GLM Gets Too Literal</h3><p>The next test involved the post-submission cycle. I have a Gemini-powered bot installed in GitHub that performs a code review for every pull request. After GLM created the PR for Issue #422, the bot left several review comments.</p><p>I used my /pr-comments-fix command to address the feedback. This is where I encountered a critical difference in how GLM-4.7 processes instructions compared to Claude.</p><h4>The Conflict: DaisyUI v5 vs. The Review Bot</h4><p>The bot’s comment: <em>“Use </em><em>btn-active with global CSS fix for consistency.&quot;</em></p><p>This wasn’t just a subjective opinion — the bot was suggesting a known broken approach. GLM and I had already spent an hour debugging btn-active styling issues during implementation and confirmed that DaisyUI v5 has a documented btn-active regression (#3652) where this class renders incorrectly. We had implemented a deliberate workaround using btn-primary directly.</p><p><strong>How GLM-4.7 handled it:</strong> It took the bot’s suggestion at face value. It didn’t evaluate whether the comment was technically valid or remember our hour of failed debugging. It simply deleted the working workaround and re-applied the broken approach.</p><p><strong>The contrast with Claude:</strong> Claude typically makes the right assumption most of the time. It usually recognizes when a new suggestion contradicts the history of the current session and avoids breaking a working fix.</p><h4>Identifying the Weakness: The Implicit vs. The Explicit</h4><p>I realized my original instructions for handling comments were too loose for a model that doesn’t rely on implicit context.</p><p><strong>The Original (Implicit) Instruction:</strong></p><p><em>For each comment:</em></p><ul><li><strong>Fix</strong> security risks and major maintainability issues, then commit</li><li><strong>Skip</strong> style preferences, trivial refactors, subjective opinions</li></ul><p>This worked for Claude because Claude filters for logic by default. For GLM-4.7, it was just a command to “fix” whatever the bot pointed out.</p><p><strong>The Correction: Making it Explicit</strong></p><p>I updated the command to provide a point-by-point workflow for every scenario:</p><p><em>For each comment:</em></p><ul><li><strong>Evaluate first:</strong> What is being asked? Is it valid? Have we tried this? Are there known issues?</li><li><strong>Based on evaluation:<br>- </strong>If known to fail → <strong>Reject</strong> with explanation: “Already tried X, doesn’t work because Y”<br>- If risky (UI, database, logic changes) → Present concern to user, let them decide<br>- If safe and valid → Fix, verify it works, then commit</li><li><strong>Skip</strong> style preferences, trivial refactors, subjective opinions</li></ul><h4>The Verdict on Context</h4><p>GLM-4.7 is a “literalist” engine. While Claude can figure out the implicit meaning based on the project’s history, GLM-4.7 requires you to define the decision tree. It won’t protect you from a bad suggestion unless you explicitly instruct it to evaluate the suggestion before executing it.</p><blockquote><em>📌 </em><strong><em>Note:</em></strong><em> I’ve updated the command with this evaluation-first approach, but I haven’t used it on a real PR yet. The next review cycle will test if this enhancement prevents the “follow then revert” pattern.</em></blockquote><h3>The Verdict — The Orchestration Strategy</h3><p>After stress-testing GLM-4.7 across my entire development lifecycle, the verdict is clear: The Orchestrator workflow is model-agnostic. By treating my CLAUDE.md and custom slash commands as a model-agnostic operating manual rather than model-specific prompts, I’ve unlocked a level of flexibility that saves both time and money.</p><p><strong>What Works</strong></p><ul><li>✅ <strong>Cross-Model Command Portability:</strong> The commands work seamlessly across both models. I applied a couple of trivial fixes to two commands (/pr-create and /pr-comments-fix) to handle GLM&#39;s literalism—adding explicit evaluation steps and correcting workflow order. The final commands are now robust enough to be consumed by both GLM and Claude without further modification.</li><li>✅ <strong>Literalism as a Quality Control:</strong> GLM’s literal nature acts as a “forcing function.” It exposed flaws in my workflow that Claude was silently “fixing” with its high intuition. Patching these gaps for GLM made the entire system more reliable for any model I use.</li><li>✅ <strong>High-Speed Hybridity:</strong> I can now use the right tool for the specific phase of the job without sacrificing the quality of the final output.</li></ul><h4>The Specialist Stack</h4><p>I’ve settled on a hybrid stack where each model plays to its inherent strengths:</p><ul><li><strong>Architecture &amp; Planning:</strong> Claude 4.5 Opus. For high-level brainstorming and initial system design, Opus remains the gold standard. It’s faster at “getting” the big picture.</li><li><strong>Implementation (All Types):</strong> GLM-4.7 Pro. Whether the task is a simple refactor or a complex multi-file feature, GLM is my daily workhorse. It is disciplined and handles the “grunt work” with high reliability.</li><li><strong>Final Review:</strong> Flexible Choice. For routine PRs, I stick with GLM. However, for anything complex or architecturally sensitive, I swap in Claude 4.5 Opus for a “Senior Architect” review to catch subtle logical errors.</li></ul><h3>The Math — The 50% Efficiency Dividend</h3><p>The real-world impact of this hybrid setup is most visible in the monthly burn. By moving my heavy implementation hours to GLM-4.7 while keeping Claude for high-level oversight, I’ve slashed my AI overhead significantly.</p><p><strong>Cost Breakdown (Quarterly)</strong></p><ul><li><strong>Claude Pro</strong> — Architecture &amp; oversight: $20/month ($60/quarter)</li><li><strong>Claude Max</strong> — The “all-in-one” I avoided: $100/month ($300/quarter)</li><li><strong>GLM-4.7 Pro (subscription)</strong> — Daily implementation: $30/month ($90/quarter)</li><li><strong>The Hybrid Setup</strong> — GLM Pro + Claude Pro: $50/month ($150/quarter)</li></ul><p><strong>The Savings</strong></p><p>Versus Claude Max-only: $150 saved per quarter. That’s a 50% reduction.</p><p><strong>The Result:</strong> I’m getting Max-quality architectural thinking and high-volume implementation capacity for half the price of a single high-tier subscription.</p><h3>Final Thoughts — How I Work Differently</h3><p>After months of relying on Claude, shifting to GLM-4.7 as my primary workhorse has fundamentally changed how I work. It’s not just about the money; it’s about the shift from Inference to Instruction.</p><h4>The Shift in Communication</h4><p><strong>With Claude:</strong> I mention an observation → Claude “reads my mind” → It takes appropriate action based on inferred intent.</p><p><strong>With GLM:</strong> I mention an observation → I need to explicitly state what’s wrong and what to do → It executes with precision.</p><h4>The Trade-off: Mind-Reading vs. Predictability</h4><p>AspectClaudeGLM-4.7<strong>Efficiency</strong>Less typing, high intuitionMore typing, high discipline<strong>Clarity</strong>Sometimes ambiguous intentLiteral and highly predictable<strong>Mental Model</strong>”It just gets me””I need to tell it what I want”</p><p><strong>Is the extra typing a problem?</strong></p><p>Not really. In fact, being forced to be explicit has made me a better developer. It reduces miscommunication and ensures that the “Why” is documented in the chat history. It’s a small trade-off for a 50% cost saving and a more predictable development cycle.</p><p><strong>The Bottom Line:</strong> The work gets done. The code ships. The tests pass. GLM-4.7 has proven that it isn’t just a “budget alternative” — it’s a professional-grade implementation engine that, when paired with the architectural brilliance of Claude, creates the most efficient dev environment I’ve ever used.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff9aa5f3b8d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Conversational AI Part 7: Supercharge Your RAG Chatbot with Tool Calling]]></title>
            <link>https://blog.gopenai.com/understanding-conversational-ai-part-7-supercharge-your-rag-chatbot-with-tool-calling-ebe3b3db2c68?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/ebe3b3db2c68</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[open-webui]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Sun, 25 Jan 2026 17:33:18 GMT</pubDate>
            <atom:updated>2026-01-27T06:48:58.995Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*popNC2Z3LOSiCWcLA257Dw.png" /><figcaption>Cartoon illustration of an AI chatbot planning and executing tool calls in a loop.</figcaption></figure><blockquote><strong><em>TL;DR<br></em></strong>- RAG retrieves knowledge, but it can’t <em>act<br>- </em>Tool calling lets an LLM execute functions and APIs<br>- With an iterative loop, the LLM can plan and chain tools to solve multi-step tasks</blockquote><p>In Part 6, I covered hybrid search and metadata filtering to improve the relevance of retrieved documents. With that foundation in place, I’m now ready to add the final major capability: <strong>tool calling</strong>.</p><p>Tool calling transforms the chatbot from a passive knowledge assistant into an <strong>intelligent agent</strong> that can take action in the real world.</p><p>This article is part of my series, <strong>“Understanding Conversational AI: Ingestion, Retrieval, Tools, and Beyond.”</strong> You can find all the articles here:</p><ol><li><a href="https://medium.com/@able_wong/getting-started-with-local-ai-ollama-open-webui-part-1-bcaafcce6df7"><strong>Getting Started with Local AI: Ollama &amp; Open WebUI</strong></a> — Your beginner’s guide to running LLMs locally with Ollama and Open WebUI.</li><li><a href="https://medium.com/@able_wong/getting-started-with-local-ai-open-webui-documents-and-tools-part-2-5f8f9c67a414"><strong>Getting Started with Local AI: Open WebUI Documents and Tools</strong></a> — An introduction to the foundational concepts of RAG and tool usage in Open WebUI.</li><li><a href="https://medium.com/@able_wong/local-ai-ingestion-pipeline-building-from-scratch-part-3-0d06adb52c35"><strong>Local AI Ingestion Pipeline: Building from Scratch</strong></a> — A deep dive into the architecture and components of a document ingestion pipeline.</li><li><a href="https://medium.com/@able_wong/part-4-building-a-rag-cli-chatbot-from-scratch-with-ollama-gemini-and-qdrant-ba759c7b2ecf"><strong>Building a RAG CLI Chatbot from Scratch (with Ollama, Gemini, and Qdrant)</strong></a> — Explaining the design and flow of a RAG-enabled chatbot from a conceptual perspective.</li><li><a href="https://medium.com/@able_wong/part-5-advanced-rag-techniques-llm-based-query-rewriting-and-hyde-dbcadb2f20d1"><strong>Advanced RAG Techniques: LLM-Based Query Rewriting and HyDE</strong></a> — An exploration of advanced methods like query rewriting and HyDE for improving retrieval accuracy.</li><li><a href="https://medium.com/@able_wong/power-up-rag-chatbot-with-hybrid-search-and-filtering-47386bade934"><strong>Power up RAG Chatbot with Hybrid Search and Filtering</strong></a> — A look into how hybrid search and metadata filtering enhance the relevance of your RAG system’s results.</li><li><strong>Supercharge Your RAG Chatbot with Tool Calling</strong> — Giving the chatbot the ability to use external tools. ← You are here</li></ol><p>Code for this part is available on GitHub under the part7-tool-calling tag:</p><ul><li><a href="https://github.com/able-wong/rag-cli-chatbot">https://github.com/able-wong/rag-cli-chatbot</a></li></ul><h3>Introduction: From Knowledge Retrieval to Intelligent Actions</h3><p>In my <a href="https://medium.com/gopenai/power-up-rag-chatbot-with-hybrid-search-and-filtering-47386bade934">previous article</a>, we built a powerful RAG chatbot with hybrid search and smart filtering. The chatbot could retrieve and synthesize information from a knowledge base effectively.</p><p>But there was a fundamental limitation: <strong><em>It could only answer questions based on existing knowledge.</em></strong></p><p>It couldn’t <em>act</em> on information or interact with external systems.</p><h4>The Real-World Problem</h4><p>Consider these real-world queries:</p><ol><li><strong>“What’s the weather like in Toronto right now?”</strong><br> → Requires calling a weather API</li><li><strong>“Book a meeting with the team for next Tuesday at 2pm”</strong><br> → Needs calendar integration</li><li><strong>“Calculate the compound interest if I invest $50k at 7% for 10 years”</strong><br> → Requires executing calculations</li><li><strong>“What’s the current exchange rate for USD to EUR?”</strong><br> → Requires calling a currency API</li></ol><p>These tasks go beyond retrieval. They require the chatbot to <strong>execute functions</strong>, <strong>call APIs</strong>, and <strong>perform computations</strong>.</p><p>This is where <strong>tool calling</strong> (also known as function calling) comes in.</p><h3>Why OpenWebUI Tool Format?</h3><p>When I set out to add tool calling to my RAG chatbot, I had several options:</p><ul><li>Build a custom tool system from scratch</li><li>Use LangChain’s abstractions</li><li>Adopt OpenWebUI’s tool format</li></ul><h4>Why Understanding the Fundamentals Matters</h4><p>This article focuses on tool calling as implemented <strong>before protocols like MCP (Model Context Protocol)</strong> became widely available.</p><p>By understanding:</p><ul><li>how tool schemas are injected into prompts,</li><li>how LLMs respond with structured tool calls,</li><li>and how the iterative execution loop works,</li></ul><p>you gain insight into what higher-level frameworks and protocols are doing <strong>under the hood</strong>. That foundational understanding is invaluable when debugging, optimizing, or building custom AI systems.</p><p><strong>Why I Chose OpenWebUI’s Format</strong> When I was looking for a way to add tools, I didn’t want to spend weeks learning a whole new system or a complicated way of writing functions. I chose OpenWebUI’s standard because:</p><ul><li><strong>It’s Just Python:</strong> I didn’t have to learn a new, complex system. If you can write a Python class, you can write a tool.</li><li><strong>Don’t Reinvent the Wheel:</strong> The <a href="https://openwebui.com/tools">OpenWebUI Tools Gallery</a> already had hundreds of tools. I could just look at how others built a web search or a calculator and go from there.</li><li><strong>Docstrings are the “UI”:</strong> I loved the idea that the LLM reads my comments to understand the code. It keeps the “logic” and the “instructions” in the same place.</li></ul><p>Here’s what an OpenWebUI-compatible tool looks like:</p><pre>from datetime import datetime<br>from zoneinfo import ZoneInfo<br>from pydantic import BaseModel, Field<br><br>class Tools:<br>    &quot;&quot;&quot;OpenWebUI-compatible tool class. Each public method becomes a callable tool.&quot;&quot;&quot;<br>    def __init__(self):<br>        self.valves = self.Valves()<br>    class Valves(BaseModel):<br>        &quot;&quot;&quot;Configuration for the tool (can be empty)&quot;&quot;&quot;<br>        priority: int = Field(default=0)<br>    def get_current_time(self) -&gt; str:<br>        &quot;&quot;&quot;<br>        Get the current time.<br>        :return: The current time as a string.<br>        &quot;&quot;&quot;<br>        return f&quot;Current time: {datetime.now().strftime(&#39;%H:%M:%S&#39;)}&quot;<br>    def get_time_in_timezone(self, timezone: str) -&gt; str:<br>        &quot;&quot;&quot;<br>        Get the current time in a specific timezone.<br>        :param timezone: Timezone name (e.g., &#39;America/Toronto&#39;, &#39;Europe/London&#39;, &#39;UTC&#39;)<br>        :return: The current time in the specified timezone.<br>        &quot;&quot;&quot;<br>        tz = ZoneInfo(timezone)<br>        now = datetime.now(tz)<br>        return f&quot;Current time in {timezone}: {now.strftime(&#39;%H:%M:%S %Z&#39;)}&quot;</pre><p><strong>Key elements:</strong></p><ul><li>A Tools class containing callable methods</li><li>Optional Valves configuration</li><li>Clear docstrings (the LLM reads these!)</li><li>Type hints for parameters</li></ul><h3>How Tool Calling Works</h3><p>Let’s start with a simple example. When a user asks: <strong><em>“What time is it?”</em></strong></p><p>this is the high-level flow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wJyfvhyy-kuCbT-uzfIJCA.png" /></figure><h4>The Key Steps</h4><ol><li><strong>Tool discovery</strong> — the chatbot loads tools from a directory</li><li><strong>Schema conversion</strong> — Python methods become JSON schemas</li><li><strong>LLM decision</strong> — the LLM chooses which tool to call</li><li><strong>Execution</strong> — the chatbot executes the tool</li><li><strong>Response</strong> — the LLM turns the result into natural language</li></ol><h4>A Simple Mental Model</h4><p>A helpful way to think about tool calling is this: <strong><em>Tool calling turns the LLM into a lightweight planner.</em></strong></p><p>Each loop answers one question: “What do I need next to answer the user?”</p><ul><li>If the answer is <em>information</em>, the LLM calls a tool</li><li>If it has enough context, it stops and responds</li></ul><p>This <strong>plan → execute → observe → repeat</strong> loop is the foundation behind most modern AI agents.</p><h4>What the LLM Sees</h4><p>The chatbot converts Python methods into JSON schemas the LLM can understand.</p><p>For a simple method like get_current_time():</p><pre>{<br>    &quot;type&quot;: &quot;function&quot;,<br>    &quot;function&quot;: {<br>        &quot;name&quot;: &quot;get_current_time&quot;,<br>        &quot;description&quot;: &quot;Get the current time.&quot;,<br>        &quot;parameters&quot;: {<br>            &quot;type&quot;: &quot;object&quot;,<br>            &quot;properties&quot;: {},<br>            &quot;required&quot;: []<br>        }<br>    }<br>}</pre><p>Parameterized tools follow the same structure. The difference is that:</p><ul><li>parameters are inferred from type hints</li><li>descriptions come from docstrings and :param lines</li></ul><p><strong>The Secret: Your Comments are Part of the Code</strong> Here’s the most important thing to remember: The LLM never actually sees your Python code. It only sees a “description” of it.</p><p>When the chatbot loads your tool, it looks at your docstrings (the comments at the top) and turns them into a JSON schema — basically a set of instructions for the AI. This means <strong>your writing is just as important as your code.</strong> If your description is vague, the LLM will get confused about when or how to use the tool.</p><h4>What the LLM Returns</h4><p>When the LLM decides to use a tool, it responds with a structured request:</p><pre>{<br>    &quot;text&quot;: &quot;&quot;,<br>    &quot;tool_calls&quot;: [<br>        {<br>            &quot;function&quot;: {<br>                &quot;name&quot;: &quot;get_current_time&quot;,<br>                &quot;arguments&quot;: {}<br>            }<br>        }<br>    ]<br>}</pre><p>The chatbot executes the tool and adds the result back into the conversation.</p><h4>The Conversation Protocol (Simplified)</h4><p>At a high level, the message flow looks like this:</p><ul><li><strong>assistant</strong> → requests a tool</li><li><strong>tool</strong> → returns a result</li><li><strong>assistant</strong> → continues reasoning or answers</li></ul><p>This pattern repeats until the LLM stops requesting tools.</p><h4>Terminal Output</h4><pre>You: What time is it?<br>🤖 Thinking...<br>🔧 Calling tools:<br>  • get_current_time()<br>  ✓ get_current_time: Current time: 13:23:45<br>🤖 Getting response from LLM...<br>💬 A: The current time is 13:23:45.</pre><h3>Tool Chaining: Planning in Steps</h3><p>Tool calling becomes truly powerful when the LLM needs to <strong>chain multiple tools</strong>.</p><p>Consider: <strong><em>“What time is it in Toronto?”</em></strong></p><p>This requires planning:</p><ol><li>Find Toronto’s timezone</li><li>Get the current time in that timezone</li></ol><p>No single tool can do this alone.</p><h4>How the LLM Extracts Parameters</h4><p>When the user asks “What time is it in Toronto?”, the LLM:</p><ol><li>Recognizes the entity “Toronto”</li><li>Sees a tool that requires a city parameter</li><li>Extracts &quot;Toronto&quot; from the question</li><li>Calls the tool with structured arguments</li></ol><pre>{<br>    &quot;tool_calls&quot;: [<br>        {<br>            &quot;function&quot;: {<br>                &quot;name&quot;: &quot;get_timezone_for_city&quot;,<br>                &quot;arguments&quot;: { &quot;city&quot;: &quot;Toronto&quot; }<br>            }<br>        }<br>    ]<br>}</pre><h4>The Chaining Flow</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iy653VDnIDaieZSVU6ndpg.png" /></figure><h4>The Iterative Loop</h4><p><strong>The Loop: How the Chatbot “Thinks”</strong> Tool calling turns the LLM into a planner. Each time the chatbot loops, it asks the model: <em>“Do you have enough info to answer the user now, or do you need to call another tool?”</em></p><p>It keeps going until the LLM is confident it has the answer. I added a max_iterations guardrail (set to 5) just in case the model gets stuck in a loop—kind of like an &quot;emergency stop&quot; for the AI.</p><pre>max_iterations = 5  # Prevent infinite loops<br>iteration = 0<br><br>while response.get(&quot;tool_calls&quot;) and iteration &lt; max_iterations:<br>    iteration += 1<br>    # Execute tools, add results to conversation history<br>    response = self.llm_client.get_llm_response(<br>        messages=self.conversation_history,<br>        tools=self.tools<br>    )<br># Loop exits when LLM returns text without tool_calls</pre><p>This is where the chatbot stops being a simple search bar and starts acting like an <strong>intelligent agent</strong>. By wrapping the LLM in this loop, we’ve effectively given it the ability to plan, execute, and observe its own actions. Instead of the user having to figure out the steps, the LLM looks at the goal, selects the right tool, and then analyzes the results to see if it needs to do more. It’s a dynamic cycle that allows the chatbot to chain multiple tools together to solve complex problems autonomously. This is what transforms a passive assistant into a proactive teammate that can navigate through a task until it’s actually finished.</p><h4>Terminal Output: Chaining in Action</h4><pre>You: what time is it in Toronto?<br>🤖 Thinking...<br>🔧 Calling tools:<br>  • get_timezone_for_city(city=Toronto)<br>  ✓ get_timezone_for_city: The timezone for Toronto is: America/Toronto<br>🤖 Getting response from LLM...<br>🔧 Calling tools:<br>  • get_time_in_timezone(timezone=America/Toronto)<br>  ✓ get_time_in_timezone: Current time in America/Toronto: 13:17:54 EDT<br>🤖 Getting response from LLM...<br>💬 A: The current time in Toronto is 13:17:54 EDT.</pre><p><strong>The LLM wasn’t just answering; it was strategizing.</strong> By chaining these tools, it bridged the gap between a simple query and a multi-step solution.</p><h3>Conclusion</h3><p>With tool calling, we’ve transformed the chatbot into an <strong>intelligent agent</strong> that can:</p><ul><li>Discover tools automatically</li><li>Execute real-time actions</li><li>Chain tools to solve multi-step problems</li><li>Work across multiple LLM providers (Ollama, Gemini)</li></ul><p>At this point, the chatbot can <strong>retrieve knowledge, reason over it, and act on the world</strong> — the same core loop behind most modern AI agents.</p><h4>Key Takeaways</h4><ol><li><strong>Tool calling bridges knowledge and action</strong></li><li><strong>Docstrings are the API</strong> — LLMs read them</li><li><strong>The loop is simple</strong> — plan → execute → observe → repeat</li><li><strong>Schemas are the contract</strong> between code and model</li><li><strong>Guardrails matter</strong> — error handling and max iterations</li></ol><h4>Looking Back: What I’ve Learned Since Starting</h4><p>I finished the code for this project in October 2025. Looking back now, I’ve realized how much the industry has shifted toward standardization:</p><ul><li><strong>The “Manual” Way is the Best Way to Learn:</strong> We built this tool system from scratch to understand how LLMs actually “think.” A new standard called <strong>MCP (Model Context Protocol)</strong> is gaining massive traction now — it is essentially the “API for AI” that works across all providers. Even though we used OpenWebUI here, the logic is the same. Learning these fundamentals first makes it much easier to make sense of these new universal protocols.</li><li><strong>Focusing on the App, Not the Plumbing:</strong> Lately, I’ve been using <strong>Vertex AI Search</strong> and <strong>Google Search grounding</strong>. These features fill the gaps I encountered while building my own RAG and tool-calling systems. The beauty is that these services move the heavy lifting of data retrieval onto the platform, allowing me to focus on the app experience rather than managing the complexity of the underlying systems.</li></ul><p>Building it “the hard way” first provided the necessary knowledge to actually make sense of the tech stack. In my next series, I’ll be moving from the “plumbing” to the “brain” — exploring tokens, context engineering, and how to use Vertex AI to build truly <strong>production-ready</strong> AI-powered apps.</p><h4>Try It Yourself</h4><p>The complete code is available on GitHub:</p><ul><li><strong>Repository</strong>: <a href="https://github.com/able-wong/rag-cli-chatbot">https://github.com/able-wong/rag-cli-chatbot</a></li><li><strong>Tag</strong>: part7-tool-calling</li></ul><pre>git clone https://github.com/able-wong/rag-cli-chatbot.git<br>cd rag-cli-chatbot<br>git checkout part7-tool-calling<br>pip install -r requirements.txt<br>python3 main.py</pre><p>Browse the <a href="https://openwebui.com/tools">OpenWebUI Tools Gallery</a> for 100+ ready-to-use tools.</p><p><strong>Questions?</strong> Drop a comment or open an issue on GitHub.</p><p>This concludes the main series on building a RAG chatbot from scratch. We’ve covered ingestion, retrieval strategies, query rewriting, HyDE, hybrid search, metadata filtering, and now tool calling.</p><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebe3b3db2c68" width="1" height="1" alt=""><hr><p><a href="https://blog.gopenai.com/understanding-conversational-ai-part-7-supercharge-your-rag-chatbot-with-tool-calling-ebe3b3db2c68">Understanding Conversational AI Part 7: Supercharge Your RAG Chatbot with Tool Calling</a> was originally published in <a href="https://blog.gopenai.com/">GoPenAI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clashing Signals: Why AI is Both a Corporate Fiction and a Real Threat]]></title>
            <link>https://medium.com/@able_wong/clashing-signals-why-ai-is-both-a-corporate-fiction-and-a-real-threat-3342d51413a4?source=rss-8da73a749de4------2</link>
            <guid isPermaLink="false">https://medium.com/p/3342d51413a4</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[economics]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[John Wong]]></dc:creator>
            <pubDate>Sat, 10 Jan 2026 07:53:17 GMT</pubDate>
            <atom:updated>2026-01-10T07:53:17.444Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YXSyQRF9Rgo1_EcQr_zf6Q.png" /><figcaption>Engineer in front of a futuristic Screen (Generated by Nano Banana)</figcaption></figure><p>There is a strange contradiction in the news lately regarding the state of our industry.</p><p>On one hand, there is the <strong>Macro View</strong>. A recent report from Oxford Economics (January 2026) suggests that “AI layoffs” at major corporations are largely a corporate fiction. The study argues that companies are using AI as a PR-friendly cover for traditional over-hiring. Their logic is hard to ignore: if AI were truly replacing humans at scale, national productivity data should be spiking. So far, the needle hasn’t moved.</p><p>On the other hand, we have the <strong>Micro View</strong> — and it tells a much more disruptive story. Take <strong>Tailwind Labs</strong>, the team behind Tailwind CSS. They recently let go of 75% of their engineering team (3 out of 4 developers). The irony is staggering: Tailwind is more popular than ever, but because AI models have scraped their documentation so effectively, developers no longer need to visit their site. For a business model built on documentation traffic, AI didn’t just automate a task; it dismantled a revenue funnel almost overnight.</p><p>Both stories are likely true. While large enterprises might be “AI-washing” their spreadsheets, the disruption on the ground for small, high-leverage teams is real.</p><h4>A Shift in Workflow: From Building to Steering</h4><p>I have seen this shift firsthand while building a startup app over the last few months. My role has fundamentally evolved from <strong>writing</strong> to <strong>steering intent.</strong> I am no longer just a coder; I am an orchestrator of multiple AI agents.</p><p>My current workflow is no longer about the “how,” but the “what”:</p><ul><li><strong>Planning:</strong> I brainstorm ideas with <strong>Claude Code</strong>, which then automatically files GitHub issues with full business requirements.</li><li><strong>Architecture:</strong> I design the GCP platform alongside the AI, reviewing class and function prototypes to ensure the foundation is solid before implementation begins.</li><li><strong>Execution &amp; Review:</strong> AI handles the bulk of the boilerplate while I use <strong>Gemini</strong> and <strong>GitHub Copilot</strong> as my primary code reviewers.</li><li><strong>Maintenance:</strong> My focus is now almost entirely on the never-ending <strong>refactoring</strong> needed to keep the system maintainable.</li></ul><h4>The “Good Enough” Threshold</h4><p>Does the AI do an excellent job? <strong>Not always.</strong> It makes poor decisions without deep context. It requires constant “steering” to stay on track.</p><p>But the impact is undeniable: by using these tools, I have essentially absorbed the work usually handled by a <strong>Product Manager, Designer, SRE, and System Architect.</strong> I am not claiming to be an expert in all those fields, but the tools allow me to perform at a “good enough” level to keep the project moving without the friction of a five-person meeting.</p><h4>The New “Value Add”</h4><p>While the national economy hasn’t seen the spike yet, the <strong>individual engineer</strong> is now a massive force multiplier.</p><p>We can do more, but we are also expected to <em>be</em> more. This presents a massive risk for the field: <strong>If you only want to be the person who takes a ticket and writes a function, that role is rapidly being automated.</strong></p><p>Our value is no longer in the output (the code). Our value is in the <strong>intent</strong>: the architecture, the business logic, and the high-level technical steering required to keep an LLM from building a house of cards.</p><h4>From Builders to Owners</h4><p>We are entering a phase where being a “productive coder” is no longer enough. The industry is shifting, and our definition of what it means to be a software engineer must shift with it.</p><p>We must move from being <strong>builders of components</strong> to being <strong>owners of systems.</strong> The macro data might not show the revolution yet, but if you look at the “team of one” effectively running an entire department’s worth of logic, the trend is impossible to ignore.</p><p><strong>Sources:</strong></p><ul><li><em>Fortune: “</em><a href="https://fortune.com/2026/01/07/ai-layoffs-convenient-corporate-fiction-true-false-oxford-economics-productivity/"><em>AI layoffs are looking more and more like corporate fiction</em></a><em>” (Jan 2026)</em></li><li><em>Neowin: “</em><a href="https://www.neowin.net/news/tailwind-lays-off-75-employees-as-ai-kills-majority-of-its-revenue/"><em>Tailwind lays off 75% of employees as AI kills majority of its revenue</em></a><em>” (Jan 2026)</em></li></ul><p><em>Note: This article was created with the help of artificial intelligence.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3342d51413a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>