<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dr. Wai Yan Nyein Naing on Medium]]></title>
        <description><![CDATA[Stories by Dr. Wai Yan Nyein Naing on Medium]]></description>
        <link>https://medium.com/@waiyan.nn18?source=rss-576efcbf19ef------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*tfQusZKXOKmTcwJ_.</url>
            <title>Stories by Dr. Wai Yan Nyein Naing on Medium</title>
            <link>https://medium.com/@waiyan.nn18?source=rss-576efcbf19ef------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 07:09:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@waiyan.nn18/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[5 Things You Should Know About Industrial AI (It’s Not Just Agents)]]></title>
            <link>https://medium.com/@waiyan.nn18/5-things-you-should-know-about-industrial-ai-its-not-just-agents-7503e7161f4d?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/7503e7161f4d</guid>
            <category><![CDATA[industrial-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Fri, 24 Apr 2026 01:28:46 GMT</pubDate>
            <atom:updated>2026-04-24T01:28:46.333Z</atom:updated>
            <content:encoded><![CDATA[<p>We like to imagine AI as one powerful model doing everything. Clean. Elegant. Almost magical.</p><p>But that’s not how real systems work.</p><p>In industry, AI looks more like a toolbox than a brain. Different pieces, each doing one job well, stitched together into something that actually holds up under pressure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0XRp1SRq8v2Up2zc" /><figcaption>Photo by <a href="https://unsplash.com/@barbarazandoval?utm_source=medium&amp;utm_medium=referral">Barbara Zandoval</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>1. Machine Learning Is Powerful, But Narrow</h3><p>Machine learning is excellent at learning from large-scale data. It detects patterns humans would miss, such as fraud behavior, equipment anomalies, and demand shifts.</p><p>But it does not understand business policy. It can estimate risk, but it does not decide what action should be taken.</p><p>It answers what is likely.<br>It does not answer what is allowed.</p><h3>2. LLMs Bring Reasoning, But Need Guardrails</h3><p>Large language models are strong at understanding intent, working across domains, and generalizing reasoning. They interpret requests, summarize context, and connect ideas naturally.</p><p>This is what makes modern AI systems feel intelligent.</p><p>But without grounding in rules or domain context, they can hallucinate. Not always obviously wrong, but subtly wrong in ways that matter in real systems.</p><p>They are powerful, but not safe enough on their own.</p><h3>3. Expert Systems Still Matter for Control</h3><p>Expert systems encode domain knowledge such as compliance rules, safety constraints, and business logic.</p><p>They are explicit and auditable, which is critical in regulated industries like finance, healthcare, and engineering.</p><p>They are not perfect. They are expensive to build and cannot cover every edge case. But they provide clarity and control where probabilistic models alone are not enough.</p><h3>4. Rule-Based Systems Turn Decisions Into Action</h3><p>Rule-based systems enforce what should happen next.</p><p>They may look simple, but they are essential for automation and workflow execution. They ensure that decisions follow business policy and operational constraints.</p><p>At the same time, rules have limits. They struggle with edge cases and changing environments. This is where machine learning, deep learning, and LLMs complement them by handling variation and generalization.</p><h3>5. Real Systems Combine All of These</h3><p>Industrial AI works because these pieces are combined.</p><ul><li>A model detects patterns and risk.</li><li>An LLM interprets intent and context.</li><li>Expert knowledge defines constraints.</li><li>Rules enforce decisions.</li><li>Workflows execute actions.</li><li>Humans step in when needed.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iPUzQ8bdYQxB-_lltDV2Ig.png" /><figcaption>Ensemble of Industrial AI Principles</figcaption></figure><p>Let’s look at a real example.</p><p>Consider<strong> Stripe Radar</strong>, a leading payment platform. It uses machine learning to analyze massive transaction data and detect fraud patterns. But the final decision is shaped by rules and business logic, deciding whether to allow, block, or review a payment. This combination helps prevent large-scale fraud while maintaining control and compliance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*7dnb--VTx4UY765U" /><figcaption>Stripe Radar AI Risk Management system</figcaption></figure><p>The same pattern appears in healthcare, manufacturing, and enterprise AI systems. Models provide intelligence, but rules and expert knowledge ensure safe and controlled decisions.</p><h3>Final Thought</h3><p>Industrial AI is not just agentic.</p><p>It is a combination of all AI principles working together.</p><p>Machine learning learns patterns.<br>LLMs reason and communicate.<br>Expert systems define knowledge.<br>Rules enforce control.</p><p>And together, they solve real business problems in the real world.</p><p><strong>References:</strong></p><ol><li><a href="https://docs.stripe.com/radar/rules?type=new">https://docs.stripe.com/radar/rules?type=new</a></li><li><a href="https://aithefuture.wordpress.com/2018/05/08/mycin/">https://aithefuture.wordpress.com/2018/05/08/mycin/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7503e7161f4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM Model Evaluation Guide (2026) အကောင်းဆုံး AI Model ကို ဘယ်လိုရွေးသင့်သလဲ ?]]></title>
            <link>https://medium.com/@waiyan.nn18/llm-model-evaluation-guide-2026-%E1%80%A1%E1%80%80%E1%80%B1%E1%80%AC%E1%80%84%E1%80%BA%E1%80%B8%E1%80%86%E1%80%AF%E1%80%B6%E1%80%B8-ai-model-%E1%80%80%E1%80%AD%E1%80%AF-%E1%80%98%E1%80%9A%E1%80%BA%E1%80%9C%E1%80%AD%E1%80%AF%E1%80%9B%E1%80%BD%E1%80%B1%E1%80%B8%E1%80%9E%E1%80%84%E1%80%B7%E1%80%BA%E1%80%9E%E1%80%9C%E1%80%B2-4e4740af6084?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e4740af6084</guid>
            <category><![CDATA[2026]]></category>
            <category><![CDATA[model-evaluation]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 02:48:04 GMT</pubDate>
            <atom:updated>2026-04-17T02:48:04.498Z</atom:updated>
            <content:encoded><![CDATA[<p>LLM (Large Language Models) အသစ်တွေက အပတ်တိုင်းနီးပါး ထွက်လာနေတဲ့ ဒီကနေ့ခေတ်မှာ <strong>“ဘယ် model က အကောင်းဆုံးလဲ”</strong> ဆိုတဲ့ မေးခွန်းဟာ အဖြေရှာရ ခက်ခဲလာပါပြီ။ အမှန်တကယ် မေးသင့်တာက <strong>“ကိုယ့်ရဲ့ Use Case အတွက် ဘယ် model က အသင့်တော်ဆုံးလဲ”</strong> ဆိုတာပါ။</p><p>ဘာကြောင့်လဲဆိုတော့ AI Model Evaluation ဆိုတာ Benchmark score တစ်ခုတည်းကြည့်ပြီး Champion ရွေးတာမျိုး မဟုတ်လို့ပါပဲ, Performance, Robustness, Cost-efficiency, Latency နဲ့ Task fit တို့ကို ပေါင်းစပ်စဉ်းစားရတဲ့ Engineering process တစ်ခု ဖြစ်ပါတယ်။</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tqof7qWKdCS1dvBhmgyl3Q.png" /><figcaption><a href="https://whatllm.org/explore">https://whatllm.org/explore</a></figcaption></figure><h3>၁။ Model Evaluation က ဘာကြောင့် အရေးကြီးသလဲ</h3><p>AI Model တစ်ခုကို သုံးတော့မယ်ဆိုရင် <strong>ကိုယ့်ရဲ့ Laptop မှာ စမ်းကြည့်လို့ အဆင်ပြေရုံ၊ Proof of Concept (PoC) အဆင့်မှာတင် အလုပ်ဖြစ်ရုံနဲ့ မလုံလောက်ပါဘူး။</strong> တကယ့် Production Workflow ထဲ ရောက်သွားတဲ့အခါ အောက်ပါ မေးခွန်းတွေက အဖြေရှိနေဖို့ လိုပါတယ်။</p><ul><li><strong>Task-Specific Quality:</strong> Factual QA မှာ ကောင်းပေမယ့် Coding task မှာရော တကယ် ပြသနာဖြေရှင်းနိုင်ရဲ့လား ?</li><li><strong>Latency &amp; Speed:</strong> Benchmark မြင့်ပေမယ့် Response ရဖို့ စက္ကန့်ပေါင်းများစွာ စောင့်နေရမှာလား ?</li><li><strong>Cost Efficiency:</strong> Quality အနည်းငယ် လျော့ပေမယ့် Cost က အများကြီး သက်သာနိုင်မလား ?</li><li><strong>Stability &amp; Integration:</strong> Tool calling နဲ့ Agent loop တွေထဲမှာ တည်တည်ငြိမ်ငြိမ် အလုပ်လုပ်နိုင်ပါ့မလား။</li></ul><h3>၂။ Core Performance Metrics များအား နားလည်ခြင်း</h3><p>Model evaluation ရဲ့ အခြေခံအဆင့်မှာ အောက်ပါ Metrics တွေကို တိုင်းတာလေ့ ရှိပါတယ်</p><h4><strong>1. Accuracy / F1 Score = </strong>Quality Checker</h4><p><strong>Accuracy</strong> က အဖြေမှန်နှုန်းကို တိုင်းတာပြီး၊ <strong>F1 Score</strong> ကတော့ အဖြေမှန်ရဖို့ ကြိုးစားရင်း လွဲချော်သွားတဲ့ False Positives နဲ့ False Negatives ကြားက Balance ကို တိုင်းတာပါတယ်။</p><h4><strong>2. BLEU / ROUGE</strong></h4><p>Text MatcherModel အဖြေနဲ့ Reference စာသား ဘယ်လောက်တူလဲဆိုတာကို နှိုင်းယှဉ်တာပါ။ <strong>BLEU</strong> က Translation (Precision) အတွက်ဖြစ်ပြီး <strong>ROUGE</strong> က Summarization (Recall) အတွက် ဖြစ်ပါတယ်။</p><p><strong>Perplexity</strong>Predictability MeterModel က နောက်လာမယ့် စကားလုံးကို ခန့်မှန်းရတာ ဘယ်လောက်ထိ ဇဝေဇဝါ ဖြစ်နေလဲဆိုတာကို တိုင်းတာတာပါ။ Score နည်းလေလေ Model က ပိုပြီး “နားလည်လွယ်” လေလေ ဖြစ်ပါတယ်။</p><h3>၃။ Modern Capability Benchmarks (2026 Context)</h3><p>2026 မှာတော့ ရိုးရိုး Benchmark တွေက အရမ်းလွယ်ကူသွားတဲ့အတွက် <a href="https://whatllm.org/">WhatLLM.org</a> ရဲ့ <strong>Intelligence Index</strong> မှာ အသုံးပြုထားတဲ့ အောက်ပါ Standard အသစ်တွေကို သုံးဖို့ လိုအပ်လာပါတယ်။</p><ul><li><strong>GPQA Diamond:</strong> PhD-level သိပ္ပံပညာရပ်ဆိုင်ရာ Deep Reasoning (Knowledge recall ထက် Multi-step တွေးခေါ်မှု) ကို စမ်းသပ်ပါတယ်။</li><li><strong>AIME 2025:</strong> Mathematical logical depth ကို တိုင်းတာပါတယ်။</li><li><strong>MMLU-Pro:</strong> Knowledge နဲ့ Reasoning ကို ပိုမိုတိကျစွာ ခွဲခြားပေးနိုင်တဲ့ MMLU upgrade ဖြစ်ပါတယ်။</li><li><strong>LiveCodeBench:</strong> Coding models တွေရဲ့ တကယ့်လက်တွေ့ code generation စွမ်းရည်ကို Contamination-free စမ်းသပ်ပါတယ်။</li><li><strong>Terminal-Bench &amp; SciCode:</strong> DevOps, System operations နဲ့ Scientific computing လိုမျိုး Coding ထက် ပိုမိုနက်နဲတဲ့ Task တွေကို တိုင်းတာပါတယ်။</li></ul><h3>၄။ Benchmark Winner က Production Winner မဟုတ်နိုင်တဲ့အကြောင်း</h3><p><strong>WhatLLM</strong> ရဲ့ 2026 Data အရ Model ရွေးချယ်မှုဟာ “Best” ဆိုတာထက် “Strategic” ဖြစ်ဖို့ ပိုအရေးကြီးပါတယ်-</p><ul><li><strong>Overall Quality:</strong> Gemini 3.1 Pro Preview (Quality Index: 57.18)</li><li><strong>Long Context:</strong> GPT-5.4 (xhigh) (1.1M tokens Window)</li><li><strong>Best Value:</strong> MiniMax-M2.7 (Cheaper &amp; Reliable)</li><li><strong>Coding Leader:</strong> GPT-5.2 (xhigh) (Top in LiveCodeBench)</li><li><strong>Open Source:</strong> GLM-4.7 / DeepSeek V3.2 (High performance with self-hosting)</li></ul><h3>၅။ Use Case အလိုက် Model ရွေးချယ်မှု Framework</h3><ol><li><strong>Fact Retrieval &amp; RAG:</strong> Grounding faithfulness က Benchmark score ထက် ပိုအရေးကြီးပါတယ်။</li><li><strong>General NLP Tasks:</strong> Summarization သို့မဟုတ် Classification အတွက် Latency နဲ့ Price per token ကို ပိုကြည့်သင့်ပါတယ်။</li><li><strong>Software Engineering:</strong> Claude Opus 4.5 (Reasoning) လိုမျိုး Multi-file understanding သို့မဟုတ် Gemini 3 Pro Preview လို Code generation မြန်တာတွေကို ရွေးသင့်ပါတယ်။</li><li><strong>On-Device Deployment:</strong> Sensitive data တွေအတွက် Gemma 3 သို့မဟုတ် Llama 3.2 လိုမျိုး Open-weight models တွေကို VRAM tier အလိုက် ရွေးချယ်ပါ။</li></ol><h3>၆။ Practical Workflow: Model စမ်းသပ်နည်း အဆင့်ဆင့်</h3><p>၁။ <strong>Define Use Case:</strong> တိကျတဲ့ လုပ်ငန်းစဉ် (Business Scope) သတ်မှတ်ပါ။</p><p>၂။ <strong>Set Success Metrics:</strong> Accuracy, Latency (under 2s) စတဲ့ KPI သတ်မှတ်ပါ။</p><p>၃။ <strong>Public + In-house Eval:</strong> Public benchmark နဲ့ Shortlist လုပ်ပြီး၊ ကိုယ့်ရဲ့ကိုယ်ပိုင် Data နဲ့ Final Decision ချပါ။</p><p>၄။ <strong>Human Evaluation:</strong> Writing quality နဲ့ Safety ပိုင်းအတွက် လူကိုယ်တိုင် ပြန်လည်စစ်ဆေးပါ။</p><p>2026 မှာ “Best Model” ဆိုတာ မရှိပါဘူး၊ ကိုယ့်ရဲ့ Task + Constraint + Reality နဲ့ အကိုက်ညီဆုံးဖြစ်မယ့် <strong>“Best-Fit Model”</strong> သာ ရှိပါတယ်။ Model evaluation ကို Quality တစ်ခုတည်းမဟုတ်ဘဲ Price, Speed နဲ့ Context Window တို့ကိုပါ တွဲဖက်စဉ်းစားခြင်းကသာ အောင်မြင်တဲ့ AI integration ကို ဖြစ်ပေါ်စေမှာပါ။</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e4740af6084" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Burmese-Coder-4B: A Burmese Coding LLM for Low-Resource Language AI]]></title>
            <link>https://medium.com/@waiyan.nn18/building-burmese-coder-4b-a-burmese-coding-llm-for-low-resource-language-ai-a9375e190d63?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/a9375e190d63</guid>
            <category><![CDATA[burmese]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[myanmar]]></category>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Sun, 12 Apr 2026 06:35:40 GMT</pubDate>
            <atom:updated>2026-04-12T06:35:40.204Z</atom:updated>
            <content:encoded><![CDATA[<p>Large language models have made coding assistance more accessible than ever. But for low-resource languages like Burmese, a major gap remains: a model may generate working code, yet still fail to explain it clearly and naturally in the user’s own language.</p><p>That was the motivation behind <strong>Burmese-Coder-4B</strong>, a Burmese coding model designed for programming assistance, technical explanation, and low-resource language evaluation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rkwSP4_51_ZSLnosN8auLQ.jpeg" /></figure><p>I’m <strong>Dr. Wai Yan Nyein Naing</strong>, a <strong>Myanmar AI researcher</strong> and Senior Research Scientist working across enterprise AI, LLMs, agentic AI, and Burmese language AI. Alongside my industry work in Singapore and the United States, I’ve also been building open resources for the Burmese AI community, including <strong>Burmese GPT</strong> and now <strong>Burmese-Coder-4B</strong>.</p><p>This article shares what I built, why it matters, and what it says about the future of custom language models for emerging language ecosystems.</p><h3>Why Burmese coding AI is still a real problem</h3><p>Most coding benchmarks and coding assistants are built around English.</p><p>That creates a hidden assumption: if the code works, the model is good enough.</p><p>But for Burmese users, that is not always true.</p><p>In practice, existing small language models often struggle with:</p><ul><li><strong>linguistic mixing</strong> across Burmese and other languages</li><li>weak Burmese technical explanation quality</li><li>unstable terminology</li><li>code that is correct, but reasoning that feels unnatural or broken</li></ul><p>For developers, students, and researchers working in Burmese, this matters. A coding model should not only produce executable code. It should also help users <strong>understand the logic in Burmese</strong> without forcing them back into English.</p><h3>The speed-run experiment</h3><p>I started with a practical question:</p><p><strong>Can a useful Burmese coding LLM be built quickly, with limited compute, and without a massive ready-made dataset?</strong></p><p>So I ran a focused experiment.</p><h3>What I did</h3><ul><li>built a dataset from scratch</li><li>applied knowledge distillation</li><li>ran unsupervised fine-tuning</li><li>performed supervised fine-tuning</li><li>added preference alignment</li></ul><h3>Compute setup</h3><ul><li>around <strong>$40 total compute cost</strong></li><li>around <strong>2.75 hours on H100 via RunPod</strong></li></ul><h3>Output</h3><ul><li>a <strong>4B-class Burmese coding model</strong></li><li>quantized and usable locally</li><li>practical for offline use</li><li>Ollama-compatible for lightweight experimentation</li></ul><p>In evaluation, Burmese-Coder-4B matched Gemma-3 4B on Pass@1 (62.0%) while improving Burmese explanation quality and reducing mixed-language drift.</p><p>That alone was encouraging. But the more important part was not just building the model. It was figuring out how to evaluate it properly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*FQICjcZSwspYhsS8.gif" /><figcaption><a href="https://giphy.com/gifs/Decimated-game-gaming-crypto">https://giphy.com/gifs/Decimated-game-gaming-crypto</a></figcaption></figure><h3>From Burmese GPT to Burmese-Coder-4B</h3><p>Before Burmese-Coder-4B, I had already released <strong>Burmese GPT</strong>, an open Burmese language model created to support Burmese text generation and help grow the open AI ecosystem for Myanmar language applications.</p><p>That work highlighted a deeper challenge: building a Burmese-capable model is only one part of the problem. The other part is making sure the model performs well on <strong>specialized tasks</strong>, such as coding and technical reasoning.</p><p>Burmese-Coder-4B builds on that direction.</p><p>It is a <strong>fine-tuned small language model optimized for Burmese technical language and coding tasks</strong>. The goal was not to create a generic chatbot. The goal was to create a model that better supports:</p><ul><li>Burmese programming prompts</li><li>technical reasoning in Burmese</li><li>code generation</li><li>clear step-by-step explanations</li><li>reduced mixed-language hallucination</li></ul><h3>The real challenge: evaluation</h3><p>One of the biggest lessons from this project was that <strong>functional correctness alone is not enough</strong>.</p><p>A model can pass a coding task and still produce poor Burmese explanations. It can generate valid code while mixing scripts, drifting across languages, or using unstable terminology.</p><p>So I built a <strong>Burmese coding model evaluation stack</strong> around two tracks:</p><h4>1. Functional correctness</h4><p>I extended HumanEval-style ideas into a Burmese-oriented setting to measure whether generated code actually works.</p><h4>2. Language-aware quality</h4><p>I built a framework to evaluate:</p><ul><li>Burmese language correctness</li><li>instruction following</li><li>code quality</li><li>explanation quality</li><li>linguistic alignment</li><li>mixed-language penalty</li></ul><p>The evaluation framework also supports <strong>LLM-as-a-judge</strong>, making it possible to plug in OpenAI-compatible models for rubric-based scoring.</p><p>This part is important because low-resource AI often gets judged using English-centric standards. That misses what real users in those language communities actually experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UPezIW2HtdEusAYAKzFP0w.png" /></figure><h3>Training Process: Adapting the Model</h3><p>To build <strong>Burmese-Coder-4B</strong>, I adapted the <strong>Gemma-3 4B</strong> family in two stages.</p><h4>Stage 1: Supervised Fine-Tuning</h4><p>The first stage used a <strong>Burmese MBPP-style dataset</strong> with <strong>974 programming tasks</strong>. Each example included:</p><ul><li>a Burmese instruction</li><li>a Python solution</li><li>a Burmese explanation of the code logic</li></ul><p>This stage was designed to teach the model not only how to generate code, but also how to explain technical reasoning clearly in Burmese.</p><p>I used <strong>parameter-efficient fine-tuning with LoRA</strong>, rather than updating the full model. This made the process much lighter and more practical.</p><h4>Stage 2: Preference Alignment with DPO</h4><p>After SFT, I applied <strong>Direct Preference Optimization (DPO)</strong> to improve linguistic stability.</p><p>The goal here was simple: make the model prefer <strong>cleaner Burmese responses</strong> over outputs with:</p><ul><li>script mixing</li><li>hallucinated foreign tokens</li><li>unstable technical phrasing</li></ul><p>In other words, the second stage was less about teaching coding ability and more about improving <strong>language quality and consistency</strong>.</p><h3>Two-Stage Adaptation Pipeline</h3><ul><li><strong>Base model:</strong> Gemma-3 4B</li><li><strong>Training data:</strong> Burmese MBPP (974 tasks)</li><li><strong>Stage 1:</strong> SFT</li><li><strong>Stage 2:</strong> DPO alignment</li><li><strong>Final model:</strong> Burmese-Coder-4B</li></ul><p>This two-stage pipeline turned out to be important.<br> The first stage taught the model <em>what to do</em>.<br> The second stage helped it do that in <strong>more natural, stable Burmese</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ErRgQ4xxFf0CGIqOBaqtng.png" /></figure><h3>Results</h3><p>Using this framework, <strong>Burmese-Coder-4B</strong> achieved a strong balance between code quality and Burmese language quality.</p><h3>Key outcomes</h3><ul><li><strong>Pass@1:</strong> 62.0%</li><li>matched the base Gemma 3 4B model on functional correctness</li><li>improved rubric quality over the base model</li><li>significantly reduced mixed-language penalty</li></ul><p>One of the most important findings was this:</p><blockquote><em>A model can preserve coding performance while becoming much more linguistically usable for Burmese users.</em></blockquote><p>That is exactly the kind of improvement that matters in practice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hAoKdUnGLaNQ0TAM4wd08A.jpeg" /></figure><h3>Why this matters beyond one model</h3><p>This project is not only about Burmese-Coder-4B.</p><p>It is also a signal about where AI is going.</p><p>We are moving into a phase where building domain-specific and language-specific models is becoming far more accessible. You no longer always need massive budgets, giant datasets, or long research cycles to build something useful.</p><p>For many applications, what matters is:</p><ul><li>a focused use case</li><li>the right data strategy</li><li>a practical fine-tuning pipeline</li><li>a better evaluation loop</li></ul><p>That is especially true for:</p><ul><li>low-resource languages</li><li>industry-specific assistants</li><li>local/private deployment</li><li>educational tools</li><li>edge and offline scenarios</li></ul><p>By the end of this decade, I expect many developers and teams to build <strong>custom small models</strong> for their own workflows, products, and communities.</p><h3>Open resources</h3><p>Alongside the model, I also open-sourced the surrounding assets needed to make the work useful for others:</p><ul><li><strong>technical report</strong></li><li><strong>training and evaluation datasets</strong></li><li><strong>language-aware benchmark framework</strong></li><li><strong>open weights</strong></li><li><strong>quantized local deployment version</strong></li></ul><p>That matters because open models alone are not enough. To support real research and community progress, people also need:</p><ul><li>data</li><li>evaluation tools</li><li>documentation</li><li>reproducible baselines</li></ul><h3>What I hope this contributes</h3><p>For Burmese and Myanmar AI, I hope this contributes in two ways:</p><h4>1. A stronger open foundation for research</h4><p>Low-resource language research needs reusable benchmarks, datasets, and evaluation methods.</p><h4>2. A practical example for other emerging language ecosystems</h4><p>The same pattern can apply beyond Burmese:</p><ul><li>start small</li><li>define the task clearly</li><li>optimize around real user needs</li><li>measure more than just benchmark accuracy</li></ul><h3>Final thoughts</h3><p>Building AI for low-resource languages is not only a technical challenge. It is also a question of accessibility, inclusion, and ecosystem growth.</p><p>Burmese-Coder-4B is one small step in that direction.</p><p>It shows that useful, localized AI systems can be built with focused effort, limited compute, and open tooling — as long as we care about the right metrics and the real needs of users.</p><p>For me, that is one of the most exciting parts of the current AI era:</p><p><strong>we are getting closer to a world where more communities can build AI in their own language, for their own needs.</strong></p><h3>Resources</h3><ul><li><strong>Technical Report </strong>[<a href="https://waiyannyeinnaing.github.io/assets/pdf/burmese_coder_4b.pdf">https://waiyannyeinnaing.github.io/assets/pdf/burmese_coder_4b.pdf</a>]</li><li><strong>Open Weights:</strong> [<a href="https://huggingface.co/WYNN747/burmese-coder-4b-SFT">https://huggingface.co/WYNN747/burmese-coder-4b-SFT</a>]</li><li><strong>Ollama / GGUF Model:</strong> [<a href="https://huggingface.co/WYNN747/burmese-coder-4b">https://huggingface.co/WYNN747/burmese-coder-4b</a>]</li><li><strong>Train Dataset:</strong> [<a href="https://huggingface.co/datasets/WYNN747/burmese-mbpp">https://huggingface.co/datasets/WYNN747/burmese-mbpp</a>]</li><li><strong>Eval Dataset:</strong> [<a href="https://huggingface.co/datasets/WYNN747/burmese-human-eval">https://huggingface.co/datasets/WYNN747/burmese-human-eval</a>]</li><li><strong>Evaluation Framework:</strong> [<a href="https://github.com/WaiYanNyeinNaing/burmese-coding-eval">https://github.com/WaiYanNyeinNaing/burmese-coding-eval</a>]</li><li><strong>Burmese GPT:</strong> [<a href="https://huggingface.co/WYNN747/Burmese-GPT">https://huggingface.co/WYNN747/Burmese-GPT</a>]</li><li><strong>Contact: </strong><a href="https://www.linkedin.com/in/wai-yan-nyein-naing/">https://www.linkedin.com/in/wai-yan-nyein-naing/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9375e190d63" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Teaching a LEGO SPIKE Robot to Outsmart Obstacles: Combining Imitation + Reinforcement Learning]]></title>
            <link>https://medium.com/@waiyan.nn18/teaching-a-lego-spike-robot-to-outsmart-obstacles-combining-imitation-reinforcement-learning-1518ccdcda2c?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/1518ccdcda2c</guid>
            <category><![CDATA[imitation-learning]]></category>
            <category><![CDATA[lego-robotics]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[robotics]]></category>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Mon, 08 Sep 2025 04:52:07 GMT</pubDate>
            <atom:updated>2025-09-08T04:52:07.782Z</atom:updated>
            <content:encoded><![CDATA[<p>Autonomous humanoid robots face a deceptively simple but fundamental challenge: <strong>walking forward while avoiding obstacles</strong>. For LEGO robots, the difficulty is amplified by limited sensors, imprecise motors, and mechanical instability. This article explores how combining <strong>Imitation Learning (IL)</strong> with <strong>Reinforcement Learning (RL)</strong> enables a LEGO SPIKE Prime robot to adaptively learn obstacle avoidance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*KHUsW5g5URYifKOUiHS_Sg.png" /></figure><h3>The Problem: Walking Is Easy. Avoiding Obstacles Is Hard.</h3><p>Forward walking in open space can be managed with basic motor control. However, once obstacles are introduced, naive control fails.</p><p>Challenges include:</p><ul><li><strong>Limited sensing</strong>: only a single ultrasonic distance sensor and the built-in gyro/IMU.</li><li><strong>Actuator constraints</strong>: LEGO motors lack precise torque control.</li><li><strong>Hardcoded fragility</strong>: rules such as “if obstacle &lt; 15 cm, turn left” work in one environment but fail in others.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uDDEmPrYaXECq4liZi8l0g.png" /><figcaption>Available Sensors</figcaption></figure><h3>Why Not Just Reinforcement Learning?</h3><p>RL is powerful for learning through trial and error, but has limitations on physical robots:</p><ul><li>Requires <strong>hundreds of episodes</strong> to converge, impractical for hardware with limited battery life.</li><li>Can lead to unsafe or repetitive behaviors (e.g., endless backing up).</li><li>Sensor noise and motor drift make learning unstable without additional shaping.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*WkTuCtwTMdnrWqMFqGbNuw.gif" /></figure><h3>A Hybrid Solution: Imitation + RL</h3><p>A hybrid approach was adopted to overcome these limitations.</p><h3>Step 1: Imitation Learning (IL)</h3><p>The macro demonstration can be represented as a fixed policy sequence:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*36-lX-nWVrdx0pvL7EZotg.png" /></figure><p>This initializes the Q-table with <strong>deterministic priors</strong> for NEAR states and a <strong>uniform prior</strong> for the DECIDE state.</p><h3>Step 2: Reinforcement Learning (RL) with Q-Table</h3><p>Q-learning with an ε-greedy policy was then applied. Rewards were shaped as follows:</p><ul><li>+0.1 for forward progress</li><li>–0.2 for entering obstacle zone</li><li>+0.5 for clearing an obstacle after DECIDE</li><li>–0.1 if DECIDE action failed to clear</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bSt3YLA_Dw0nwQbTwXvDEg.png" /></figure><h3>LEGO Setup with Pybricks</h3><ul><li><strong>Hub:</strong> LEGO SPIKE Prime running Pybricks firmware</li><li><strong>Motors:</strong> Left leg (Port C), Right leg (Port D), Arm (Port B)</li><li><strong>Sensors:</strong> Ultrasonic (Port F), built-in gyro/IMU for yaw correction</li><li><strong>States:</strong> FAR, NEAR_0–3 (macro steps), DECIDE</li><li><strong>Actions:</strong> FORWARD, STOP, BACKUP, TURN_LEFT, TURN_RIGHT, etc.</li></ul><p>The gyro was critical to stabilize forward walking and reduce circling, ensuring rewards aligned with meaningful progress.</p><h3>Learning Outcomes</h3><h3>Imitation Phase (~5 episodes)</h3><ul><li>FAR → FORWARD=1.0</li><li>NEAR states → deterministic single actions</li><li>DECIDE → uniform (Resume=0.33, Left=0.33, Right=0.33)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g1EhB06grukq2B32uYTSAQ.png" /></figure><h3>RL Phase (~10 episodes)</h3><p>The Q-table shifted to:</p><ul><li>DECIDE → Resume=0.35, Left=0.17, Right=0.48</li></ul><p>This indicates that the robot learned a preference for <strong>TURN_RIGHT</strong> in its training environment, adapting beyond imitation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*NwftBy4v3cvH8S9dyw4ZOg.gif" /><figcaption>First early Episode of Agent Learning</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*JrDbMeLlxz_e0A05Iq6uuw.gif" /><figcaption>Agent action after learning the policy</figcaption></figure><h3>Key Takeaway: Obstacle Avoidance Learned, Not Hardcoded</h3><p>The essential contribution of this approach is <strong>adaptive obstacle avoidance</strong>. Unlike rigid rules, the hybrid IL+RL strategy enabled the robot to:</p><ul><li>Generalize beyond demonstrations across different environments.</li><li>Reduce training episodes compared to pure RL.</li><li>Learn practical avoidance strategies from interaction.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/238/1*giEQGBX2x8V0Nnsc7gFW_g.gif" /><figcaption>The agent is generalized and can avoid obstacles in new environments</figcaption></figure><h3>Lessons Learned</h3><ul><li><strong>Hardcoded rules</strong>: brittle and environment-specific.</li><li><strong>Pure RL</strong>: adaptive but sample-inefficient and risky.</li><li><strong>IL + RL hybrid</strong>: safe, efficient, and practical for real LEGO robots.</li><li><strong>Gyro integration</strong>: critical for stable forward walking and consistent rewards.</li></ul><h3>Conclusion</h3><p>Combining imitation learning with reinforcement learning allowed a LEGO humanoid robot to autonomously learn obstacle avoidance in just 15 episodes. The hybrid approach demonstrates how accessible hardware like LEGO SPIKE Prime can serve as a powerful platform for experimenting with modern robotics AI — bridging the gap between rigid rule-based programming and the adaptability of embodied learning.</p><h3><strong>Github Code Link</strong></h3><p><a href="https://github.com/WaiYanNyeinNaing/imitation-learning-lego-spike">https://github.com/WaiYanNyeinNaing/imitation-learning-lego-spike</a></p><h4><strong>Authors</strong></h4><ul><li>Dr. Wai Yan Nyein Naing ,</li><li>Aye Chan Kyal (<a href="https://www.instagram.com/chan.gadgetbots">https://www.instagram.com/chan.gadgetbots</a>)</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1518ccdcda2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering AI Communication: Getting the Best Results from Gen AI Applications]]></title>
            <link>https://medium.com/@waiyan.nn18/mastering-ai-communication-getting-the-best-results-from-gen-ai-applications-3d2cf1ef65fb?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d2cf1ef65fb</guid>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Sat, 05 Jul 2025 07:58:10 GMT</pubDate>
            <atom:updated>2025-07-05T07:58:10.803Z</atom:updated>
            <content:encoded><![CDATA[<p>Artificial intelligence (AI) is rapidly changing how we work and learn. From students to professionals, we all interact with AI applications daily. But to get the best results, understanding how to effectively communicate with AI is crucial. This article explores three key approaches to interacting with AI: automation, augmentation, and agency.</p><p><strong>Three Key Approaches to AI Interaction</strong></p><p>When using AI applications, consider your objective. Are you aiming for automation, augmentation, or agency?</p><ol><li><strong>Automation: </strong>This involves using AI to automate repetitive tasks. For example, you might use AI to analyze incoming emails, prioritizing important messages and generating draft replies. To achieve this, clearly define the specific task for the AI, outlining the desired input and output. Be precise in your instructions; the clearer your instructions, the better the automation.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ao1BVaf66N7OEYajSoiQ8w.png" /><figcaption>Generated with Gemini</figcaption></figure><p><strong>2. Augmentation:</strong> This approach leverages AI to enhance your capabilities, acting as a collaborative partner. Imagine creating a social media post. You could query the AI for trending topics, share your ideas and theme, request relevant images, and then instruct the AI to assemble the elements into a video. Augmentation is about brainstorming with the AI, combining its capabilities with your own creativity and expertise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FBfMrKRfVnS_7YVFRJtzCQ.png" /><figcaption>Generated with Gemini</figcaption></figure><p><strong>3. Agency:</strong> This involves using AI as an agent to build something new. For instance, you could use an AI agent to develop a new application. Instead of writing extensive code, you provide the AI with a detailed description of the application’s functionality and desired results. The AI acts as a tool to help you build the application based on your instructions. Think of it as delegating the development process to an AI assistant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m_UpjetiykVySe90JHRTjQ.png" /><figcaption>Generated with Gemini</figcaption></figure><p><strong>Effective Communication Strategies</strong></p><p>Before engaging any AI application, carefully consider:</p><p><strong><em>Your Goal: </em></strong><em>What are you trying to achieve? What specific outcome do you desire?<br></em><strong><em>Your Approach: </em></strong><em>Will you automate, augment, or use the AI as an agent?<br></em><strong><em>Clear Instructions: </em></strong><em>Provide precise and detailed instructions, using clear and unambiguous language.</em></p><p>By understanding these three approaches and implementing effective communication strategies, you can maximize the potential of AI applications and achieve optimal results. Stay tuned for future posts exploring specific techniques and examples.</p><p>Reference:</p><p><a href="https://www.anthropic.com/ai-fluency">https://www.anthropic.com/ai-fluency</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d2cf1ef65fb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tailored by Technology: The Impact of AI on Personalization]]></title>
            <link>https://medium.com/@waiyan.nn18/tailored-by-technology-the-impact-of-ai-on-personalization-3e5aa2284cb2?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e5aa2284cb2</guid>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Sun, 26 May 2024 17:12:21 GMT</pubDate>
            <atom:updated>2024-05-26T17:12:21.579Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>What is AI Personalization?</strong> Imagine watching a sci-fi movie like “Her,” where the AI assistant, Samantha, understands and responds to Theodore’s emotions and needs, creating deeply personalized interactions. AI personalization, while not quite at the level of Samantha, aims to provide similarly tailored experiences by using artificial intelligence to analyze user behavior, preferences, and data to predict and meet individual needs. This technology makes each user’s experience unique and more relevant to their interests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c8ilQYXOxThTcFuGEKiE9g.png" /><figcaption>Created by OpenAI-GPT4-o</figcaption></figure><p><strong>Consumer Expectations:</strong> AI-powered personalization is changing the game. Today, 71% of consumers expect tailored experiences, and 76% get frustrated when they don’t get them. This makes personalization essential for businesses. AI helps companies deliver real-time, relevant content, boosting engagement and loyalty. Personalized strategies can increase revenue by 10–15%, making customers feel valued and understood.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aeDUzt3-qm3nm-OCL0F1tw.png" /><figcaption>Created by OpenAI-GPT4-o</figcaption></figure><p>Technologies like OpenAI’s GPT-4o are leading the way. GPT-4o is faster, more versatile, and handles text, voice, and images seamlessly, making interactions more natural and intuitive. In retail, finance, and healthcare, AI’s ability to analyze data and predict needs is transforming operations. Retailers can offer customized shopping experiences, financial institutions can provide tailored advice, and healthcare providers can engage patients more personally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tQdeMU5sbqlYSuS_i07_WQ.png" /><figcaption>Created by OpenAI-GPT4-o</figcaption></figure><p><strong>Data Privacy Concerns:</strong> However, AI personalization raises data privacy concerns. AI systems need a lot of user data, raising questions about data collection, storage, and usage. Consumers want their data to be safe and companies to be transparent.</p><p>One solution is using open-source large language models (LLMs) or on-device LLMs. These allow companies to control their data, reducing the risk of breaches. Running LLMs locally offers advantages like greater data control and reduced latency but also comes with higher costs and complexity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gYr0kBTMBUKkZ0J39UUwWg.png" /><figcaption>Created by OpenAI-GPT4-o</figcaption></figure><p><strong>Conclusion:</strong></p><p>The potential of AI personalization is boundless, but it must be balanced with data privacy. We’re on the brink of developing even more sophisticated AI solutions that will enhance customer experiences while protecting their data.</p><p>How are you using AI to personalize experiences in your business?</p><p>#AI #Personalization #DigitalTransformation #CustomerExperience #DataPrivacy</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e5aa2284cb2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing Efficiency: Selecting the Right Vector Database for Your LLM/RAG Project]]></title>
            <link>https://medium.com/@waiyan.nn18/optimizing-for-performance-selecting-the-right-vector-database-for-your-llm-rag-project-194fc9f99a77?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/194fc9f99a77</guid>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Sat, 09 Dec 2023 01:26:26 GMT</pubDate>
            <atom:updated>2023-12-09T02:27:44.878Z</atom:updated>
            <content:encoded><![CDATA[<p>In the dynamic world of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) applications, selecting the right vector database requires a nuanced understanding of various factors. This includes hardware specifications, data characteristics, software configurations, database architecture, and the efficiency of chosen query algorithms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1O11ctr7eO72r4Z4pMzvMw.png" /><figcaption>Generated by Dall E</figcaption></figure><p><strong>Key Factors Influencing Vector Database Performance:</strong></p><p><strong>Hardware:</strong></p><ul><li>CPU: Core count and speed significantly impact processing and search speeds.</li><li>RAM: Adequate RAM ensures smooth handling of complex embeddings.</li><li>Storage: SSDs offer faster retrieval compared to HDDs.</li></ul><p><strong>Data Characteristics:</strong></p><ul><li>Document Size: Larger documents require more resources.</li><li>Embedding Chunk Size: Striking a balance between vector embedding size and performance is crucial.</li><li>Data Access Patterns: Query frequency affects system load and resource allocation.</li></ul><p><strong>Indexing Algorithm:</strong></p><ul><li>Indexing Techniques: Using hashing, quantization, or graph-based techniques, a vector database indexes vectors by mapping them to a given data structure. This enables a faster search.</li></ul><p>(Depending on your prioritization of speed versus accuracy, you can choose a hashing algorithm like Locality-Sensitive Hashing (LSH) for faster search or a graph-based technique like Hierarchical Navigable Small World (HNSW) for more accurate results with a balanced performance.)</p><p><strong>Querying Techniques:</strong></p><ul><li>Querying Techniques: In vector databases, querying entails matching query vectors with those indexed vector in the dataset to find the closest neighbors, where experimenting with methods like Cosine similarity, Euclidean distance, or dot product may enhance results based on the specific needs for speed or performance.</li></ul><p><strong>Database Architectures:</strong></p><ul><li>Single-node: Suitable for smaller datasets, offering simplicity but limited scalability and algorithmic diversity.</li><li>Multi-node: Ideal for larger datasets, offering greater scalability and supporting a wider range of complex query algorithms.</li><li>Performance Limitations: Single-node architectures may limit algorithm effectiveness for large scale database.</li><li>Cost and Complexity: Setting up and managing a multi-node cluster requires more expertise, effort, and resources compared to single-node systems due to additional hardware and software needs.</li></ul><h3>Making the Right Choice:</h3><ol><li>Define Application Needs: Consider data volume, performance expectations, and budget.</li><li>Choose Architecture &amp; Algorithm: Select single-node or multi-node based on scalability, workload, and desired query algorithm sophistication.</li><li>Research &amp; Evaluate: Analyze features, performance benchmarks, and algorithmic capabilities of different databases.</li><li>Conduct Practical Tests: Evaluate databases using real-world data and workflows representative of your application.</li><li>Make an Informed Decision: Select a database that aligns with your application’s specific requirements, balancing performance, scalability, cost, and algorithmic efficiency.</li></ol><h3>Conclusion:</h3><p>Choosing the most suitable vector database for LLMs and RAG applications demands a multifaceted approach. By considering the interplay of hardware, software, architecture, and efficient query algorithms, you can select a database that meets your current needs and scales effectively to future demands.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=194fc9f99a77" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Evaluation Metrics of Large Language Models (LLMs)]]></title>
            <link>https://medium.com/@waiyan.nn18/understanding-and-benchmarking-evaluation-metrics-of-large-language-models-llms-in-2023-9a4a858f782b?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a4a858f782b</guid>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Mon, 04 Dec 2023 19:53:48 GMT</pubDate>
            <atom:updated>2023-12-05T04:21:31.433Z</atom:updated>
            <content:encoded><![CDATA[<h3>Understanding Evaluation Metrics of Large Language Models (LLMs) Part 1</h3><p>In the rapidly evolving field of AI, understanding how large language models (LLMs) perform is crucial. These models are the backbone of numerous applications, from natural language processing to complex decision-making systems.</p><p>However, evaluating their effectiveness can be a complex task, involving a variety of metrics. In this post, we’ll demystify these metrics and explain what they really tell us about the performance of different models.</p><p>We’ll then apply this understanding to analyze a specific set of models, showcasing how these metrics reflect their capabilities.</p><p>Let’s dive into each metric with example sentences to better understand how they are used to evaluate large language models (LLMs):</p><ol><li><strong>RAM Usage:</strong></li></ol><ul><li>The RAM usage is an important factor, as it indicates the computational efficiency and resource requirements of the models.</li></ul><p><strong>2. LAMBADA (Perplexity)</strong>:</p><ul><li>Perplexity measures how well a model predicts a sample. A lower perplexity indicates a model is better at predicting the next item in a sequence.</li><li><strong>Example</strong>: Suppose a language model is predicting the next word in the sentence “The cat sat on the ___.” If the model has a high probability for appropriate words like ‘mat’, ‘floor’, or ‘couch’, it would have lower perplexity. Conversely, if it predicts unrelated words like ‘apple’, the perplexity would be higher.</li></ul><p><strong>3. LAMBADA (Accuracy)</strong>:</p><ul><li>Lambada accuracy in this context refers to how correctly a model can complete or predict text based on given context.</li><li><strong>Example</strong>: For a sentence like “I left my keys in the ___”, if the model correctly predicts ‘car’ or ‘house’, it is considered accurate. If it predicts irrelevant words, it’s inaccurate.</li></ul><p><strong>4. HellaSwag (Accuracy Normalized)</strong>:</p><ul><li>HellaSwag accuracy measures a model’s ability to make commonsense inferences, potentially adjusted (normalized) for varying difficulty levels.</li><li><strong>Example</strong>: In a scenario where the model is given a prompt like “When it starts to rain, people usually”, and it needs to choose the most commonsensical continuation. A correct prediction like ‘seek shelter’ or ‘open their umbrellas’ would contribute to higher accuracy.</li></ul><p><strong>5. Winogrande (Accuracy)</strong>:</p><ul><li>Winogrande accuracy measures a model’s ability to understand and reason through commonsense questions.</li><li><strong>Example</strong>: Given a sentence like “The trophy didn’t fit in the suitcase because it was too big.” If the model understands that ‘it’ refers to ‘the trophy’ and not ‘the suitcase’, it shows good commonsense reasoning.</li></ul><p><strong>6. PIQA (Accuracy)</strong>:</p><ul><li>PIQA accuracy assesses a model’s understanding of physical interactions in various scenarios.</li><li><strong>Example</strong>: For a question like “What should you do to make a cup of tea?”, if the model correctly identifies actions like ‘boil water’ and ‘put a tea bag in a cup’, it demonstrates accurate physical commonsense reasoning.</li></ul><p>6. <strong>CoQA (F1)</strong>:</p><ul><li>CoQA F1 score is a balance between precision and recall, commonly used in classification tasks.</li><li><strong>Example</strong>: In a conversational question-answering task, if a user asks “What is the capital of France?” and the model answers “Paris”, the F1 score considers both the correctness (precision) and the completeness (recall) of the answer. A high F1 score would indicate the model is not only accurate but also provides complete and relevant answers.</li></ul><p>Here are some the benchmark performances of open-source Large Language Models (LLMs) in 2023:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*GXoeW0jX93LA-zz9NUNcYw.png" /></figure><p>You can find further details on the updated benchmarking results for Large Language Models on Hugging Face :</p><p>https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard</p><p>References:</p><p><a href="https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYpb63e1ZR3aePczz3zlbJW-Y4/edit#gid=0">https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYpb63e1ZR3aePczz3zlbJW-Y4/edit#gid=0</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a4a858f782b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The AI Learning Curve: Do You Really Need Maths to Get Started?]]></title>
            <link>https://medium.com/@waiyan.nn18/the-ai-learning-curve-do-you-really-need-maths-to-get-started-563b7f971a09?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/563b7f971a09</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[beginners-guide]]></category>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Wed, 08 Feb 2023 04:01:02 GMT</pubDate>
            <atom:updated>2023-02-08T04:01:02.769Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*gMl67jdIuAnmh6GC8jsM4A.jpeg" /><figcaption><a href="https://www.coursera.org/specializations/mathematics-machine-learning">Image Source</a></figcaption></figure><p>AI is one of the most exciting and rapidly evolving fields today, and it has the potential to change the world in countless ways. But for many people, the idea of learning AI can be intimidating, especially if they think they need to have a strong mathematical background to do so.</p><p>Here’s the good news: you don’t need to be a math prodigy to get started with AI. In fact, you can start by simply understanding the basic concepts of AI, machine learning, and deep learning. Then, you can dive into the tools like <em>TensorFlow, PyTorch, scikit-learn, and Python</em> that make it easy to apply these concepts in practice.</p><p>Think of it like cooking. You don’t need to be a master chef to make a delicious meal. You just need to understand the basic ingredients, how to use the tools in your kitchen, and how to follow a recipe. Similarly, with AI, you don’t need to understand the math behind every algorithm to make it work. You just need to know how to use the tools and understand the basic concepts.</p><p>Of course, as you gain experience, you’ll want to dive deeper into the math and statistics that underlie AI. But that’s okay, too. You can start by learning the basics of statistics and probability, which will help you process data and understand the algorithms that are used in machine learning. Then, you can move on to deep learning and neural networks, learning <em>TensorFlow or Pytorch </em>deep learning frameworks and the basics of linear algebra and calculus along the way.</p><p>In the end, reading research papers and understanding the math behind the algorithms is a valuable goal, but it’s not a requirement to get started with AI. With the right approach and resources, anyone can learn the basics of AI and machine learning and then build upon that foundation to achieve their goals. So don’t be intimidated by the math. Just take it one step at a time and enjoy the journey!</p><p><strong>Remark:</strong></p><p>While it is possible to learn AI and machine learning without a strong mathematical background, it is important to note that if you want to develop new AI algorithms or read state-of-the-art research papers, a good understanding of certain mathematical concepts will be necessary.</p><p>These are the specific categories of mathematical concepts mainly used in AI:</p><p><strong>Linear Algebra:</strong> Matrices, vectors, eigenvalues, and eigenvectors.</p><p><strong>Calculus: </strong>Differentiation, partial derivatives, optimization.</p><p><strong>Probability:</strong> Bayes Theorem, Random Variables, Conditional Probabilities, and probability distributions.</p><p><strong>Statistics: </strong>Mean, variance, covariance, hypothesis testing, maximum likelihood estimation.</p><p><strong>Information Theory:</strong> Entropy, cross-entropy, Kullback-Leibler divergence.</p><p><strong>Optimization: </strong>Gradient descent, conjugate gradient, quasi-Newton methods.</p><p>Resources:</p><ol><li>Course : <a href="https://www.coursera.org/specializations/mathematics-machine-learning">https://www.coursera.org/specializations/mathematics-machine-learning</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=563b7f971a09" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The AI Battle Heats Up: Google’s Bard to Challenge ChatGPT’s Reign]]></title>
            <link>https://medium.com/@waiyan.nn18/the-ai-battle-heats-up-googles-bard-to-challenge-chatgpt-s-reign-5cf28d343e54?source=rss-576efcbf19ef------2</link>
            <guid isPermaLink="false">https://medium.com/p/5cf28d343e54</guid>
            <dc:creator><![CDATA[Dr. Wai Yan Nyein Naing]]></dc:creator>
            <pubDate>Tue, 07 Feb 2023 01:06:10 GMT</pubDate>
            <atom:updated>2023-02-07T01:06:10.174Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RT7DoahKDT9GlLksthGDcw.png" /><figcaption>Reference: <a href="https://blog.google/technology/ai/bard-google-ai-search-updates/">An important next step on our AI journey</a></figcaption></figure><p>ChatGPT has taken the world by storm with its ability to answer a wide range of questions and hold conversations with ease. This development sparked a heated debate among tech enthusiasts, some claiming that ChatGPT’s information retrieval and summary capabilities could potentially mark the end of Google’s reign as the dominant search engine.</p><p>However, Google is not one to be left behind and the tech giant will soon release its own conversational AI service, Bard, designed to compete with ChatGPT and reassert its position as the king of search.</p><p>Bard is set to be built using Google’s latest and greatest AI technologies, based on LaMDA, a large AI language model for dialogue applications. LaMDA’s goal is to combine the world’s knowledge with the power and intelligence of large language models, providing fresh and high-quality responses to user queries. The service will be initially released with a lightweight model version of LaMDA, which requires less computing power and will be tested with trusted testers before being made more widely available to the public.</p><p>So, what sets Bard apart from ChatGPT? For starters, Bard is designed specifically as an intelligent search functional conversational chatbot, whereas ChatGPT is a general-purpose language model. This means that Bard has a singular focus on providing relevant answers to user queries, whereas ChatGPT can do much more, but might not always be as focused. Additionally, Bard, powered by LaMDA, will have access to real-time information from the web, giving it the ability to provide fresh and up-to-date answers to user queries. In comparison, ChatGPT was trained on a dataset that only goes up to 2021, so it cannot access the latest information available on the web.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6wqIrzlnVPgroKq3BsWIrg.jpeg" /><figcaption>Reference: <a href="https://blog.google/technology/ai/bard-google-ai-search-updates/">An important next step on our AI journey</a></figcaption></figure><p>The AI war between the tech giants has only just begun. Both Google and OpenAI are looking to bring the benefits of AI into their everyday products, and the search space is the first battleground. Google aims to deepen its understanding of information and turn it into useful knowledge more efficiently, and soon, AI-powered features will be introduced to its search that distill complex information and multiple perspectives into easy-to-digest formats. Meanwhile, OpenAI is looking to continue pushing the boundaries of what is possible with language models.</p><p>So, the question remains: Who will come out on top in the AI war? Only time will tell, but one thing is for sure, the future of AI is going to be shaped by the outcome of this battle between ChatGPT and Bard. The impact of this AI war on the future of technology and search will be significant and far-reaching.</p><p>References:</p><ol><li>An important next step on our AI journey: <a href="https://blog.google/technology/ai/bard-google-ai-search-updates/">https://blog.google/technology/ai/bard-google-ai-search-updates/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cf28d343e54" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>