<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vinod Polinati on Medium]]></title>
        <description><![CDATA[Stories by Vinod Polinati on Medium]]></description>
        <link>https://medium.com/@vinodpolinati?source=rss-a4e64f23935f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*YdWel4PTIf35YxtlGaa-OA.gif</url>
            <title>Stories by Vinod Polinati on Medium</title>
            <link>https://medium.com/@vinodpolinati?source=rss-a4e64f23935f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 01:59:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vinodpolinati/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Recursive Language Models: Could This Be the Real Fix for Long-Context AI in 2026?]]></title>
            <link>https://medium.com/@vinodpolinati/recursive-language-models-could-this-be-the-real-fix-for-long-context-ai-in-2026-070328df9329?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/070328df9329</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[recursive-language-models]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Wed, 14 Jan 2026 15:12:53 GMT</pubDate>
            <atom:updated>2026-01-14T15:12:53.860Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wNUcdOpr4oyj7IC4.png" /><figcaption><strong>A recursive language model (RLM) maps text to text, using recursive queries to itself or other LMs to handle massive contexts efficiently.</strong></figcaption></figure><p>For the past couple of years, the big story in LLMs has been <strong>bigger context windows</strong>. We went from 8K tokens → 128K → 1M → claims of 10M+ tokens. Models like Gemini, GPT-5 series, Claude, and Qwen boasted they could swallow entire books, massive codebases, or years of conversation history in one gulp.</p><p>But here’s the uncomfortable truth most of us have experienced: <strong>even the biggest windows rot</strong>.</p><p>Put a complex, multi-hop question into a 500K+ token prompt, and suddenly the model starts forgetting key details, hallucinating, or giving shallow answers. Researchers call it <strong>“context rot”</strong> or the <strong>“lost in the middle”</strong> effect. Performance drops sharply as length increases — even on tasks where simple “needle-in-a-haystack” retrieval still looks perfect.</p><p>In late 2025, MIT CSAIL researchers Alex L. Zhang, Tim Kraska, and Omar Khattab dropped a paper (arXiv:2512.24601, Dec 31, 2025) that flips the script entirely: <strong>Recursive Language Models (RLMs)</strong>.</p><p>Instead of forcing the model to process everything in one giant forward pass, RLMs let the LLM treat the massive input as an <strong>external, programmable object</strong> — and recursively call <strong>itself</strong> (or sub-instances) on only the relevant pieces.</p><p>Prime Intellect is already calling it <strong>“the paradigm of 2026”</strong>. Here’s why this might actually matter more than the next 10× context-size jump.</p><h3>The Real Problem: Context Rot Is Deeper Than We Thought</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h7zr7avnkSuKQr1O08Ytog.png" /><figcaption>clean performance drop chart showing context rot (classic needle-in-haystack style decline)</figcaption></figure><p>Long-context benchmarks like classic Needle-in-a-Haystack often report near-100% accuracy because they’re simple keyword matches. But real tasks are messier:</p><ul><li>Multi-hop reasoning across documents</li><li>Aggregating insights from hundreds of entries</li><li>Understanding massive code repos</li><li>QA over long transcripts or logs</li></ul><p>Even frontier models show 40–80% degradation on these as context grows. Recent studies (2025) confirm: performance drops with length alone — even when the model perfectly retrieves the relevant info. Attention dilutes, positional biases creep in, and the model just… gets confused.</p><p>Bigger windows help a bit, but they’re expensive (quadratic compute) and hit hard limits.</p><h3>Recursive Language Models: Let the LLM Be Its Own Programmer</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xECgEUpQObGN7iqY9CkG3A.png" /><figcaption>How an RLM works: The root model interacts with a persistent Python REPL, slices the massive prompt, and recursively delegates to sub-instances (from the original RLM paper/blog).</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rnAxMlxSDKyXJAp54UIQKg.png" /><figcaption>Real RLM code executions: Probing with regex/keywords, batch classification, and stitching recursive outputs (examples from the paper).</figcaption></figure><p>The breakthrough insight is surprisingly elegant:</p><blockquote><em>Don’t dump the entire 10M-token document into the prompt. Load it as a string variable in a persistent Python REPL. Let the model write code to peek, slice, search, filter — and most importantly — </em><strong><em>recursively spawn sub-calls to itself</em></strong><em> on tiny, focused chunks.</em></blockquote><p>It’s like giving your LLM infinite external memory and the ability to delegate to “helper versions” of itself.</p><p><strong>High-Level Flow (Simplified):</strong></p><ol><li>Huge input lands as a variable big_data in a stateful Python REPL.</li><li>The “root” LLM gets a clean, short prompt: the user query + instructions about the REPL.</li><li>Root decides: “I need to find sections mentioning X” → writes Python (regex, string ops, etc.) → REPL executes.</li><li>Root sees results → if too broad/complex, it spawns a <strong>recursive sub-LLM call</strong> with just the extracted snippet + a focused sub-query.</li><li>Sub-LLMs bubble results back up (they can recurse further if needed).</li><li>Root aggregates everything and delivers the final answer.</li></ol><p>The root LLM’s own context stays tiny and high-quality — it never sees the full haystack.</p><p>Here’s a simple text-based diagram to visualize the recursion tree:</p><pre>User Query + 10M-Token Input<br>          ↓<br>[Root LLM – depth 0]   ← Short context only<br>   │<br>   ├─ Writes &amp; executes Python: search / extract relevant chunks<br>   │<br>   ↓ Gets results<br>          │<br>   ├─ Done? → Final Answer<br>   │<br>   └─ Needs deeper dive → Spawns<br>                   ↓<br>              [Sub-LLM – depth 1]   ← Only 4K–8K relevant tokens<br>                   │<br>                   ├─ Further slices / analyzes → Spawns Sub-Sub-LLM (depth 2)<br>                   │<br>                   ↓ Results return upward<br>          │<br>[Root synthesizes clean, verified answer]</pre><h3>Why RLMs Crush Traditional Long-Context Approaches</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7I4ww7LomzERTsarfHnNKQ.png" /></figure><p>Quick comparison table (based on the paper’s experiments with GPT-5, Qwen models, and benchmarks like BrowseComp+, OOLONG, CodeQA):</p><p>FeatureGiant Context Window LLMRecursive Language Model (RLM)Max Practical Input1M–10M tokens (with rot)Effectively unbounded (100×+ beyond window)Degradation with LengthSevere (context rot)Minimal — root context stays small &amp; focusedCost per QueryHigh (full context every time)Often lower (tiny sub-calls)Reasoning QualityDiluted attentionSharp, step-by-step decompositionImplementationRequires model retrainingPure inference wrapper (works on any API)Bonus Emergent AbilitiesLimitedSelf-verification, noise filtering, recursion</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZoLgu1jVrnk4uYAzbhZPVQ.png" /></figure><p>The paper shows RLMs handling 100× longer inputs while <strong>outperforming</strong> base models, summary agents, and other scaffolds — sometimes 2–3× better quality at similar/lower cost.</p><h3>The Bigger Vision: Smarter Agents, Not Just Bigger Memory</h3><p>RLMs aren’t just a long-context hack. They’re a step toward models that <strong>actively manage their cognition</strong> — decomposing problems, verifying steps, delegating subtasks, and scaling to video, lifelong memory, or complex agents.</p><p>Prime Intellect is already building production tools around RLMs (RLMEnv) and betting big: training models natively for recursive behavior could unlock the next leap in long-horizon agents.</p><h3>So… Is This the Post-Scaling Era?</h3><p>We spent years chasing bigger models and bigger contexts. Now the smartest path forward might be <strong>smarter inference-time thinking</strong>.</p><p>Recursive Language Models remind us: true intelligence isn’t about cramming more into one glance. It’s about knowing <strong>what to look at</strong>, <strong>when</strong>, and <strong>how deeply</strong>.</p><p>In 2026, we might stop bragging about context-window size… and start asking: <em>How recursively smart is your model?</em></p><p>What do you think — will RLMs replace giant windows, or just complement them? Have you tried anything similar in your projects?</p><p>Drop a comment below, clap if this sparked ideas, and follow for more breakdowns of bleeding-edge AI research!</p><p><strong>Further Reading:</strong></p><ul><li>Full Paper: <a href="https://arxiv.org/abs/2512.24601?referrer=grok.com">Recursive Language Models (arXiv:2512.24601)</a></li><li>Alex Zhang’s Original Blog (Oct 2025): <a href="https://alexzhang13.github.io/blog/2025/rlm/?referrer=grok.com">Recursive Language Models</a></li><li>Prime Intellect: <a href="https://www.primeintellect.ai/blog/rlm?referrer=grok.com">“Recursive Language Models: the paradigm of 2026”</a></li><li>GitHub Codebase: <a href="https://github.com/alexzhang13/rlm?referrer=grok.com">alexzhang13/rlm</a></li></ul><p>#AI #LargeLanguageModels #RecursiveAI #LongContext #MITResearch #FutureOfAI</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=070328df9329" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Built an End-to-End LLM Orchestration Engine]]></title>
            <link>https://medium.com/@vinodpolinati/how-i-built-an-end-to-end-llm-orchestration-engine-b86e429ba717?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b86e429ba717</guid>
            <category><![CDATA[full-stack]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Sat, 27 Dec 2025 03:32:23 GMT</pubDate>
            <atom:updated>2025-12-27T03:32:23.846Z</atom:updated>
            <content:encoded><![CDATA[<p>I built a compact orchestration engine to run structured interviews with an LLM, capture transcripts, and provide both a quick Python frontend and a React UI. This post documents the architecture, implementation details, prompt design, run instructions, and lessons learned — enough to reproduce or adapt the system for your own needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t7Qbef69i6vN0CrxWOhweg.png" /></figure><h3>Repo layout</h3><p>Git Repo : <a href="https://github.com/vinod-polinati/llm-orchestration-engine"><strong>github.com/vinod-polinati/llm-orchestration-engine</strong></a></p><ul><li>main.py — entrypoint that wires components and starts the chosen frontend.</li><li>llm_service.py — LLM adapter (API wrapper, retries, response normalization).</li><li>interview_engine.py — orchestration logic: interview flow, turns, and metadata capture.</li><li>prompts.py — centralized prompt templates and few-shot examples.</li><li>frontend.py — lightweight Python/CLI UI for quick experimentation.</li><li>frontend-react/ — React app for a more polished interview experience.</li><li>save_transcript.py — utilities to persist session transcripts into transcripts/.</li><li>transcripts/ — saved session JSON files for replay and analysis.</li><li>requirements.txt and Readme.md — dependencies and run instructions.</li></ul><h3>High-level architecture</h3><ul><li>Backend orchestration (interview flows and state).</li><li>LLM adapter (abstracts the model provider and handles retries/rate-limit).</li><li>Frontends: quick Python CLI and a React-based UI.</li><li>Simple persistence: transcript JSON files for each session.</li></ul><p>Components are intentionally small and focused so each can be iterated on independently:</p><ul><li>llm_service.py only handles API calls, keys, and backoff/retries.</li><li>interview_engine.py models the interview and logs every turn with metadata.</li><li>prompts.py contains reusable fragments so prompt A/B tests are easy.</li></ul><h3>Architecture diagram</h3><p>Below is a high-level architecture diagram showing the main components and interactions. Replace this Mermaid block with an exported image if you prefer a PNG/SVG in an assets/ folder.</p><pre>graph TD<br>  subgraph Frontends<br>    A[Python CLI]<br>    B[React UI]<br>  end<br>  C[Interview Engine]<br>  D[LLM Service]<br>  E[Transcripts / Storage]</pre><pre>  A --&gt;|user actions| C<br>  B --&gt;|user actions| C<br>  C --&gt;|prompts| D<br>  D --&gt;|responses| C<br>  C --&gt;|save session| E</pre><h3>Backend implementation (core ideas)</h3><h3>Orchestration: interview_engine.py</h3><ul><li>Defines interview stages and question lists.</li><li>Assembles prompts using templates from prompts.py.</li><li>Calls the LLM adapter and records structured turn data (raw prompt, model output, normalized output, timestamps, prompt version).</li></ul><p>This separation keeps the engine deterministic and easy to test: the engine decides what to ask next and the llm_service only answers.</p><h4>Sequence diagram — orchestration flow</h4><p>This sequence diagram shows a typical turn: the engine builds a prompt, sends it to the LLM, receives a response, and persists the turn.</p><pre>sequenceDiagram<br>  participant User<br>  participant Engine as Interview Engine<br>  participant LLM as LLM Service<br>  participant Storage as Transcript Store</pre><pre>  User-&gt;&gt;Engine: answer / trigger next question<br>  Engine-&gt;&gt;LLM: send prompt (from `prompts.py`)<br>  LLM--&gt;&gt;Engine: response (text + metadata)<br>  Engine-&gt;&gt;Storage: save turn (raw prompt + response + meta)<br>  Engine--&gt;&gt;User: display response / next question</pre><h3>LLM adapter: llm_service.py</h3><ul><li>Abstracts provider details (API base URL, API keys via environment variables).</li><li>Implements exponential backoff and error handling for rate limits or transient failures.</li><li>Returns structured responses including text and optional usage metadata so the engine can log cost/latency information.</li></ul><p>Design note: keeping provider logic isolated makes switching providers or adding local models straightforward.</p><h3>Entrypoint: main.py</h3><ul><li>Reads configuration and environment variables (API keys, model names, prompt variants).</li><li>Instantiates llm_service and interview_engine.</li><li>Starts whichever frontend you prefer (CLI via frontend.py or the React app during development).</li></ul><h3>Prompt engineering (prompts.py)</h3><ul><li>Centralize system prompts, question templates, and few-shot examples in prompts.py.</li><li>Keep prompt fragments modular so you can A/B test pieces independently.</li><li>Version your prompts (timestamp or version tag) and include that metadata in each saved transcript so you can trace outputs back to prompt text.</li></ul><p>Principles used:</p><ul><li>Provide a clear system role and response format constraints.</li><li>Supply minimal context needed per turn, not the entire session every time (unless required).</li><li>Prefer structured output instructions (JSON) when post-processing is important.</li></ul><h4>Prompt composition diagram</h4><p>Use this diagram to visualize how a final prompt is composed from templates, system instructions, and context.</p><pre>flowchart LR<br>  S[System Prompt]<br>  C[Context / Session State]<br>  Q[Question Template]<br>  F[Few-shot Examples]<br>  P[Final Prompt]</pre><pre>  S --&gt; P<br>  C --&gt; P<br>  Q --&gt; P<br>  F --&gt; P</pre><h3>Frontends</h3><h3>Python frontend (frontend.py)</h3><ul><li>A minimal, dependency-light runner to start an interview from the terminal.</li><li>Great for quick iterations and debugging prompts without building the React app.</li></ul><h3>React frontend (frontend-react)</h3><ul><li>A more polished, interactive UI that can stream and render responses as tokens arrive.</li><li>Code lives under frontend-react/src (entrypoint: App.tsx).</li><li>The React app can either call backend HTTP endpoints (if you add them) or be used alongside the Python engine during development.</li></ul><h4>Frontend interaction diagram</h4><pre>graph LR<br>  UIReact[React UI]<br>  UIPy[Python CLI]<br>  Engine[Interview Engine]<br>  API(API Server - optional)</pre><pre>  UIReact --&gt;|WS/HTTP| API<br>  UIPy --&gt;|local function| Engine<br>  API --&gt;|calls| Engine</pre><h3>Transcripts &amp; storage</h3><ul><li>save_transcript.py writes session JSON files into transcripts/.</li><li>Each transcript contains:</li><li>Session metadata (timestamp, prompt versions, model name).</li><li>Turn-level entries: speaker (user/system/LLM), raw prompt, raw model output, normalized output, timestamps.</li></ul><p>Saving everything (raw prompt + response + metadata) makes debugging, evaluation, and improvement much easier.</p><h4>Storage diagram</h4><p>If you later scale, swap file-based storage for object storage or a database as shown below.</p><pre>graph TD<br>  Engine --&gt;|writes| LocalFiles[transcripts/*.json]<br>  Engine --&gt;|writes| S3[(S3 / Blob storage)]<br>  LocalFiles --&gt;|backup| S3<br>  Analytics[Analytics / Dashboard] --&gt;|reads| S3</pre><h3>Running locally</h3><p>Quick start:</p><pre>python -m venv .venv<br>source .venv/bin/activate<br>pip install -r requirements.txt<br>python main.py</pre><p>To run the front-end React developer server:</p><pre>cd frontend-react<br>npm install<br>npm start</pre><p>Adjust environment variables for your API keys or model configuration before running the backend.</p><h3>Testing and deployment notes</h3><ul><li>Tests are lightweight; much of the validation is manual prompt iteration plus inspection of transcripts.</li><li>For production, consider:</li><li>Replacing local transcript files with a durable store (S3, database).</li><li>Adding authentication and HTTPS for any HTTP endpoints.</li><li>CI for linting and tests, and CD for deploying the React build and backend.</li></ul><h3>Challenges &amp; solutions</h3><ul><li>Prompt drift / non-determinism: record prompt versions and enforce strict response schemas when required.</li><li>Rate limits and transient failures: backoff + retries implemented in llm_service.py.</li><li>Streaming UX: buffer tokens and progressively render in the React UI.</li></ul><h3>Lessons learned</h3><ul><li>Keep prompts separate from orchestration code — it makes iteration far faster.</li><li>Small, well-defined components (engine, adapter, UI) simplify testing and refactors.</li><li>Persist full context (prompt text + metadata) with transcripts; it’s invaluable for debugging.</li></ul><h3>Next steps</h3><ul><li>Add HTTP API endpoints so the React UI can communicate with the backend in production.</li><li>Add metrics (latency, token usage) and a small dashboard.</li><li>Expand automated tests for engine flows and edge cases.</li></ul><h3>Conclusion</h3><p>This project demonstrates how a compact set of focused modules — llm_service.py, interview_engine.py, prompts.py, and small frontends — can produce a complete end-to-end LLM orchestration system that’s easy to iterate on. If you want, I can convert this into a publish-ready blog post with images and an architectural diagram, or prepare a README update with step-by-step run instructions.</p><p>Files referenced in this post:</p><ul><li><a href="https://medium.com/frontend.py">frontend.py</a></li><li><a href="https://medium.com/frontend-react/src/App.tsx">frontend-react/src/App.tsx</a></li><li><a href="https://medium.com/llm_service.py">llm_service.py</a></li><li><a href="https://medium.com/interview_engine.py">interview_engine.py</a></li><li><a href="https://medium.com/prompts.py">prompts.py</a></li><li><a href="https://medium.com/save_transcript.py">save_transcript.py</a></li><li><a href="https://medium.com/main.py">main.py</a></li><li><a href="https://medium.com/requirements.txt">requirements.txt</a></li><li><a href="https://medium.com/Readme.md">Readme.md</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b86e429ba717" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chatterbox: The Leading Family of Open-Source AI Voice Models That’s Changing the Game]]></title>
            <link>https://medium.com/@vinodpolinati/chatterbox-the-leading-family-of-open-source-ai-voice-models-thats-changing-the-game-150991d5cf8d?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/150991d5cf8d</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[resemble-ai]]></category>
            <category><![CDATA[ai-voice-model]]></category>
            <category><![CDATA[text-to-speech]]></category>
            <category><![CDATA[chatterbox]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Wed, 24 Dec 2025 20:50:10 GMT</pubDate>
            <atom:updated>2025-12-24T20:55:53.822Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>How Chatterbox Is Redefining Open-Source AI Voice</strong></h3><p><em>How a small team at Resemble AI built a text-to-speech model that outperforms ElevenLabs — and made it completely free</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*x9Lt4kW4btb1gPjh.jpg" /></figure><p>The text-to-speech landscape just experienced a seismic shift. In an industry dominated by expensive proprietary solutions like ElevenLabs and Google Cloud TTS, a scrappy three-person team at Resemble AI released something that shouldn’t exist: an open-source voice model that’s not only competitive with commercial offerings but actually outperforms them in blind tests.</p><p>Meet <strong>Chatterbox</strong> — the MIT-licensed, production-grade TTS model that’s exploded to over 1 million downloads on Hugging Face and 11,000+ GitHub stars in just months. If you haven’t heard about it yet, you’re about to discover why developers, content creators, and enterprises are abandoning their paid subscriptions.</p><h3>📊 Quick Stats That Matter</h3><blockquote>According to blind evaluation data released by Resemble AI, Chatterbox achieved a <strong>63.75% user preference rate</strong> when pitted against ElevenLabs. While manufacturer-supplied benchmarks should always be approached with skepticism, independent community testing confirms that Chatterbox produces comparable “smoothness” and stability, particularly in long-form content where other open-source models often degrade.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4E1Xhyvfv1VlZjlr" /></figure><h3>The Model Family</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cQuLxmJ4NIhJ7969RtI5Zw.png" /></figure><h3>The Feature Set That Matters</h3><h3>1. Zero-Shot Voice Cloning</h3><pre>┌──────────────────────────────────────────────────┐<br>│         VOICE CLONING WORKFLOW                   │<br>└──────────────────────────────────────────────────┘<br><br>5-10 seconds          Chatterbox         Your Text<br>Reference Audio   →   Processing    →    + Cloned Voice<br>     <br>[Audio Wave]          [🧠 AI Model]      [New Audio Wave]<br>     <br>✅ No training needed<br>✅ Instant results<br>✅ High fidelity</pre><p>Give Chatterbox just 5–10 seconds of audio, and it can clone any voice. No training required. No fine-tuning needed. Just upload a reference clip and start generating speech that sounds remarkably like the target voice.</p><p>This is transformative. Where other solutions require extensive training data or paid subscriptions to clone voices, Chatterbox does it instantly and for free.</p><h3>2. Emotion Exaggeration Control</h3><p>Here’s where Chatterbox gets genuinely innovative. It’s the first open-source TTS model to offer fine-grained emotion control through an exaggeration parameter:</p><pre>EMOTION SCALE<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>0.0 ────────────────────────────────────<br>    😐 Flat, monotone<br>    &quot;Hello. Welcome.&quot;<br><br>0.5 ────────────────────────────────────<br>    🙂 Natural, conversational (DEFAULT)<br>    &quot;Hey! Welcome back.&quot;<br><br>1.0+ ───────────────────────────────────<br>    🎭 Dramatic, theatrical<br>    &quot;WOW! Welcome!! So great to see you!&quot;</pre><ul><li><strong>0.5</strong>: Natural, conversational tone (default)</li><li><strong>1.0+</strong>: Dramatically expressive, theatrical delivery</li></ul><p>This level of control is unprecedented in open-source TTS. Want your AI assistant to sound more enthusiastic? Dial up the exaggeration. Need neutral, professional narration? Dial it down. This granularity enables use cases that were previously impossible without expensive voice actors.</p><h3>3. Paralinguistic Tags (Turbo)</h3><p>Chatterbox Turbo introduces native support for non-speech sounds through simple tags:</p><pre>&quot;Hi there [chuckle], thanks for calling. Let me [pause] check that for you.&quot;</pre><p><strong>Available Tags:</strong></p><pre>[laugh]    [cough]    [chuckle]    [sigh]    [gasp]    [pause]<br>  😄         🤧          😊          😮‍💨        😲         ⏸️</pre><p>Tags like [laugh], [cough], [chuckle], [sigh], and [gasp] add natural realism to generated speech. This makes conversations feel genuinely human rather than robotic—a crucial feature for building believable AI agents.</p><h3>4. Ultra-Low Latency</h3><pre>LATENCY COMPARISON<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br>Traditional TTS:  ████████████████ 1000-2000ms<br>ElevenLabs:       ████████ 500-800ms<br>Chatterbox:       ██ &lt;200ms ⚡<br>Chatterbox Turbo: █ 472ms (first chunk)<br>Real-time Factor: 0.499 (faster than real-time!)</pre><p>With sub-200ms latency, Chatterbox enables truly real-time applications. Voice assistants, live dubbing, interactive gaming — all become viable use cases. The Turbo variant achieves a remarkable real-time factor of 0.499 on an RTX 4090, with first-chunk latency around 472ms.</p><h3>5. Built-in Watermarking</h3><p>Every audio file generated by Chatterbox includes Resemble AI’s PerTh (Perceptual Threshold) watermarking — imperceptible neural watermarks that survive MP3 compression and audio editing while maintaining nearly 100% detection accuracy. This isn’t just a feature; it’s responsible AI deployment in practice.</p><p>The watermarker operates on psychoacoustic principles, exploiting how humans perceive audio by encoding data into inaudible frequency regions. When speech produces peaks at specific frequencies, PerTh embeds structured tones within a few hertz that remain imperceptible to listeners while being robust against removal attempts.</p><h3>The Benchmarks That Matter</h3><p>Let’s talk numbers. In blind evaluations conducted through Podonos, <strong>63.75% of evaluators preferred Chatterbox over ElevenLabs</strong> when comparing speech naturalness and quality. That’s not just competitive — that’s dominant.</p><pre>BLIND LISTENING TEST RESULTS<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br>Chatterbox:     ████████████████████████████ 63.75%<br>ElevenLabs:     ███████████████ 36.25%<br>✅ Higher naturalness<br>✅ Better prosody<br>✅ More authentic emotion</pre><h3>Performance Metrics:</h3><ul><li><strong>Training Data</strong>: 500K hours of cleaned speech</li><li><strong>Latency</strong>: Sub-200ms for real-time applications</li><li><strong>Voice Cloning</strong>: 5–10 second reference clips</li><li><strong>Languages</strong>: 23+ languages with cross-lingual capabilities</li><li><strong>Model Sizes</strong>: 350M (Turbo) to 500M (Original) parameters</li><li><strong>License</strong>: MIT (completely free for commercial use)</li></ul><h3>Head-to-Head Comparisons:</h3><pre>QUALITY COMPARISON CHART<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>Chatterbox        ████████████████████ 95/100<br>ElevenLabs Turbo  ██████████████████ 90/100<br>Cartesia Sonic    █████████████████ 85/100<br>VibeVoice 7B      ████████████████ 80/100<br>XTTS-v2           ██████████████ 75/100<br>Recent benchmarks show Chatterbox Turbo matching or exceeding:</pre><ul><li>ElevenLabs Turbo v2.5 (speed and quality)</li><li>Cartesia Sonic 3 (expressiveness)</li><li>VibeVoice 7B (naturalness)</li></ul><p>The model maintains high intelligibility, natural prosody, and emotional expressiveness while being completely free and self-hostable.</p><h3>The Competition: Where Does Chatterbox Stand?</h3><p>The open-source TTS landscape has exploded in 2025. Let’s compare Chatterbox against the major players:</p><pre>Model           License    Cloning  Speed   Quality  Maintenance<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br>Chatterbox      MIT        ✅ Yes   ⚡⚡⚡   ⭐⭐⭐⭐⭐  🟢 Active<br>XTTS-v2         Non-Comm   ✅ Yes   ⚡⚡    ⭐⭐⭐⭐   🟡 Community<br>MeloTTS         MIT        ❌ No    ⚡⚡⚡   ⭐⭐⭐    🟢 Active<br>Bark            Non-Comm   ❌ No    ⚡      ⭐⭐⭐    🟡 Slow<br>Fish Audio      Non-Comm   ✅ Yes   ⚡⚡    ⭐⭐⭐⭐⭐  🟢 Active<br>Kokoro          Apache     ❌ No    ⚡⚡⚡   ⭐⭐     🟢 Active</pre><h3>The Pros: Why Developers Are Choosing Chatterbox</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BSNsoCFFs8YcoCab" /></figure><pre>WHY CHATTERBOX WINS<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>✅ MIT License          → Use anywhere, no restrictions<br>✅ 63.75% Win Rate      → Better than ElevenLabs<br>✅ 6 Lines of Code      → Dead simple integration<br>✅ Active Development   → Regular updates from Resemble AI<br>✅ Multi-Platform       → CUDA, ROCm, Apple Silicon, CPU<br>✅ Built-in Ethics      → Automatic watermarking<br>✅ 1M+ Downloads        → Proven in production</pre><h3>1. Licensing Freedom</h3><p>MIT license means zero restrictions. Use it commercially, modify it, integrate it into your products — no royalties, no usage caps, no vendor lock-in.</p><h3>2. Production-Ready Quality</h3><p>This isn’t a research prototype. Chatterbox has been benchmarked against commercial systems and consistently wins. The audio quality is genuinely impressive — natural intonation, appropriate pauses, emotional depth.</p><h3>3. Simple Implementation</h3><p>Six lines of Python code to generate speech:</p><pre>import torchaudio as ta<br>from chatterbox.tts import ChatterboxTTS<br><br>model = ChatterboxTTS.from_pretrained(device=&quot;cuda&quot;)<br>text = &quot;Hello, world!&quot;<br>wav = model.generate(text)<br>ta.save(&quot;output.wav&quot;, wav, model.sr)</pre><p>No complex configuration. No wrestling with dependencies. It just works.</p><h3>4. Comprehensive Documentation</h3><p>From basic usage to advanced voice conversion, the documentation is thorough and developer-friendly. The community has created extensive guides, Docker images, and integration examples.</p><h3>5. Active Development</h3><p>Unlike many open-source projects that fade after initial release, Chatterbox is actively maintained by Resemble AI. Regular updates, bug fixes, and new features keep it at the cutting edge.</p><h3>6. Hardware Flexibility</h3><p>Runs on NVIDIA GPUs (CUDA), AMD GPUs (ROCm), Apple Silicon (MPS), and even CPUs. The Turbo variant is optimized for efficiency, making it accessible even on modest hardware.</p><h3>7. Ethical AI Features</h3><p>Built-in watermarking demonstrates a commitment to responsible AI development. You can deploy with confidence, knowing generated content can be traced and verified.</p><h3>The Cons: What to Watch Out For</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mzRHKTQq_ASK0bGt" /></figure><pre>⚠️  THINGS TO KNOW<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>❗ Character Limits     → Best under 1-2K chars per generation<br>❗ English-Optimized    → Other languages work but less polished<br>❗ GPU Recommended      → CPU works but slower<br>❗ 40-sec Max Duration  → Need chunking for longer content<br>❗ Reference Quality    → Clean audio = better results<br>❗ Emotion Tuning       → Requires experimentation<br>❗ Mandatory Watermark  → Can&#39;t be disabled</pre><h3>1. Character Limits for Optimal Quality</h3><p>Like most voice cloning models, Chatterbox works best with inputs under 1,000–2,000 characters. Longer texts may experience voice drift or speed inconsistencies. The solution is chunking — split long texts into smaller segments and concatenate the audio outputs.</p><h3>2. English-Optimized Performance</h3><p>While Chatterbox Multilingual supports 23 languages, the most natural and expressive results come from English inputs. Other languages work but may lack some of the nuance of the English variant.</p><h3>3. Hardware Requirements</h3><p>While it can run on CPU, practical production use requires a GPU. The original model needs around 8–16GB VRAM; Turbo is more efficient but still benefits from GPU acceleration. This isn’t unique to Chatterbox — it’s true of all modern TTS models.</p><h3>4. Limited Duration Per Generation</h3><p>Single generations are typically capped at 40 seconds for best quality. This is manageable with proper chunking strategies but adds complexity for audiobook-length content.</p><h3>5. Voice Reference Quality Matters</h3><p>The quality of your reference audio directly impacts output quality. Clean, clear recordings of 10+ seconds produce the best results. Background noise or poor recording quality will degrade the cloned voice.</p><h3>6. Emotion Control Requires Experimentation</h3><p>While the emotion exaggeration parameter is powerful, finding the right settings for your use case requires trial and error. The default 0.5 works well for most scenarios, but specific applications may need tuning.</p><h3>7. Watermarking Can’t Be Disabled</h3><p>If you need audio without watermarks for specific applications, this is a non-negotiable feature. It’s designed for responsible AI deployment, which is admirable, but may not fit every use case.</p><h3>Real-World Applications</h3><p>Developers are already building impressive applications with Chatterbox:</p><pre>🎬 CONTENT CREATION<br>   └─ YouTubers generating voiceovers in minutes<br>   └─ 80,000-word novel narrated overnight<br>   └─ Podcast production automation</pre><pre>🎮 GAMING<br>   └─ Dynamic NPC dialogue<br>   └─ Character emotion responses<br>   └─ Real-time voice interactions</pre><pre>📚 ACCESSIBILITY<br>   └─ Multilingual educational content<br>   └─ Course material generation<br>   └─ Audio descriptions</pre><pre>📞 CUSTOMER SERVICE<br>   └─ Natural-sounding voice agents<br>   └─ Emotionally appropriate responses<br>   └─ 24/7 automated support</pre><pre>📖 AUDIOBOOK PRODUCTION<br>   └─ Professional-quality at low cost<br>   └─ Multiple voice characters<br>   └─ Rapid production cycles</pre><p><strong>Content Creation</strong>: YouTubers and podcasters are generating voiceovers in minutes instead of hours. One user generated narration for an entire 80,000-word novel overnight.</p><p><strong>Gaming</strong>: Game studios are using Chatterbox for dynamic NPC dialogue, with paralinguistic tags adding authentic character reactions.</p><p><strong>Accessibility</strong>: Educational platforms are creating course materials in multiple languages with consistent voice quality.</p><p><strong>Customer Service</strong>: Companies are deploying voice agents with natural-sounding responses and appropriate emotional tone.</p><p><strong>Audiobook Production</strong>: Independent authors are producing professional-quality audiobooks at a fraction of traditional costs.</p><h3>Getting Started</h3><p>Installation is straightforward:</p><pre># Clone the repository<br>git clone https://github.com/resemble-ai/chatterbox.git<br>cd chatterbox</pre><pre># Install dependencies<br>pip install -e .</pre><p><strong>Basic Usage (6 lines!):</strong></p><pre>import torchaudio as ta<br>from chatterbox.tts import ChatterboxTTS<br><br>model = ChatterboxTTS.from_pretrained(device=&quot;cuda&quot;)<br>text = &quot;Hello, world!&quot;<br>wav = model.generate(text)<br>ta.save(&quot;output.wav&quot;, wav, model.sr)</pre><pre>DEPLOYMENT OPTIONS<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>🐍 Python SDK        → Direct integration<br>🐳 Docker Images     → One-command deployment  <br>🌐 Web UI (Gradio)   → Browser-based testing<br>🚀 FastAPI Server    → Production API<br>☁️  Hugging Face     → Instant demo (no install)</pre><p>For those wanting a complete API server with web UI, several community projects provide turnkey solutions:</p><ul><li><strong>Chatterbox-TTS-Server</strong> by devnen: Full-featured FastAPI server with Gradio UI</li><li><strong>Chatterbox-TTS-Extended</strong> by petermg: Advanced pipeline with batch processing</li><li>Ready-to-use Docker images for one-command deployment</li></ul><p>The Hugging Face Spaces demo lets you test Chatterbox instantly in your browser without any installation.</p><h3>The Bigger Picture</h3><p>Chatterbox represents something significant: the democratization of professional voice AI. For years, high-quality speech synthesis was locked behind expensive APIs and restrictive licenses. Startups paid thousands monthly. Individual developers couldn’t compete.</p><pre>THE SHIFT IN VOICE AI<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>BEFORE                      AFTER (Chatterbox)<br>💰 $1000s/month            → 💸 $0<br>🔒 Proprietary             → 🔓 Open-source MIT<br>☁️  Cloud-only             → 💻 Run locally<br>📊 Usage limits            → ♾️  Unlimited<br>🎛️  Basic controls         → 🎚️  Fine-grained<br>👔 Enterprise-only         → 👨‍💻 Everyone</pre><p>That era is ending. Chatterbox proves that open-source voice AI can match — and exceed — commercial quality. The implications are profound:</p><ul><li><strong>Innovation accelerates</strong> when anyone can build voice-enabled applications</li><li><strong>Costs plummet</strong> when there are no per-character API fees</li><li><strong>Privacy improves</strong> when processing can happen locally</li><li><strong>Customization expands</strong> when you control the entire stack</li></ul><p>Resemble AI could have kept this proprietary. They could have charged premium prices. Instead, they chose to release it under MIT license and advance the entire field.</p><h3>Looking Forward</h3><p>The voice AI space is evolving rapidly. Chatterbox sits at the intersection of quality, accessibility, and innovation. As the community continues to contribute, we can expect:</p><pre>ROADMAP &amp; FUTURE DEVELOPMENTS<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>🌍 More languages &amp; accents<br>📝 Improved long-form generation  <br>🎭 Better emotion control granularity<br>🎵 Advanced voice mixing &amp; interpolation<br>🔌 Integration with more frameworks<br>🚀 Performance optimizations<br>🤝 Community contributions</pre><p>For now, Chatterbox stands as the best open-source TTS solution for serious applications. It’s not perfect — no software is — but it’s genuinely impressive how much they’ve achieved.</p><h3>Conclusion</h3><p>If you’re building anything that involves synthetic speech, Chatterbox deserves your attention. It’s not just competitive with commercial solutions — in many blind tests, it’s preferred. It’s not just open-source — it’s MIT licensed with no strings attached. It’s not just a research project — it’s production-ready and actively maintained.</p><pre>THE CHATTERBOX ADVANTAGE<br>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━<br><br>✨ Quality:      Beats ElevenLabs 63.75% of the time<br>🆓 Cost:         $0 forever (MIT license)<br>⚡ Speed:        Sub-200ms latency<br>🌍 Reach:        23+ languages<br>🎯 Control:      Fine-grained emotion &amp; expression<br>🔧 Integration:  6 lines of code<br>🚀 Ready:        1M+ downloads, battle-tested</pre><p>The voice AI revolution isn’t coming — it’s here. And it’s open-source.</p><h3>🔗 Try Chatterbox Today</h3><p>ResourceLink :</p><p><strong>GitHub : </strong><a href="https://github.com/resemble-ai/chatterbox">github.com/resemble-ai/chatterbox</a> <br><strong>Hugging Face : </strong><a href="https://huggingface.co/ResembleAI/chatterbox">huggingface.co/ResembleAI/chatterbox</a> <br><strong>Live Demo : </strong><a href="https://huggingface.co/spaces/ResembleAI/Chatterbox">Hugging Face Spaces</a>📚 <br><strong>Documentation : </strong><a href="https://www.resemble.ai/chatterbox/">Resemble AI Website</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=150991d5cf8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Built an Automated Pickleball Rally Splitter Using Python and YOLOv8]]></title>
            <link>https://medium.com/@vinodpolinati/how-i-built-an-automated-pickleball-rally-splitter-using-python-and-yolov8-b890a44650a4?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b890a44650a4</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[spatio-temporal]]></category>
            <category><![CDATA[court-detection]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Tue, 23 Dec 2025 03:32:21 GMT</pubDate>
            <atom:updated>2025-12-23T03:32:21.387Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/816/0*L8uKmaoOhYe7aVnx.png" /></figure><p>Pickleball is exploding in popularity, and as a tech enthusiast, I wanted to bring computer vision to the sport. My goal: automatically split a full match video into individual rallies using AI. Here’s how I built a robust rally detection and video-splitting system, step by step.</p><h3>1. Project Setup</h3><p>Here’s the github repository , follow accordingly : <a href="https://github.com/vinod-polinati/pickleball-rally-detection">https://github.com/vinod-polinati/pickleball-rally-detection</a></p><p>I started by creating a dedicated Python virtual environment to keep dependencies clean:</p><p>I then installed the core libraries:</p><ul><li>opencv-python for video processing</li><li>numpy for numerical operations</li><li>ultralytics for YOLOv8 object detection</li><li>moviepy and matplotlib for future visualization and editing</li></ul><p>All dependencies are listed in requirements.txt for easy setup.</p><h3>2. Object Detection with YOLOv8</h3><p>I used the <a href="https://docs.ultralytics.com/">Ultralytics YOLOv8</a> model, which is state-of-the-art for real-time object detection. I downloaded the yolov8x.pt weights and loaded them in Python using:</p><p>YOLOv8 can detect both people and sports balls, which is perfect for pickleball.</p><h3>3. Video Processing Pipeline</h3><p>The main script, rallysplitv.py, is the heart of the system. Here’s the workflow:</p><ul><li><strong>Load the match video</strong> using OpenCV.</li><li><strong>Frame-by-frame detection:</strong> For each frame, YOLOv8 predicts bounding boxes for players (class 0) and the ball (class 32).</li><li><strong>Ball filtering:</strong> The code filters out false positives (like shoes) and ignores oversized detections.</li><li><strong>Ball tracking:</strong> It tracks the ball’s center position, frame by frame.</li><li><strong>Teleport detection:</strong> If the ball “jumps” too far between frames (likely a missed detection or occlusion), it marks a forced cut.</li><li><strong>Timeline creation:</strong> It builds a timeline array, marking frames where the ball is visible and valid.</li></ul><h3>4. Rally Segmentation Logic</h3><p>The rally detection is based on:</p><ul><li><strong>Continuous ball presence:</strong> A rally is considered active as long as the ball is detected, with a small gap tolerance for brief occlusions.</li><li><strong>Forced cuts:</strong> If the ball “teleports” (moves unrealistically far in one frame), the rally is split.</li><li><strong>Minimum rally duration:</strong> Very short segments are ignored to avoid noise.</li></ul><p>The code then slices the timeline into start/end times for each rally.</p><h3>5. Video Slicing with FFmpeg</h3><p>For each detected rally, the script calls FFmpeg to extract the corresponding video segment:</p><p>All rally clips are saved in the rallies_v18 folder.</p><h3>6. Debugging and Visualization</h3><p>A debug mode overlays detections and cut points on the video, saving a debug_v18.mp4 for visual inspection. This was invaluable for tuning thresholds and logic.</p><h3>7. Running the System</h3><p>To use the system:</p><ol><li>Place your match video as input_match.mp4 in the project folder.</li><li>Run the script:</li><li>Find all rally clips in the rallies_v18 directory.</li></ol><h3>8. Key Learnings</h3><ul><li><strong>YOLOv8</strong> is fast and accurate for sports analytics.</li><li><strong>Careful post-processing</strong> (like filtering out shoes and handling ball occlusions) is crucial for reliable results.</li><li><strong>Automated video slicing</strong> with FFmpeg is efficient and lossless.</li></ul><h3>Conclusion</h3><p>This project shows how modern AI and open-source tools can automate sports video analysis. The code is modular and can be extended for stats, player tracking, or visualization. If you’re interested in computer vision for sports, give it a try!</p><p><em>Questions or want to collaborate? Drop a comment below!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b890a44650a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Real-Time Basketball Pose & Tracking System — Step by Step]]></title>
            <link>https://medium.com/@vinodpolinati/building-a-real-time-basketball-pose-tracking-system-step-by-step-9aa3cda6dee9?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/9aa3cda6dee9</guid>
            <category><![CDATA[spatio-temporal]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[mediapipe]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Sun, 21 Dec 2025 18:11:45 GMT</pubDate>
            <atom:updated>2025-12-22T07:54:16.401Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Basketball Vision Analytics System — Step by Step</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*tfrERWhJ2ReSnr4t.gif" /><figcaption>temporal-action-detection</figcaption></figure><blockquote><em>A practical walkthrough of my open-source basketball analytics pipeline: player/ball detection, tracking, shot/possession logic, and hand analysis.</em></blockquote><blockquote><strong><em>GitHub Repo:</em></strong><em> </em><a href="https://github.com/vinod-polinati/basketball-vision-analytics.git"><em>https://github.com/vinod-polinati/basketball-vision-analytics.git</em></a></blockquote><h3>What is this project?</h3><p>This project is a real-time basketball video analytics system. It detects players and the ball, tracks them across frames, infers shot attempts and possession, and can analyze hand-ball interactions.<br>All code is available here: <a href="https://github.com/vinod-polinati/basketball-vision-analytics.git">GitHub Repo</a>.</p><h3>Codebase Structure</h3><ul><li>src/main.py — Main pipeline: loads video, runs detection, tracking, shot/possession logic, and annotation.</li><li>src/detect.py — Uses YOLOv8 to detect players (class 0) and ball (class 32) in each frame.</li><li>src/tracker.py — Uses Norfair to track players and ball, assigning consistent IDs.</li><li>src/annotate.py — Draws bounding boxes, labels, and shot banners on frames.</li><li>src/mediapipe_hands.py — Uses MediaPipe to detect hands and check ball-hand overlap.</li><li>src/mediapipe_main.py — (Optional) Example of integrating hand detection into the main pipeline.</li><li>yolov8x.pt — YOLOv8 weights (downloaded separately).</li><li>requirements.txt — Python dependencies.</li></ul><h3>Environment Setup</h3><ul><li>Python 3.11 recommended</li><li>Install dependencies:</li></ul><pre>python -m venv myenv<br>source myenv/bin/activate<br>pip install -r requirements.txt</pre><ul><li>Download yolov8x.pt from the <a href="https://github.com/ultralytics/ultralytics/releases">Ultralytics YOLOv8 release page</a> and place it in the project root.</li></ul><h3>How the Pipeline Works</h3><h3>1. Detection (src/detect.py)</h3><ul><li>Loads YOLOv8 model.</li><li>For each frame, detects players and ball:</li><li>Players: class 0</li><li>Ball: class 32</li><li>Returns bounding boxes for both.</li></ul><h3>2. Tracking (src/tracker.py)</h3><ul><li>Uses Norfair to track players and ball across frames.</li><li>Assigns unique IDs to each player and the ball.</li><li>Tracking is based on bounding box center points and configurable distance thresholds.</li></ul><h3>3. Shot &amp; Possession Logic (src/main.py)</h3><ul><li>Tracks ball and player positions over time.</li><li>Determines which player is closest to (or overlapping with) the ball for possession.</li><li>Detects shot attempts by monitoring the ball’s vertical velocity (upward movement) and recent possession.</li><li>Annotates the shooter and displays a shot banner when a shot is detected.</li></ul><h3>4. Annotation (src/annotate.py)</h3><ul><li>Draws bounding boxes and labels for players and the ball.</li><li>Highlights the shooter in red.</li><li>Displays a semi-transparent banner for shot attempts.</li></ul><h3>5. Hand Detection (Optional, src/mediapipe_hands.py)</h3><ul><li>Uses MediaPipe to detect hands in the frame.</li><li>Can check if the ball overlaps with any detected hand (useful for advanced analytics).</li></ul><h3>Example: Main Pipeline (src/main.py)</h3><pre>from detect import detect<br>from tracker import track_players, track_ball<br>from annotate import draw_box, draw_shot_banner</pre><pre># ...<br>players, balls = detect(frame)<br>tracked_players = track_players(players)<br>tracked_ball = track_ball(balls)<br># ...</pre><ul><li>For each frame: detect → track → assign possession → detect shot → annotate → save.</li></ul><h3>How to Run</h3><ol><li>Place your input video as video.mp4 in the project root (or change the filename in src/main.py).</li><li>Activate your environment:</li></ol><pre>source myenv/bin/activate</pre><ol><li>Run the main script:</li></ol><pre>python src/main.py</pre><ol><li>Output is saved as output.mp4.</li></ol><h3>Customization &amp; Extensions</h3><ul><li>Adjust detection or tracking thresholds in src/main.py and src/tracker.py for your video quality.</li><li>Integrate hand detection by calling get_hands(frame) from src/mediapipe_hands.py.</li><li>Add more analytics (e.g., passes, rebounds) by extending the logic in src/main.py.</li></ul><h3>Troubleshooting</h3><ul><li>If you get model errors, check that yolov8x.pt is in the project root.</li><li>For slow performance, reduce video resolution or use a smaller YOLOv8 model.</li><li>If Norfair tracking is unstable, tune the distance_threshold in src/tracker.py.</li></ul><h3>Final Thoughts</h3><p>This system is modular and easy to extend. The code is clean, with each function doing one job. You can use it as a foundation for more advanced basketball analytics, or adapt it for other sports.</p><p><strong>Full code and updates:</strong> <a href="https://github.com/vinod-polinati/basketball-vision-analytics.git">https://github.com/vinod-polinati/basketball-vision-analytics.git</a></p><p>Happy building!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9aa3cda6dee9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ResNet: Line-by-Line Explanation and Building One from Scratch]]></title>
            <link>https://medium.com/@vinodpolinati/resnet-line-by-line-explanation-and-building-one-from-scratch-398fd5d30d90?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/398fd5d30d90</guid>
            <category><![CDATA[vanishing-gradient]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[deep-neural-networks]]></category>
            <category><![CDATA[resnet]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Fri, 17 Jan 2025 06:32:01 GMT</pubDate>
            <atom:updated>2025-01-17T06:32:01.091Z</atom:updated>
            <content:encoded><![CDATA[<h4>Challenges with Deep Neural Networks</h4><blockquote>As neural networks grow deeper, their theoretical performance on training datasets should improve. However, in practice, deeper networks often underperform due to issues like the <strong>vanishing gradient problem</strong>. During backpropagation in very deep networks, gradients in earlier layers tend to diminish to near-zero, drastically slowing the learning process.</blockquote><h4>Introduction to ResNet</h4><p>ResNet (Residual Network) addresses this limitation, making it feasible to train very deep neural networks without performance degradation. The key innovation in ResNet is the <strong>skip connection</strong>, which allows models to propagate information directly across layers, bypassing intermediate computations.</p><h4>Skip Connection</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bP9DrF7i2_afI3vb1aUXzA.png" /></figure><p>A <strong>skip connection</strong> adds the activation from a previous layer directly to a later layer, ensuring that:</p><h4><strong>Forward Propagation</strong>:</h4><blockquote><strong>Later layers at least retain the performance of earlier ones.</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZUM652WgRkuRuOIwQMNnKQ.png" /></figure><blockquote><strong>Where 𝑎[𝑙] is fast-forwarded from the earlier layer. In the iteration process, as weights go to zero, 𝑧 would decrease to zero, in the worst scenario, even when 𝑧 totally lost its effect, activation value 𝑎[𝑙+2] would at least have the value of 𝑎[𝑙].</strong></blockquote><blockquote><strong>In the backpropagation process, the gradient can be directly backpropagated to earlier layers so that there would be fewer multiplications in the backpropagation, thus accelerate the learning process.</strong></blockquote><h4><strong>Backward Propagation</strong>:</h4><blockquote><strong>Gradients can flow back directly to earlier layers, reducing the chance of vanishing gradients and speeding up learning.</strong></blockquote><h4>Identity Block</h4><p>The <strong>identity block</strong> is a ResNet building block used when input and output dimensions match. It adds the input to the output of the main convolutional path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Hbh3x0KOcvvKXx8_3Drcw.png" /></figure><h3><strong>Steps :</strong></h3><h4><strong>First Component</strong>:</h4><ul><li>1x1 convolution (conv_name_base + &#39;2a&#39;) with F1F_1 filters, valid padding, and stride (1,1).</li><li>Batch normalization (bn_name_base + &#39;2a&#39;) on the channels axis.</li><li>ReLU activation.</li></ul><h4><strong>Second Component</strong>:</h4><ul><li>3x3 convolution (conv_name_base + &#39;2b&#39;) with F2F_2 filters, same padding, and stride (1,1).</li><li>Batch normalization (bn_name_base + &#39;2b&#39;).</li><li>ReLU activation.</li></ul><h4><strong>Third Component</strong>:</h4><ul><li>1x1 convolution (conv_name_base + &#39;2c&#39;) with F3F_3 filters, valid padding, and stride (1,1).</li><li>Batch normalization (bn_name_base + &#39;2c&#39;).</li><li><strong>No ReLU activation</strong>.</li></ul><h4><strong>Final Step</strong>:</h4><blockquote>Add the shortcut (input) to the main path output, then apply ReLU activation.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KCTtLgyrfK5uotfIAQ8RPQ.png" /></figure><h4>Convolutional Block</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sQvR-86kc2TikfoNkeULMQ.png" /></figure><p>The <strong>convolutional block</strong> handles cases where input and output dimensions differ. A convolution layer in the shortcut path adjusts the input size.</p><p>Steps:</p><h4><strong>Shortcut Path</strong>:</h4><ul><li>1x1 convolution (conv_name_base + &#39;1&#39;) with F3F_3 filters, valid padding, and stride ss.</li><li>Batch normalization (bn_name_base + &#39;1&#39;).</li></ul><h4><strong>Main Path</strong> (three components, similar to the identity block):</h4><ul><li>1x1 convolution (conv_name_base + &#39;2a&#39;), stride ss, F1F_1 filters.</li><li>3x3 convolution (conv_name_base + &#39;2b&#39;), stride (1,1), F2F_2 filters.</li><li>1x1 convolution (conv_name_base + &#39;2c&#39;), stride (1,1), F3F_3 filters.</li></ul><h4><strong>Final Step</strong>:</h4><blockquote>Add the shortcut to the main path output, then apply ReLU activation.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*by4HJ1JnOoObEMqBj_9p0A.png" /></figure><h4>Building ResNet-50</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4i80F1hbpty4lDIi8O3EtA.png" /></figure><p>ResNet-50 consists of a stack of convolutional and identity blocks organized into 5 stages:</p><h4><strong>Input Stage</strong>:</h4><ul><li>Zero-padding: Pad input with (3,3).</li><li>Convolution: 64 filters, (7,7) kernel, stride (2,2), followed by BatchNorm and ReLU.</li><li>MaxPooling: (3,3) window, stride (2,2).</li></ul><h4><strong>Stage 2</strong>:</h4><ul><li>Convolutional block: Filters = [64, 64, 256], stride = 1.</li><li>2 Identity blocks: Filters = [64, 64, 256].</li></ul><h4><strong>Stage 3</strong>:</h4><ul><li>Convolutional block: Filters = [128, 128, 512], stride = 2.</li><li>3 Identity blocks: Filters = [128, 128, 512].</li></ul><h4><strong>Stage 4</strong>:</h4><ul><li>Convolutional block: Filters = [256, 256, 1024], stride = 2.</li><li>5 Identity blocks: Filters = [256, 256, 1024].</li></ul><h4>Stage 5:</h4><ul><li>Convolutional block: Filters = [512, 512, 2048], stride = 2.</li><li>2 Identity blocks: Filters = [512, 512, 2048].</li></ul><h4>Output Stage:</h4><blockquote>Average pooling: (2,2) window.</blockquote><blockquote>Flatten and Dense layers: Fully connected layer with softmax activation for classification.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hhxrrjLmpmwdiCrfpJlPlA.png" /></figure><h4>Model Summary</h4><p>Run model.summary() to view the architecture, filter sizes, and layer details.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=398fd5d30d90" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decoding Sea Ice and SAR Imagery:]]></title>
            <link>https://medium.com/@vinodpolinati/decoding-sea-ice-and-sar-imagery-ac33a974e01c?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac33a974e01c</guid>
            <category><![CDATA[sea-ice]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[satellite-aperture-radar]]></category>
            <category><![CDATA[sar-imagery]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Fri, 17 Jan 2025 03:32:00 GMT</pubDate>
            <atom:updated>2025-01-17T03:32:00.680Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Insights into Polar Environments and Radar Technologies</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*2ueEfr2XTwq-9R1x.png" /></figure><h4><strong>Sea Ice and Types :</strong></h4><p>Sea ice is frozen seawater that floats on the ocean surface and is categorized based on its age, thickness, and physical properties. Let’s break it down:</p><h4>First-Year Ice (FYI)</h4><ul><li><strong>Lifespan</strong>: Forms and melts within a single year.</li><li><strong>Thickness</strong>: Up to 2 meters.</li><li><strong>Characteristics</strong>:</li><li>Saltier compared to multi-year ice due to limited brine drainage.</li><li>Softer and weaker, making it more prone to deformation and melting.</li></ul><h4>Multi-Year Ice (MYI)</h4><ul><li><strong>Lifespan</strong>: Survives multiple melt seasons, often persisting for years.</li><li><strong>Thickness</strong>: Exceeds 2 meters.</li></ul><h4><strong>Characteristics</strong>:</h4><ul><li>Lower salinity as brine drains out over time.</li><li>Harder and more resistant to melting.</li><li>Has a distinct bluish appearance due to prolonged compaction and density.</li></ul><h4>New Ice</h4><ul><li><strong>Formation</strong>: Recently developed ice, a precursor to thicker ice forms.</li><li><strong>Types</strong>:</li><li><strong>Frazil Ice</strong>: Small, needle-like ice crystals.</li><li><strong>Grease Ice</strong>: A thin, slushy layer that appears oily.</li></ul><h4><strong>Characteristics</strong>:</h4><ul><li>Thin and flexible.</li><li>Evolves into thicker forms like nilas or young ice over time.</li><li>Sea ice is primarily found in the Arctic and Antarctic regions, where it shapes the local environment and serves as a critical habitat for diverse species.</li></ul><h3>Synthetic Aperture Radar (SAR) Imagery: Revolutionizing Sea Ice Analysis</h3><p>Synthetic Aperture Radar (SAR) has become an indispensable tool for studying sea ice. By utilizing radar signals to generate high-resolution images, SAR overcomes the limitations of optical imaging, offering unparalleled insights into the structure and dynamics of sea ice.</p><h4>Key Features of SAR</h4><ul><li>Operates under all weather conditions, including clouds, rain, and fog.</li><li>Functions effectively during both day and night.</li><li>Penetrates through vegetation, ice, and even dry soil, providing detailed surface data.</li></ul><h4>How SAR Works</h4><p>SAR emits microwave signals that interact with the Earth’s surface. The radar measures the strength, phase, and time delay of the returned signals, processing them into detailed images. This makes it particularly suited for monitoring complex environments like the polar regions.</p><h4>Polarisation in SAR</h4><p>Polarisation enhances SAR’s ability to classify sea ice by analyzing electromagnetic wave orientations.</p><ul><li><strong>Single Polarisation</strong>: Transmits and receives in the same orientation (e.g., HH or VV).</li><li><strong>Dual Polarisation</strong>: Transmits in one orientation (e.g., HH) and receives in two (e.g., HH and HV).</li><li><strong>Quad Polarisation</strong>: Transmits and receives in all combinations (HH, HV, VH, VV).</li></ul><p>This versatility enables precise classification of sea ice types and surface properties.</p><h4>Applications and Advantages</h4><h4><strong>Applications</strong>:</h4><ul><li><em>Monitoring sea ice extent, thickness, and movement.</em></li><li><em>Assessing surface roughness and moisture content.</em></li><li><em>Tracking changes in surface deformation and coherence.</em></li></ul><h4><strong>Advantages</strong>:</h4><ul><li>Independent of sunlight and atmospheric conditions.</li><li>Provides high-resolution imaging for detailed analysis.</li></ul><h4>Challenges</h4><p>While SAR offers unmatched capabilities, it comes with challenges like complex image interpretation, high costs, and the need for expert analysis.</p><h3>L Band vs. C Band: Choosing the Right SAR Frequency</h3><p>SAR systems operate across different frequency bands, with L Band and C Band being the most commonly used for sea ice monitoring.</p><h3>L Band</h3><ul><li><strong>Frequency</strong>: ~1.2 GHz.</li><li><strong>Wavelength</strong>: 15–30 cm.</li></ul><h4><strong><em>Characteristics</em></strong>:</h4><ul><li>Deeper penetration through soil and ice.</li><li>Suitable for long-term monitoring of forests and ice sheets.</li><li>Lower resolution compared to C Band.</li></ul><h3>C Band</h3><ul><li><strong>Frequency</strong>: 4–8 GHz.</li><li><strong>Wavelength</strong>: 3.75–7.5 cm.</li></ul><h4><strong>Characteristics</strong>:</h4><ul><li>Moderate penetration, sensitive to surface structures.</li><li>Ideal for monitoring surface-level changes, including ice and snow.</li><li>Higher resolution, widely used in satellite missions like Sentinel-1.</li></ul><p>Each band has its strengths, with the choice depending on the specific monitoring requirements and resolution needs.</p><h3>Conclusion</h3><blockquote>Sea ice, with its diverse forms and crucial environmental roles, is a subject of immense scientific interest. Understanding its behavior is key to addressing broader questions about climate change and polar ecosystems. Tools like Synthetic Aperture Radar are at the forefront of this effort, offering unprecedented insights into the dynamics of sea ice.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac33a974e01c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[U2-Net: Revolutionizing Salient Object Detection]]></title>
            <link>https://medium.com/@vinodpolinati/u2-net-revolutionizing-salient-object-detection-5cb10717a209?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/5cb10717a209</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[u2net]]></category>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[residual-block]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Thu, 16 Jan 2025 04:52:30 GMT</pubDate>
            <atom:updated>2025-01-16T04:52:30.508Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LINEpYE55VNosns0upadbA.png" /><figcaption>Architecture of U2-Net</figcaption></figure><p>U2-Net is an architectrure which is specifically designed for Salient Object Detection (SOD)</p><ul><li>It’s a two level nested U shaped architecture build using Residual U Blocks ( RSU )</li><li>U2 Net does not use pre trained backbone which makes it really light.</li><li>There are mainly two algorithms for visual saliency:</li><li><strong>FP</strong> (fixation prediction) and <strong>SOD</strong>. ( Salient Object Detection )</li><li>The first of these algorithms aim to define the area in which visual attention is concentrated, but the second try to identify the entire entity that has attracted attention and thus also its precise contours.</li><li>The first deep SOD algorithms were based on MLPs (multi-layer perceptrons), but with the development of FCN (fully convolutional network) models the whole pipeline became much more efficient</li></ul><h3>What are RSU Blocks ?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2vSzvXYOjqM27L6eL_Xxmg.png" /><figcaption>Residual-U Blocks</figcaption></figure><p><em>RSU blocks are a type of residual blocks used in deep learning , residual blocks are a key component of modern deep learning models , which are used to address the vanishing gradient problem.<br>Residual blocks are skip-connection blocks that learn residual functions<br>These are foundational cells of ResNet — which is a model for extracting features of an image.<br>These blocks are used in various innovations and also transformers such as BERT.<br></em>RSU’s are basically the smaller version of U-Net<br>Combination of both U-Net and residual U blocks<br>In the above diagram we use the Convolution + Bass Normalization and ReLU.</p><h4><strong>What is Salient Object Detection?</strong></h4><ul><li>It’s a process of segmenting the main object from an input image by producing high quality mask.</li><li>Using the high quality mask we can extract the fine details of the image as shown below</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/931/1*R4VszhflQeiclONYYCO1mg.png" /><figcaption>Using U2-Net</figcaption></figure><h4><strong>Issues with existing methods:</strong></h4><ul><li>Most existing methods just use pretrained methods like ResNet , VGG etc , and these backbones are designed for image classification, so they lack local details and global information.</li><li>These backbone architecture losses the essential high-level details during he downsampling process in the initial layers.</li></ul><h4><strong>Main contributions of the U2 Net</strong></h4><ul><li>The U2- Net maintains high res features with low memory and computation cost.</li><li>The U2-Net is build using the Residual U-blocks which helps in extracting features efficiently.</li><li>It achieves SOTA performance on six datasets and runs in real time with 30 frames on 1080Ti with a resolution of 320 x 320</li></ul><h3><strong>Architecture of U2-Net :</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*43_fXyLtjWsvIBIBG5Pl-Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LINEpYE55VNosns0upadbA.png" /></figure><ul><li>Two level nested U-Net architecture</li><li>Each of the block uses Residual U-blocks (RSU)</li><li>If we see Encoder 5 , 6 &amp; Decoder 5,6 in the diagram we see that is there is no upsampling or downsampling but there has been only Convolution , batch normalisation and ReLU , because till that state the height and width of feature map is so small that if we downsample or upsample we loose the essential information.</li><li>So instead of using the upsample / downsample the author uses a diluted convolution. <br> <br> <strong>There are two variants of U2-Net:</strong></li><li>U2-Net : 30 Fps</li><li>U2-Net Lite : 40 Fps</li></ul><blockquote><em>Runs on<br>- GPU : 1080Ti<br>- Input Shape : 320 x 320 x 3</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cb10717a209" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[U-Net : A deep dive into Image Segmentation]]></title>
            <link>https://medium.com/@vinodpolinati/u-net-a-deep-dive-into-image-segmentation-05b872642ed1?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/05b872642ed1</guid>
            <category><![CDATA[image-segmentation]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <category><![CDATA[vgg19]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Wed, 15 Jan 2025 21:42:13 GMT</pubDate>
            <atom:updated>2025-01-15T21:42:13.876Z</atom:updated>
            <content:encoded><![CDATA[<h3>U-Net : A deep dive into Image Segmentation</h3><ul><li>A special convolutional-neural-network architecture used for biomedical image segmentation, popularly used for any segmentation task.</li><li>U-Net is basically a complete convolutional network designed to learn from fewer training samples</li></ul><h3><strong>What is U-Net?</strong></h3><ul><li>A U shaped Encoder — Decoder architecture used mostly for image segmentation</li><li>It has 4 encoder blocks on the left side and 4 decoder blocks on the right side</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pvZk7Bqx1wMyBHPthjrLgg.png" /><figcaption>Architecture of U Net</figcaption></figure><h3><strong>Working of U Net :</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_sl7pgzKXAQ_quarfO0G1Q.png" /><figcaption>Simple understanding of U-Net</figcaption></figure><ul><li>Encoder receives input image and extracts features using multiple convolutions</li><li>The decoder then upsamples features using transpose convolution</li><li>Then concatenates them with a process called skip connection</li><li>As a result we get a segmentation mask as an output</li><li>Skip connection skips some of layers in neural network and feeds output of one layer as the input to the next layers</li></ul><h4><strong><em>Lets dive deep into U-Net and get to know how it works :</em></strong></h4><ul><li>U Net is a special architecture for Image Segmentation</li><li>Each layer below represents the stages of CNN</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gxPyd6kfulYYwgCH7WgWLg.png" /><figcaption>CNN Architecture</figcaption></figure><ul><li>We can probably use VGG for segmentation , as it is used in the encoder part of the U-Net Architecture</li><li>VGG 19 Architecture is shown below</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SnFDASUdm61Sxza0GVeqgQ.png" /><figcaption>VGG 19 Architecture</figcaption></figure><h3>Layers in VGG-19</h3><blockquote>Max Pool Layer <br>Convolutional Layer<br>Max Pool<br>Convolutional<br>Dense Layers<br>[ Note : Conv, Max Pool can be n no. of layers ]<br>Dense Layer we have a whole bunch of neurons connected to each other</blockquote><h3><strong>U-Net is Designed for Semantic Segmentation</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mWp9czDXZOrVu6e95YUV5A.png" /><figcaption>Differences between them</figcaption></figure><blockquote>- Semantic Segmentation is if we paint a pixel corresponding to humans in this image that is semantic segmentation where every pixel either represents a human or a background<br>- <strong>U-Net is designed to do this task</strong><br>- If we can separate each individual then we can call it as Instance Segmentation , this is an extension of Semantic Segmentation</blockquote><h3><strong>ARCHITECTURE OF U-NET</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pvZk7Bqx1wMyBHPthjrLgg.png" /><figcaption>U NET ARCHITECTURE</figcaption></figure><blockquote>There are 2 paths in U — Net<br> — — → Path on the left is called contraction path , also called encoder path<br> — — → Path on the right is called expansion path , also called the decoder path <br> — — → In between the data is concatenated , this helps give localised information</blockquote><h3>Encoder Part of the U-Net</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oLfJpB-V3ENngbA6rO8o3g.png" /></figure><blockquote><strong>— — → </strong>The top left is where the input layer is , the image is 128 * 128 * 3 <br>( Here 3 means the image is coloured ).<br> — — →Add a feature space of 16 ,<br> — — → Output of the image will be 128*128*16 . These are basically convolutional operations .<br> — — → The kernel size is 3*3<br> — — → We are adding a padding = same (when we run a 3*3 conv , it adds an extra pixels to the edges so that the output image is same as input image )<br> <strong>— — →</strong> Next step is max pooling step of using 2 * 2 and stride = 2<br> — — → Max pooling is basically put a 2 * 2 matrix and then within that select the maximum values and replace the 2 * 2 matrix with that maximum values<br> — — → After doing max pooling we get to the second stage</blockquote><h3>The second stage of U Net</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c7bayUxA2diJZMBXdB3spw.png" /></figure><blockquote>— — → Dimensions would be half of the above as we use a 2*2 matrix<br> — — → As we use 2 conv operations which means a feature space of 32 .<br> — — → So it becomes <em>64 * 64 * 32</em>.<br> — — → We’ll repeat this process all the way until the last stage</blockquote><h3><em>The final stage of encoder part in U Net</em></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lOCgRi0toSJzokvjxZ08LA.png" /></figure><ul><li>As you in the last stage the dimensions are <em>8x8x128</em> and <em>8x8x256</em></li></ul><h3><strong>Decoder part in the U-Net</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e2sz_fiAZf3QX5puhQpDFQ.png" /></figure><blockquote>— — → The result of the upsampling, like this box right there would be <em>16 * 16 * 128</em> , because we are going from 8 to 16 over there and then it’s <em>2 * 2</em> .<br> — — → To that we are gonna add C4.<br> — — → So , the block( circle I rounded off ) is nothing but U6 + C4<br> — — → So basically the 2 rectangles represent each of <em>16 * 16 * 128</em> and when you add it , it becomes <em>16 * 16 * 256 </em>and then it goes thru a couple of convolution steps.<br> — — → So the result would be 16 * 16 * 128 now we up sample again<br> — — → <strong>Repeat the steps 🔁</strong><br> — — → In between the convolutional steps , we are going to dropout ( which means randomly select 10 % of the pixels and drop them from any of these calculations )<br> — — → <strong>Note : We use dropout to prevent overfitting</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ubMyAoZqGt4x2k4eB6Ur8g.png" /></figure><h4><strong><em>The coding portion of the above diagram is provided below along with an explanation.</em></strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6YBOS76osMOb9T92qFiC8Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ubMyAoZqGt4x2k4eB6Ur8g.png" /><figcaption>The code, along with the diagram, is provided for easier understanding.</figcaption></figure><h3><strong><em>The code explanation</em></strong></h3><blockquote><em>C1 is convolution operations with dimensions of </em><strong><em>16</em></strong><em> and </em><strong><em>3 x 3 </em></strong><em>, we are using ReLU as activation function and padding as same</em></blockquote><blockquote><em>P1 is defined as Max Pooling of </em><strong><em>2 x 2</em></strong><em> , apply on c1</em></blockquote><blockquote><em>That’s it , it continues we do 2 convolutional and and one max pool and we may add dropout’s like 20% here and so on .</em></blockquote><blockquote><em>Finally when we get to output layer, we have optimiser — Adam seems to be the best one we have and the loss function is what the optimiser is trying to minimise and the metrics we keep track of as the algorithm progresses is the accuracy.</em></blockquote><blockquote><em>Repeat the steps for each and every layer</em></blockquote><blockquote><strong><em>Conclusion<br></em></strong>U-Net’s simple yet powerful design has transformed image segmentation, excelling in tasks like biomedical segmentation with fewer training samples. Its encoder-decoder architecture, skip connections, and efficient use of layers make it a go-to model for accurate segmentation. Dive in, experiment, and unlock U-Net’s full potential in your projects! 🚀</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05b872642ed1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transfer Learning: Accelerating AI with Pre-Trained Knowledge]]></title>
            <link>https://medium.com/@vinodpolinati/transfer-learning-accelerating-ai-with-pre-trained-knowledge-fbd37f4c104c?source=rss-a4e64f23935f------2</link>
            <guid isPermaLink="false">https://medium.com/p/fbd37f4c104c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transfer-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Vinod Polinati]]></dc:creator>
            <pubDate>Fri, 28 Jun 2024 15:54:26 GMT</pubDate>
            <atom:updated>2024-06-28T15:54:26.309Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Transfer Learning is a powerful technique in the field of artificial intelligence that leverages pre-trained models to solve new but related tasks more efficiently. By transferring knowledge from one domain to another, transfer learning accelerates the training process and improves performance, making it an invaluable tool in modern AI applications</blockquote><h3>What Is Transfer Learning?</h3><p>Transfer Learning involves taking a pre-trained model that has been trained on a large dataset and fine-tuning it for a specific, smaller task. Instead of starting from scratch, the model builds upon the pre-existing knowledge.</p><h3>How Transfer Learning Works</h3><ol><li><strong>Pre-Trained Model</strong>: A model is first trained on a large, general dataset. Popular examples include models like ResNet, BERT, and GPT, which have been trained on massive image and text datasets.</li><li><strong>Feature Extraction</strong>: The pre-trained model’s lower layers, which have learned to extract general features, are used as a starting point.</li><li><strong>Fine-Tuning</strong>: The model is then fine-tuned on a smaller, task-specific dataset, adjusting the higher layers to adapt to the new task.</li></ol><p>This process allows the model to quickly adapt to new tasks by leveraging previously learned features and patterns.</p><h3>Applications of Transfer Learning</h3><p>Transfer learning has a broad range of applications, making it a versatile tool in various fields:</p><h4>1. Image Recognition</h4><p>Transfer learning is widely used in image recognition tasks. Pre-trained models like VGG, Inception, and ResNet, trained on large image datasets like ImageNet, are fine-tuned for specific tasks such as medical imaging, facial recognition, and object detection.</p><h4>2. Natural Language Processing</h4><p>In NLP, models like BERT, GPT, and RoBERTa, pre-trained on vast text corpora, are fine-tuned for specific tasks such as sentiment analysis, question answering, and language translation.</p><h4>3. Speech Recognition</h4><p>Pre-trained models for speech recognition can be fine-tuned to recognize specific accents, dialects, or specialized vocabulary, improving accuracy and performance.</p><h4>4. Time Series Forecasting</h4><p>Transfer learning can be applied to time series data, where a model trained on historical data from one domain can be adapted to forecast trends in another related domain.</p><h4>5. Medical Diagnosis</h4><p>In healthcare, transfer learning helps in developing models that can analyze medical images or patient records to diagnose diseases, even with limited labeled data.</p><h3>Advantages of Transfer Learning</h3><ul><li><strong>Efficiency</strong>: Reduces the amount of data and computational power needed by leveraging pre-trained models.</li><li><strong>Speed</strong>: Accelerates the training process, allowing for quicker deployment of models.</li><li><strong>Performance</strong>: Often improves performance on specific tasks by building on the robust features learned from large datasets.</li></ul><h3>Conclusion</h3><p>Transfer Learning is revolutionizing the way we approach AI and machine learning tasks. By reusing and fine-tuning pre-trained models, it brings efficiency, speed, and enhanced performance to various applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbd37f4c104c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>