<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sanath Raj on Medium]]></title>
        <description><![CDATA[Stories by Sanath Raj on Medium]]></description>
        <link>https://medium.com/@Sanath_raj?source=rss-72310a12e867------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cjhxssWbWJrQ6-rxpqxNxA.jpeg</url>
            <title>Stories by Sanath Raj on Medium</title>
            <link>https://medium.com/@Sanath_raj?source=rss-72310a12e867------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 14:56:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@Sanath_raj/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build Your Own MCP Server That Runs Entirely on the Laptop. Here’s How.]]></title>
            <link>https://medium.com/@Sanath_raj/i-built-my-own-mcp-server-that-runs-entirely-on-my-laptop-heres-how-19a54c72dd7e?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/19a54c72dd7e</guid>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Sun, 24 May 2026 10:35:36 GMT</pubDate>
            <atom:updated>2026-06-01T07:34:15.123Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A local RAG assistant connected to Claude Desktop — no cloud, no API costs, no data leaving the laptop .</em></p><p>I’m typing a question into Claude Desktop. Claude responds with a detailed answer, cites its source, tells me it ran at 38 tokens per second, and ends with “🔒 100% local.”</p><p>No OpenAI API. No cloud RAG service. No data sent anywhere.</p><p>The answer came from a 135MB language model running on my laptop CPU, searching a local vector database, through a custom MCP server</p><h3>What Is MCP and Why Does It Matter?</h3><p>MCP stands for Model Context Protocol. It’s an open standard created by Anthropic that lets AI clients like Claude Desktop discover and call tools on external servers — the same way a browser talks to a web server, but for AI tool use.</p><p>The magic is in the simplicity. An MCP server declares what tools it has. The client calls them like functions. The protocol handles everything in between.</p><pre>Claude Desktop  →  &quot;What tools do you have?&quot;<br>MCP Server      →  &quot;I have: search_documents, ask_document, add_document...&quot;<br>Claude Desktop  →  &quot;Call ask_document with question=&#39;What causes diabetes?&#39;&quot;<br>MCP Server      →  [runs RAG pipeline, returns answer]<br>Claude Desktop  →  Shows answer to user</pre><p>Before MCP, connecting an AI to your own tools meant custom integrations, proprietary APIs, or prompt hacking. MCP standardises all of it. Build a server once, connect any MCP-compatible client.</p><h3>The Architecture</h3><p>Here’s what I built:</p><pre>Claude Desktop (MCP Client)<br>        ↓  stdio — MCP JSON-RPC protocol<br>rag_mcp_server.py (FastMCP)<br>        ├── ChromaDB  (vector store + embeddings)<br>        │   └── all-MiniLM-L6-v2  (local embedding model)<br>        └── llama-server  (localhost:8080)<br>            └── SmolLM 135M Q4_K_M  (local GGUF model)</pre><p>Every component runs locally:</p><ul><li><strong>ChromaDB</strong> — stores and retrieves document embeddings</li><li><strong>all-MiniLM-L6-v2</strong> — converts text to vectors, runs on CPU</li><li><strong>llama-server</strong> — serves the GGUF model via REST API</li><li><strong>SmolLM 135M</strong> — the language model that generates answers</li><li><strong>FastMCP</strong> — the Python library that handles the MCP protocol</li></ul><p>Total external dependencies after setup: zero.</p><h3>Why a Local MCP Server?</h3><p>Most MCP server tutorials point at cloud APIs — Slack, GitHub, Google Drive. Those are useful but they have a fundamental limitation: your data leaves your machine.</p><p>For a local MCP server the value proposition is different:</p><p><strong>Privacy by architecture.</strong> Medical questions, legal documents, financial data — nothing leaves the laptop. Not by policy, by design.</p><p><strong>Zero ongoing cost.</strong> No per-token billing. No API subscription. The model runs as many queries as you want.</p><p><strong>Works offline.</strong> Once set up, no internet required. The model, the vector store, the server — all local.</p><p><strong>Full control.</strong> You own the knowledge base. You decide what goes in. You can inspect every query and response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wTQh1_ItypjTQN8JwNLnUQ.png" /></figure><h3>Step 1: The Local Language Model</h3><p>First, I needed a model that could run without a GPU. I used SmolLM 135M from HuggingFace — designed specifically for on-device use.</p><p>Download it in GGUF format (quantized to Q4_K_M — about 90MB) and serve it with llama.cpp:</p><pre>llama-server `<br>  --model &quot;C:\Users\you\Downloads\smollm_135M_q4_k_m.gguf&quot; `<br>  --host 0.0.0.0 `<br>  --port 8080 `<br>  --threads 6 `<br>  --ctx-size 2048 `<br>  --log-disable</pre><p>This exposes a REST API on localhost:8080. Any code that talks to OpenAI can now talk to your local model with zero changes.</p><p>On my laptop integrated Radeon graphics (using the Vulkan build of llama.cpp), this runs at about 38–40 tokens per second. Fast enough for real-time responses.</p><h3>Step 2: The Vector Database</h3><p>For RAG you need somewhere to store document embeddings. I used ChromaDB — it runs as a Python library with no separate server process, and persists to disk automatically.</p><pre>import chromadb<br>from chromadb.utils import embedding_functions</pre><pre># Persistent local database — survives restarts<br>client = chromadb.PersistentClient(path=str(Path.home() / &quot;medical_rag_db&quot;))</pre><pre># Local embedding model — no API calls<br>embed_fn = embedding_functions.SentenceTransformerEmbeddingFunction(<br>    model_name=&quot;all-MiniLM-L6-v2&quot;<br>)</pre><pre>collection = client.get_or_create_collection(<br>    name=&quot;medical_docs&quot;,<br>    embedding_function=embed_fn,<br>    metadata={&quot;hnsw:space&quot;: &quot;cosine&quot;},<br>)</pre><p>all-MiniLM-L6-v2 is a 90MB sentence transformer that runs entirely on CPU. It converts text to 384-dimensional vectors that ChromaDB uses for similarity search.</p><p>I seeded the database with 10 medical topics — body temperature, heart rate, blood pressure, diabetes, hydration, liver function, respiratory health, infections, medications, and mental health. Each is a paragraph of factual content with a source label.</p><pre>collection.add(<br>    ids=[&quot;med_001&quot;],<br>    documents=[&quot;Normal human body temperature is 98.6°F (37°C). Fever is defined as above 100.4°F...&quot;],<br>    metadatas=[{&quot;source&quot;: &quot;General Medicine Reference&quot;}],<br>)</pre><h3>Step 3: The MCP Server</h3><p>This is where FastMCP comes in. It’s a Python library that handles all the MCP protocol boilerplate — JSON-RPC, tool discovery, schema generation — so you can focus on writing tools as plain Python functions.</p><pre>from fastmcp import FastMCP</pre><pre>mcp = FastMCP(<br>    name=&quot;Medical RAG Assistant&quot;,<br>    instructions=&quot;A local medical knowledge base powered by SmolLM...&quot;<br>)</pre><p>A tool is just a decorated function with a docstring. The docstring becomes the tool description that Claude uses to decide when to call it.</p><h3>Tool 1: search_documents</h3><p>Returns raw retrieved chunks from ChromaDB. Useful when you want to see what’s in the knowledge base without generating an answer.</p><pre>@mcp.tool()<br>def search_documents(query: str, n_results: int = 3) -&gt; str:<br>    &quot;&quot;&quot;<br>    Search the local medical knowledge base and return relevant text chunks.<br>    Use this to retrieve context before generating an answer.<br>    &quot;&quot;&quot;<br>    results = collection.query(<br>        query_texts=[query],<br>        n_results=n_results,<br>        include=[&quot;documents&quot;, &quot;metadatas&quot;, &quot;distances&quot;],<br>    )</pre><pre>    docs      = results[&quot;documents&quot;][0]<br>    metas     = results[&quot;metadatas&quot;][0]<br>    distances = results[&quot;distances&quot;][0]</pre><pre>    output = f&quot;Found {len(docs)} chunks for: &#39;{query}&#39;\n\n&quot;<br>    for doc, meta, dist in zip(docs, metas, distances):<br>        relevance = round((1 - dist) * 100, 1)<br>        output += f&quot;[Source: {meta[&#39;source&#39;]} — {relevance}% relevance]\n{doc}\n\n&quot;</pre><pre>    return output.strip()</pre><h3>Tool 2: ask_document</h3><p>The full RAG pipeline. Retrieves context from ChromaDB, constructs a prompt, calls llama-server, returns the generated answer.</p><pre>@mcp.tool()<br>def ask_document(question: str) -&gt; str:<br>    &quot;&quot;&quot;<br>    Answer a medical question using RAG: retrieves relevant context from the<br>    local knowledge base, then generates an answer using the local SmolLM model.<br>    No data leaves the laptop.<br>    &quot;&quot;&quot;<br>    # Step 1: Retrieve<br>    results = collection.query(<br>        query_texts=[question],<br>        n_results=3,<br>        include=[&quot;documents&quot;, &quot;metadatas&quot;],<br>    )<br>    context = &quot;\n\n&quot;.join([<br>        f&quot;[{meta[&#39;source&#39;]}]: {doc}&quot;<br>        for doc, meta in zip(results[&quot;documents&quot;][0], results[&quot;metadatas&quot;][0])<br>    ])</pre><pre>    # Step 2: Generate<br>    response = requests.post(<br>        &quot;http://localhost:8080/v1/chat/completions&quot;,<br>        json={<br>            &quot;model&quot;: &quot;smollm&quot;,<br>            &quot;messages&quot;: [<br>                {<br>                    &quot;role&quot;: &quot;system&quot;,<br>                    &quot;content&quot;: (<br>                        &quot;You are a concise medical information assistant. &quot;<br>                        &quot;Answer using only the provided context in 2-3 sentences. &quot;<br>                        &quot;Always add: &#39;For medical advice, consult a healthcare professional.&#39;&quot;<br>                    ),<br>                },<br>                {<br>                    &quot;role&quot;: &quot;user&quot;,<br>                    &quot;content&quot;: f&quot;Context:\n{context}\n\nQuestion: {question}&quot;,<br>                },<br>            ],<br>            &quot;max_tokens&quot;: 150,<br>            &quot;temperature&quot;: 0.1,<br>        },<br>        timeout=60,<br>    )</pre><pre>    data   = response.json()<br>    answer = data[&quot;choices&quot;][0][&quot;message&quot;][&quot;content&quot;].strip()<br>    tps    = data[&quot;timings&quot;][&quot;predicted_per_second&quot;]<br>    tokens = data[&quot;usage&quot;][&quot;completion_tokens&quot;]</pre><pre>    return f&quot;{answer}\n\n---\nSource: local SmolLM 135M | {tokens} tokens | {tps:.1f} tok/s | 🔒 100% local&quot;</pre><h3>Tool 3: add_document</h3><p>Lets you add new content to the knowledge base from inside Claude Desktop. No code required — just ask Claude to add something.</p><pre>@mcp.tool()<br>def add_document(text: str, source: str = &quot;User provided&quot;) -&gt; str:<br>    &quot;&quot;&quot;Add a new medical document to the local knowledge base.&quot;&quot;&quot;<br>    new_id = f&quot;user_{collection.count() + 1:04d}&quot;<br>    collection.add(<br>        ids=[new_id],<br>        documents=[text.strip()],<br>        metadatas=[{&quot;source&quot;: source}],<br>    )<br>    return f&quot;✅ Added {new_id} from &#39;{source}&#39;. Total: {collection.count()} documents.&quot;</pre><h3>Starting the server</h3><pre>if __name__ == &quot;__main__&quot;:<br>    mcp.run(transport=&quot;stdio&quot;)</pre><p>stdio transport means Claude Desktop launches this script as a subprocess and communicates via stdin/stdout.</p><h3>Step 4: Connecting to Claude Desktop</h3><p>Claude Desktop looks for MCP servers in a config file:</p><pre>C:\Users\you\AppData\Roaming\Claude\claude_desktop_config.json</pre><p>Add your server:</p><pre>{<br>  &quot;mcpServers&quot;: {<br>    &quot;medical-rag&quot;: {<br>      &quot;command&quot;: &quot;cmd&quot;,<br>      &quot;args&quot;: [<br>        &quot;/c&quot;,<br>        &quot;C:\\Users\\you\\medical_mcp\\start_mcp.bat&quot;<br>      ]<br>    }<br>  }<br>}</pre><p>The batch file (start_mcp.bat) simply calls Python with your server script:</p><pre>@echo off<br>python.exe C:\Users\you\medical_mcp\rag_mcp_server.py</pre><p>Using a batch file as the entry point avoids PATH resolution issues that Windows has with Microsoft Store Python installations.</p><p>Restart Claude Desktop. The 🔧 tools icon appears in the chat input. Click it — your tools are listed.</p><h3>What It Looks Like in Practice</h3><p>Once connected, you just chat normally. Claude decides when to use your tools.</p><p><strong>You type:</strong></p><blockquote><em>What are the symptoms of a heart attack?</em></blockquote><p><strong>Claude calls</strong> ask_document(&quot;What are the symptoms of a heart attack?&quot;)</p><p><strong>ChromaDB retrieves</strong> the Emergency Medicine chunk (94% relevance)</p><p><strong>SmolLM generates:</strong></p><blockquote><em>Symptoms of a heart attack include chest pain or pressure, pain radiating to the left arm or jaw, shortness of breath, nausea, cold sweats, and lightheadedness. Heart attacks require immediate emergency care — call 911. For medical advice, consult a healthcare professional.</em></blockquote><blockquote><em>Source: local SmolLM 135M | 52 tokens | 38.4 tok/s | 🔒 100% local</em></blockquote><p>The whole thing — retrieval + generation — takes about 3 seconds.</p><h3>The Numbers</h3><p>Running on laptop (integrated Radeon, Vulkan build):</p><p>Metric Value Generation speed ~38–40 tok/s Response time (short answer) 2–4 seconds Model size 90 MB Vector DB size (10 docs) ~5 MB RAM used (model + server) ~400 MB API cost $0</p><p>Internet required <em>None (</em>after setup)</p><h3>The Hardest Part: Windows Path Hell</h3><p>Getting Claude Desktop to launch the MCP server on Windows was the trickiest part. Three issues I hit:</p><p><strong>1. Microsoft Store Python has a sandboxed path.</strong> The python.exe installed from the Store lives in an AppData path that Claude Desktop couldn&#39;t resolve directly. Workaround: use a .bat file as the launcher.</p><p><strong>2. PowerShell’s </strong><strong>Out-File adds a BOM.</strong> Writing the config JSON with Out-File -Encoding utf8 adds a UTF-8 BOM character that makes Claude Desktop throw a JSON parse error. Fix: use Python&#39;s json.dump() with open(..., encoding=&#39;utf-8&#39;) to write the file.</p><p><strong>3. Claude Desktop caches the config.</strong> Even after editing the config, old sessions persisted. Fix: taskkill /F /IM &quot;Claude.exe&quot; to force-kill before restarting.</p><p>None of these are showstoppers but they cost an hour of debugging without knowing the cause.</p><h3>What You Can Build With This Pattern</h3><p>The medical RAG demo is just one application. The same pattern works for:</p><p><strong>Personal knowledge base</strong> — index your notes, articles, PDFs. Ask Claude questions about your own writing.</p><p><strong>Company documentation</strong> — index internal docs, runbooks, policies. Query them from Claude Desktop without uploading to any cloud service.</p><p><strong>Code search</strong> — index a codebase. Ask “how does the authentication flow work?” and get answers grounded in your actual code.</p><p><strong>Legal document analysis</strong> — index contracts and filings. Ask questions with zero data leaving the premise.</p><p>The MCP server doesn’t care what’s in ChromaDB. Swap the medical documents for anything.</p><h3>Extending the Server</h3><p>A few things I’d add for production use:</p><p><strong>Chunking large documents.</strong> Right now each document is one chunk. For longer documents you’d split by paragraph or sentence with overlap, store each chunk separately, and reassemble relevant chunks at retrieval time.</p><p><strong>Better embeddings.</strong> all-MiniLM-L6-v2 is fast and small. For better retrieval quality, bge-large-en-v1.5 or nomic-embed-text give significantly better results at the cost of more RAM.</p><p><strong>Larger model.</strong> SmolLM 135M is proof-of-concept quality. SmolLM 1.7B at Q4_K_M (~1.1GB) produces noticeably better answers and still fits on any modern laptop.</p><p><strong>Metadata filtering.</strong> ChromaDB supports filtering by metadata fields. You could tag documents by category and let Claude filter to {&quot;category&quot;: &quot;cardiology&quot;} before retrieving.</p><h3>Resources</h3><ul><li><strong>FastMCP</strong> — <a href="https://gofastmcp.com/">gofastmcp.com</a> — the simplest way to build MCP servers in Python</li><li><strong>llama.cpp</strong> — <a href="https://github.com/ggml-org/llama.cpp">github.com/ggml-org/llama.cpp</a> — CPU/GPU inference for GGUF models</li><li><strong>ChromaDB</strong> — <a href="https://trychroma.com/">trychroma.com</a> — local vector database</li><li><strong>SmolLM</strong> — <a href="https://huggingface.co/HuggingFaceTB">huggingface.co/HuggingFaceTB</a> — small models designed for on-device use</li><li><strong>MCP specification</strong> — <a href="https://modelcontextprotocol.io/">modelcontextprotocol.io</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19a54c72dd7e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Running an AI Assistant Fully Offline With llama.cpp and a 135MB Model. Here’s How.]]></title>
            <link>https://medium.com/@Sanath_raj/running-an-ai-assistant-fully-offline-with-llama-cpp-and-a-135mb-model-heres-how-36f56da3853c?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/36f56da3853c</guid>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Sat, 23 May 2026 12:28:55 GMT</pubDate>
            <atom:updated>2026-05-23T12:35:46.449Z</atom:updated>
            <content:encoded><![CDATA[<p><em>No GPU. No API key. No cloud. Just a Ryzen CPU, llama.cpp, and a 135MB model that answered medical questions at 40 tokens per second.</em></p><p>Everyone opens a URL on their phone. They type questions and get answers streaming back in real time.</p><p>No one touched OpenAI. No data left the room. The entire thing ran on a simple laptop sitting on a desk.</p><p>That’s edge AI. And it’s more accessible than most people think.</p><h3>Why Edge AI Matters</h3><p>Every time you send a prompt to ChatGPT, three things happen that you don’t think about:</p><ol><li>Your data travels to a server you don’t control</li><li>You pay per token (or pay a subscription)</li><li>You need an internet connection</li></ol><p>For a lot of real-world use cases, those three things are dealbreakers.</p><p>A diagnostic assistant in a rural clinic with no reliable WiFi. A legal document tool where client data can never leave the premises. A factory floor assistant in an RF-shielded building. A healthcare chatbot that has to be HIPAA-compliant by architecture, not by policy.</p><p>In all of these, <strong>the model must come to the data — not the other way around.</strong></p><p>Edge AI is the answer. And thanks to tools like llama.cpp and the new generation of Small Language Models (SLMs), it’s no longer reserved for companies with dedicated ML infrastructure.</p><h3>The Stack</h3><p>Here’s everything I used. Total cost after the laptop: <strong>$0</strong>.</p><p>Component What it is <strong>SmolLM 135M</strong> A tiny language model by HuggingFace, designed for on-device use <strong>GGUF Q4_K_M</strong> A quantized format that shrinks the model 3x with minimal quality loss <strong>llama.cpp</strong> A C++ inference engine that runs GGUF models on CPU — and Vulkan GPU <strong>llama-server</strong> Built-in REST API server, OpenAI-compatible <strong>Python http.server</strong> Serves the demo HTML page to phones on the local network</p><p>The laptop: <strong>AMD Ryzen 5 5500U with integrated Radeon graphics</strong>. No discrete GPU. No CUDA. This is the kind of machine you’d find in an office, a classroom, or a field kit.</p><h3>Step 1: Get the Model</h3><p>I downloaded SmolLM 135M in GGUF Q4_K_M format from HuggingFace. The file is about 90MB — small enough to fit on basically anything.</p><p>Q4_K_M means 4-bit quantization with K-quant grouping at medium block size. The community’s consensus pick for CPU inference: best balance of size, speed, and quality.</p><p>The full pipeline if you’re generating it yourself in Google Colab:</p><pre>HuggingFace model (HF format)<br>        ↓  convert_hf_to_gguf.py<br>   GGUF F16  (~500MB for 135M)<br>        ↓  llama-quantize Q4_K_M<br>   GGUF Q4_K_M  (~90MB)  ← this is what you deploy</pre><h3>Step 2: Install llama.cpp on Windows</h3><p>The winget package is gone — llama.cpp moved repos. Go directly to the GitHub releases page:</p><pre>https://github.com/ggml-org/llama.cpp/releases/latest</pre><p>For Windows with an AMD or Intel CPU, download:</p><pre>llama-[version]-bin-win-vulkan-x64.zip</pre><p>Vulkan is the key insight here. <strong>The Ryzen 5 5500U has integrated Radeon graphics.</strong> Vulkan talks directly to it. You get GPU acceleration without NVIDIA, without CUDA, without any additional drivers.</p><p>Extract it and add to PATH:</p><pre>Expand-Archive -Path &quot;C:\Users\you\Downloads\llama-b9295-bin-win-vulkan-x64.zip&quot; `<br>               -DestinationPath &quot;C:\Users\you\llama.cpp&quot;</pre><pre>[Environment]::SetEnvironmentVariable(<br>    &quot;PATH&quot;,<br>    [Environment]::GetEnvironmentVariable(&quot;PATH&quot;,&quot;User&quot;) + &quot;;C:\Users\you\llama.cpp&quot;,<br>    &quot;User&quot;<br>)</pre><p>Close and reopen PowerShell, then verify:</p><pre>llama-cli --version<br># version: 9295 (95405ac65)<br># built with Clang 19.1.5 for Windows x86_64</pre><h3>Step 3: First Inference Test</h3><pre>llama-cli `<br>  --model &quot;C:\Users\you\Downloads\smollm_135M_q4_k_m.gguf&quot; `<br>  --prompt &quot;&lt;|im_start|&gt;user\nTranslate to French: Good morning.&lt;|im_end|&gt;\n&lt;|im_start|&gt;assistant\n&quot; `<br>  --n-predict 50 `<br>  --threads 6 `<br>  --ctx-size 512 `<br>  --no-display-prompt `<br>  --log-disable</pre><p>Output:</p><pre>Bonjour.<br>[ Prompt: 64.7 t/s | Generation: 39.8 t/s ]</pre><p><strong>39.8 tokens per second.</strong> On a budget laptop CPU with integrated graphics. No cloud, no GPU, no API key.</p><h3>Step 4: Launch the Local API Server</h3><pre>llama-server `<br>  --model &quot;C:\Users\you\Downloads\smollm_135M_q4_k_m.gguf&quot; `<br>  --host 0.0.0.0 `<br>  --port 8080 `<br>  --threads 6 `<br>  --ctx-size 2048 `<br>  --n-predict 80 `<br>  --log-disable</pre><p>--host 0.0.0.0 is the critical flag. It makes the server visible to every device on your local network — not just your own machine. That&#39;s what allows phones in the room to connect.</p><p>When you see HTTP server listening on 0.0.0.0:8080, the server is ready.</p><p>Test it:</p><pre># Write the request to a file (avoids PowerShell JSON escaping issues)<br>@&quot;<br>{<br>  &quot;model&quot;: &quot;smollm&quot;,<br>  &quot;messages&quot;: [<br>    {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a concise medical assistant. Answer in 1-2 sentences only.&quot;},<br>    {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What is the normal human body temperature?&quot;}<br>  ],<br>  &quot;max_tokens&quot;: 80,<br>  &quot;temperature&quot;: 0.1<br>}<br>&quot;@ | Out-File -FilePath &quot;C:\Users\you\request.json&quot; -Encoding utf8</pre><pre>curl.exe -s <a href="http://localhost:8080/v1/chat/completions">http://localhost:8080/v1/chat/completions</a> `<br>  -H &quot;Content-Type: application/json&quot; `<br>  -d &quot;@C:\Users\you\request.json&quot;</pre><p>Response:</p><pre>{<br>  &quot;choices&quot;: [{<br>    &quot;message&quot;: {<br>      &quot;content&quot;: &quot;The normal human body temperature is typically between 98.6°F (37°C) and 99°F.&quot;<br>    }<br>  }],<br>  &quot;timings&quot;: {<br>    &quot;prompt_per_second&quot;: 45.7,<br>    &quot;predicted_per_second&quot;: 37.9<br>  }<br>}</pre><p><strong>37.9 tokens per second. Correct answer. Zero internet.</strong></p><p>Notice the response also includes cached_tokens: 25 on the second request — the server cached the system prompt from the previous call. The KV cache means repeated requests get faster, not slower.</p><h3>Step 5: The Audience Demo UI</h3><p>I built a single HTML file that phones can open in a browser. No app install. No QR code app. Just a URL.</p><p>The key parts:</p><pre>const SERVER = &#39;http://192.x.x.x:8080&#39;;  // your laptop&#39;s local IP</pre><pre>async function ask() {<br>  const res = await fetch(`${SERVER}/v1/chat/completions`, {<br>    method: &#39;POST&#39;,<br>    headers: { &#39;Content-Type&#39;: &#39;application/json&#39; },<br>    body: JSON.stringify({<br>      model: &#39;smollm&#39;,<br>      messages: [<br>        {<br>          role: &#39;system&#39;,<br>          content: &#39;You are a concise medical assistant. Answer in 2-3 sentences only.&#39;<br>        },<br>        { role: &#39;user&#39;, content: question }<br>      ],<br>      max_tokens: 120,<br>      temperature: 0.1,<br>      stream: true   // stream tokens as they generate<br>    })<br>  });</pre><pre>  // Stream the response token by token<br>  const reader = res.body.getReader();<br>  // ... parse SSE chunks, update UI<br>}</pre><p>The endpoint is OpenAI-compatible. The streaming works exactly the same as the OpenAI API. Any tool or library built for OpenAI can point at http://192.x.x.x:8080 and work without any code changes.</p><p>Serve the HTML file:</p><pre>cd C:\Users\you<br>python -m http.server 3000</pre><p>Share the URL with the room:</p><pre>http://192.x.x.x:3000/demo.html</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*mP11wcaZMiya3DqhJQPn3w.png" /></figure><h3>What the Numbers Look Like</h3><p>Running SmolLM 135M Q4_K_M on a Ryzen 5 5500U via Vulkan:</p><p>Metric Value Generation speed ~40 tokens/sec Prompt processing ~45 tokens/sec Response time (short answer) 2–3 seconds Model file size ~90 MB RAM usage ~200 MB Power consumption ~15W (laptop idle level) Internet required None API cost $0</p><p>For context, a typical ChatGPT response streams at around 40–60 tokens/sec too — but that’s running on a cluster of A100s in a data center. We’re matching that on a laptop CPU.</p><h3>The Teaching Moment</h3><p>When the whole room hits the server at once, something interesting happens. Response times climb. The queue builds. Throughput drops.</p><p>That’s not a bug — it’s the lesson.</p><blockquote><em>“This is what resource contention looks like at the edge. In the cloud you’d spin up another instance. Here, this is all you’ve got. What would you do in production?”</em></blockquote><p>Answers: request queuing, rate limiting, a more capable device, or a smarter routing layer. All real engineering decisions that edge AI forces you to think about.</p><p>The second lesson comes from the privacy angle. Every prompt submitted by everyone in the room stayed on that laptop. No terms of service. No data retention. No model training on your inputs. That’s not a feature — it’s architectural.</p><h3>Limitations and Honest Trade-offs</h3><p>SmolLM 135M is genuinely tiny. It will:</p><ul><li>Ramble if you don’t constrain output length (--n-predict 80 helps)</li><li>Make things up with more confidence than it should</li><li>Struggle with complex multi-step reasoning</li><li>Occasionally produce nonsense on edge cases</li></ul><p>This is the quality-size-speed triangle. At 135M parameters you’re trading a lot of quality for portability. For a demo task like translation or simple Q&amp;A it works. For clinical decision support — absolutely not.</p><p>The right framing : <strong>this isn’t a replacement for GPT-4. It’s proof of concept that the deployment model works.</strong> Swap in a 1.7B or 7B model when you have more RAM and the quality improves dramatically.</p><h3>The Deployment Spectrum</h3><pre>CLOUD ←──────────────────────────────────────────→ EDGE</pre><pre>GPT-4        Llama 70B    Llama 8B    SmolLM 1.7B    SmolLM 135M<br>Data center  On-prem GPU  Gaming PC   Your Laptop    Raspberry Pi<br>Best quality High quality Good        Demo-grade     Proof of concept<br>$$$          $$           $           Free           $80 device</pre><p>For production edge deployment the sweet spot is usually 1B–7B models on devices with 8GB+ RAM. SmolLM 1.7B at Q4_K_M is ~1.1GB and runs comfortably on any modern laptop, delivering noticeably better quality than 135M.</p><h3>Choosing the Right Windows Binary</h3><h3>Choosing the Right Windows Binary</h3><p>The llama.cpp releases page provides multiple Windows builds depending on your hardware configuration.</p><ul><li>CPU only (no GPU acceleration)<br>→ llama-bin-win-cpu-x64.zip</li><li>AMD or Intel integrated graphics (most laptops)<br>→ llama-bin-win-vulkan-x64.zip</li><li>NVIDIA GPU<br>→ llama-bin-win-cuda-12.4-x64.zip</li><li>AMD Radeon discrete GPU (RX series)<br>→ llama-bin-win-hip-radeon-x64.zip</li></ul><p>For most users with a standard Ryzen or Intel laptop, the Vulkan build is the best option. It enables GPU acceleration through integrated graphics without requiring CUDA or an NVIDIA GPU.</p><p>The full pipeline — from loading a base model to running it locally on a laptop with an audience of phones hitting it simultaneously — is something you can build in a day. No expensive infrastructure. No paid APIs. No GPU required for inference.</p><p>Thats Edge AI.It’s here. And it runs on the laptop already in your bag.</p><h3>Resources</h3><ul><li><strong>llama.cpp</strong> — <a href="https://github.com/ggml-org/llama.cpp">github.com/ggml-org/llama.cpp</a></li><li><strong>SmolLM</strong> — <a href="https://huggingface.co/HuggingFaceTB">huggingface.co/HuggingFaceTB</a></li><li><strong>GGUF format</strong> — <a href="https://github.com/ggml-org/ggml">github.com/ggml-org/ggml</a></li><li><strong>HuggingFace GGUF models</strong> — search “GGUF” on huggingface.co for pre-converted models</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36f56da3853c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I tried to fine-tune Phi-3 Mini. Here’s everything that broke — and what it taught me about loss.]]></title>
            <link>https://medium.com/@Sanath_raj/i-tried-to-fine-tune-phi-3-mini-heres-everything-that-broke-and-what-it-taught-me-about-loss-42204d94891c?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/42204d94891c</guid>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Sun, 10 May 2026 12:18:06 GMT</pubDate>
            <atom:updated>2026-05-10T12:31:04.775Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Two bugs, four hours of wasted GPU time and also got a real feel of cross-entropy.</em></p><p>This weekend, I wanted to fine-tune a small language model on domain-specific question answering. I picked Microsoft’s Phi-3 Mini — 3.8B parameters, fast, runs on a free Colab T4. I found a notebook, loaded a medical QA dataset, and hit run.</p><p>Within minutes, the model was producing fluent Ukrainian. Then, after fixing that, it produced only commas and periods.</p><p>Here’s what actually happened, why it happened, and what the journey taught me about cross-entropy loss — the number you’re told to watch but rarely told to understand.</p><h3>The setup: why LoRA on a 3.8B model?</h3><p>Full fine-tuning of Phi-3 Mini requires around 60 GB of VRAM. A free Colab T4 has 16 GB. LoRA (Low-Rank Adaptation) bridges that gap by freezing the base model’s weights entirely and training only small adapter matrices injected into the attention layers.</p><p>Instead of updating 3.8 billion weights, you update roughly 0.5% of them — the adapter delta expressed as:</p><pre>W_effective = W_base + α · (A × B)</pre><p>Where A and B are small low-rank matrices, and α is a scaling factor. The base weights stay frozen; only the adapters are trained. It&#39;s a clever approximation that makes fine-tuning accessible on consumer hardware.</p><h3>What went wrong: two bugs, two broken outputs</h3><h3>Bug 1 — Garbled multilingual garbage</h3><p>The base model output mixed Ukrainian, French, and German characters at random. Generation just never stopped. The root cause: the EOS (end-of-sequence) token lookup for &lt;|end|&gt; silently returned unk_token_id. The model had no idea when to stop.</p><h3>Bug 2 — Punctuation collapse</h3><p>After fixing the EOS issue and running the fine-tuned model, output collapsed to just commas and periods. No words. Just punctuation. The cause: only 500 training samples across 1 epoch. The model learned that commas and periods are statistically safe tokens — they appear everywhere — and defaulted to producing nothing else.</p><h3>The three fixes</h3><pre># Fix 1 — safe EOS token resolution<br>eos_ids = list(set([tok.eos_token_id] + [<br>    tok.convert_tokens_to_ids(t) for t in [&quot;&lt;|end|&gt;&quot;, &quot;&lt;|endoftext|&gt;&quot;]<br>    if tok.convert_tokens_to_ids(t) != tok.unk_token_id<br>]))</pre><pre># Fix 2 — decode only new tokens, not the prompt<br>new_tokens = outputs[0][prompt_len:]</pre><pre># Fix 3 — minimum viable training<br>train_samples = 2000  # was 500<br>num_epochs    = 3     # was 1<br>max_length    = 512   # was 256</pre><p>Fix 1 builds a list of candidate stop tokens and filters out any that resolved to unk_token_id. Fix 2 prevents the model from &quot;echoing&quot; the prompt back into its own output. Fix 3 is the bluntest: 500 samples × 1 epoch simply isn&#39;t enough for a model to learn structure.</p><h3>Six other things that will silently kill your fine-tune</h3><h3>1. Dataset formatting</h3><p>This is the most common failure mode. The model needs consistent, structured formatting — Alpaca-style or ChatML. Raw text or inconsistent separators produce a model that can’t generalise.</p><pre>### Instruction:<br>Translate English to SQL<br>### Input:<br>Get all users from the table<br>### Response:<br>SELECT * FROM users;</pre><h3>2. Learning rate</h3><p>The default LR often fails with LoRA. Too low and you get no learning; too high and training becomes unstable. A good starting range:</p><pre>learning_rate = 2e-4  # solid starting point for LoRA<br>learning_rate = 1e-4  # more conservative alternative</pre><h3>3. LoRA rank too small</h3><p>A low rank limits model capacity. If your task is complex, increase it:</p><pre>r          = 16   # up from 8<br>lora_alpha = 32   # up from 16</pre><h3>4. Wrong target modules</h3><p>If LoRA isn’t applied to the right attention layers, adaptation is minimal. At minimum:</p><pre>target_modules = [&quot;q_proj&quot;, &quot;k_proj&quot;, &quot;v_proj&quot;, &quot;o_proj&quot;]</pre><h3>5. Dataset too small</h3><p>1,500 samples can work — but only if quality is consistently high. In practice, 3,000–10,000 samples is a safer floor. Synthetic augmentation helps when real data is scarce.</p><h3>6. Evaluating on loss alone</h3><p>Loss can look fine while outputs are still broken, and vice versa. Print sample generations during eval. Compare base vs fine-tuned side by side. Numbers alone are not the signal.</p><h3>Training progress</h3><p>Here’s how loss moved across the first 200 steps. Both training and validation loss track closely together.</p><p>Loss dropped ~0.29 over 100 steps. With only two checkpoints it’s hard to read the curve shape — ideally you’d log every 20–50 steps to see whether the descent is linear, plateauing, or accelerating. These numbers are still well above the &lt;2.5 target, which is expected this early in training.</p><h3>Understanding cross-entropy loss — what the number actually means</h3><p>Every fine-tuning tutorial tells you “loss should be below 2.5.” Almost none explain what that number is.</p><p>Cross-entropy is not MAE. It’s not RMSE. It’s a measure of how <em>surprised</em> the model is by the correct next token.</p><pre>loss = −log(p)</pre><p>Where p is the probability the model assigned to the correct answer at that position. At every token in the training text, the model predicts a distribution over its ~32,000 token vocabulary. The loss is the negative log of however much probability it gave the right answer.</p><p>Scenariop (assigned)</p><p>Loss = −log(p)</p><p>Reading”The sky is ___” → blue0.70 ==&gt;0.36 Confident, correct</p><p>”Patient has ___” → diabetes0.05 ==&gt;3.00 Very less Confident, Mostly wrong</p><p>Untrained model (random)1/32000 ==&gt;10.37Pure guessing</p><p>The loss for a full training run is the average of this value across every token in every sample. That’s the number falling during training.</p><h3>Why −log and not something simpler?</h3><p>You could penalise wrong answers with (1 − p) — it also goes from 0 to 1. But compare the two on a bad prediction:</p><pre># Model gives 10% probability to the correct answer<br>linear_penalty = 1 - 0.10  # = 0.90  (9× worse than 90% confident)<br>log_penalty    = -log(0.10) # = 2.30  (22× worse than 90% confident)</pre><p>The log penalty grows far faster near zero. That steep curve creates pressure for the model to not just be right, but to be <em>confidently</em> right — which is what produces fluent text rather than hedged, probabilistic mush.</p><blockquote>“Lower loss means the model is more confidently correct, on average, across all token positions in the training set.”</blockquote><p>~0.3Confident and correct</p><p>~1.5Learning something</p><p>~2.5Borderline — watch outputs</p><p>10+Untrained / broken</p><h3>The T4 time trap</h3><p>The original notebook estimated 35–40 minutes of training. The actual run clocked 0.07–0.08 iterations per second over 2,250 steps — closer to 4 hours. The fix is straightforward: use max_steps=300, which overrides epoch-based counting. You get a trained model in ~35 minutes without changing sample size or epoch count.</p><pre>trainer = SFTTrainer(<br>    ...<br>    max_steps=300,  # overrides epoch-based counting on free GPU<br>)</pre><h3>Key takeaways</h3><ul><li><strong>500 samples × 1 epoch is not enough.</strong> The model collapses to punctuation. Minimum: 1,500–2,000 samples × 3 epochs.</li><li><strong>EOS token lookup can silently fail.</strong> Always build a list of candidate stop tokens and filter out unk_token_id.</li><li><strong>Cross-entropy loss is average surprise.</strong> Lower means the model is more confidently correct across all token positions.</li><li><strong>The −log isn’t arbitrary.</strong> It punishes overconfident wrong answers exponentially. Linear penalties don’t create the same pressure.</li><li><strong>Use </strong><strong>max_steps on free GPUs.</strong> Epoch-based counting can wildly overshoot on large datasets and kill your Colab session.</li><li><strong>Don’t evaluate on loss alone.</strong> Print sample outputs. Compare base vs fine-tuned. The numbers don’t tell the whole story.</li></ul><p>If you take one thing away (this for me was)</p><blockquote>Fine-tuning a small model is less about finding the right architecture and more about data quality, token hygiene, and understanding what your loss metric is actually measuring. Get those right first, then tune everything else.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42204d94891c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Catastrophic Forgetting]]></title>
            <link>https://medium.com/@Sanath_raj/understanding-catastrophic-forgetting-1c5f35cd4e1c?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c5f35cd4e1c</guid>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Fri, 08 Nov 2024 11:10:04 GMT</pubDate>
            <atom:updated>2024-11-08T11:10:04.760Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Lyt91q3tey4XVaps" /><figcaption>Photo by <a href="https://unsplash.com/@sofiameli?utm_source=medium&amp;utm_medium=referral">Irina</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Catastrophic forgetting is a term that has stuck with me ever since I first encountered it in the context of machine learning. It encapsulates a profound challenge that many artificial intelligence systems face, particularly those that learn sequentially</p><h3>What is Catastrophic Forgetting?</h3><p>Catastrophic forgetting occurs when a neural network forgets previously learned information upon learning new information. This is especially prevalent in systems that are trained sequentially on different tasks. For instance, if a model is trained to recognize cats and then retrained to recognize dogs, it may completely lose its ability to recognize cats.</p><h3>The Mechanism Behind It</h3><p>The underlying mechanism of catastrophic forgetting can be attributed to the way neural networks adjust their weights during training. When new data is introduced, the model updates its weights to minimize the loss for the new task, often at the expense of performance on previously learned tasks. This leads to a significant drop in accuracy for earlier tasks, hence the term “catastrophic.”</p><h3>Why Does It Matter?</h3><p>Understanding catastrophic forgetting is crucial for developing more robust AI systems. In real-world applications, models often encounter new data over time and must adapt without losing prior knowledge. This is particularly important in fields like robotics, natural language processing, and autonomous driving, where continuous learning is essential.</p><h3>Strategies to Mitigate Catastrophic Forgetting</h3><h3>1. Regularization Techniques</h3><p>Regularization methods, such as Elastic Weight Consolidation (EWC), help preserve important weights associated with previously learned tasks while allowing others to change. By adding a penalty term to the loss function, EWC encourages the model to maintain performance on old tasks.</p><h3>2. Memory Augmentation</h3><p>Incorporating external memory systems can also mitigate forgetting. Techniques like memory replay involve storing examples from previous tasks and periodically retraining the model on these examples alongside new data. This helps reinforce old knowledge while integrating new information.</p><h3>3. Multi-Task Learning</h3><p>Training models on multiple tasks simultaneously can help prevent catastrophic forgetting. By sharing representations across tasks, models can learn more generalized features that are beneficial for all tasks involved.</p><h3>Real-World Applications</h3><p>Understanding and addressing catastrophic forgetting has significant implications across various domains:</p><ul><li>Robotics: Robots must adapt to new environments while retaining knowledge about previous ones.</li><li>Healthcare: AI systems in healthcare need to learn from new medical data without losing insights from earlier studies.</li><li>Natural Language Processing: Language models should evolve with new linguistic patterns while maintaining fluency in previously learned languages or dialects.</li></ul><h3>Implications for Generative AI</h3><p>In the context of generative AI, catastrophic forgetting can hinder the model’s ability to produce high-quality outputs across diverse domains. For example:</p><ul><li>Text Generation: A language model may excel at generating poetry but struggle with technical writing if it forgets its earlier training.</li><li>Image Synthesis: An image generation model trained on various styles might lose its ability to replicate specific artistic techniques when retrained on new styles.</li></ul><h3>Conclusion</h3><p>Catastrophic forgetting presents a unique challenge in the development of intelligent systems capable of lifelong learning. By employing strategies such as regularization techniques, memory augmentation, and multi-task learning, researchers can create more resilient AI that retains knowledge over time.</p><p>Links:<br><a href="https://cobusgreyling.medium.com/catastrophic-forgetting-in-llms-bf345760e6e2">https://cobusgreyling.medium.com/catastrophic-forgetting-in-llms-bf345760e6e2</a><br><a href="https://arxiv.org/abs/2308.08747">https://arxiv.org/abs/2308.08747</a><br><a href="https://www.yurts.ai/blog/navigating-the-challenges-of-fine-tuning-and-catastrophic-forgetting">https://www.yurts.ai/blog/navigating-the-challenges-of-fine-tuning-and-catastrophic-forgetting</a></p><p><a href="https://arxiv.org/pdf/1612.00796">https://arxiv.org/pdf/1612.00796</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c5f35cd4e1c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Anomaly Detection using AutoEncoders]]></title>
            <link>https://medium.com/@Sanath_raj/anomaly-detection-using-autoencoders-ad92aa0493d3?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad92aa0493d3</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[anomaly-detection]]></category>
            <category><![CDATA[autoencoder]]></category>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Fri, 17 Dec 2021 10:18:42 GMT</pubDate>
            <atom:updated>2021-12-17T10:18:42.332Z</atom:updated>
            <content:encoded><![CDATA[<p>Anomaly detection is the process of finding unusual behaviors /abnormalities in data. An anomaly can be understood as one that deviates significantly from the normal characteristics behavior of the data.</p><p>Let’s begin with what is Anomaly detection:</p><p>•Data points, events, and/or observations that deviate from a dataset’s normal behavior</p><p>•Depends on the ability to define what is normal</p><p>•Anomaly detection is the identification of data points, items, observations, or events that do not conform to the expected pattern of a given group.</p><p>•These anomalies occur infrequently</p><p><strong>What are the techniques we have for detecting Anomalies</strong>:</p><p>•Statistical process control charts</p><p>•Threshold based methods</p><p>•Box plot charts</p><p>•Clustering technique</p><p>• Using Autoencoders</p><p>•Automatic Outlier Detection</p><p>•Isolation Forest</p><p>•Minimum Covariance Determinant</p><p>•Local Outlier Factor</p><ul><li>One-Class SVM</li></ul><p><strong>Understanding Autoencoders:</strong></p><p>What is an autoencoder</p><p><em>Autoencoder is an artificial neural network that learns how to efficiently compress and encode data then learns how to reconstruct the data back from the reduced encoded representation to a representation that is as close to the original input as possible.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*09xkH-3fxar4oqF2HCqk7g.png" /></figure><p>•Idea behind Autoencoders</p><p><strong><em>Encoder</em></strong><em> : a function </em><strong><em>f</em></strong><em> that compresses the input into a latent-space representation. f(x) = h</em></p><p><strong><em>Decoder</em></strong><em>: a function </em><strong><em>g</em></strong><em> that </em>reconstructs<em> the input from latent space representation. g(h) ~ x.</em></p><p>Applications of autoencoders :</p><p>•Anomaly detection</p><p>•Data denoising (ex. images, audio)</p><p>•Information retrieval</p><p>•Dimensionality Reduction</p><p>•Feature Extraction</p><p>•Image colorization</p><p>So with these basics, we can now dive into the topic. Let’s take a simple use case that consists of records with and without anomalies. Below are the steps that we will use to approach the use case.</p><p><strong>Approach flow</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1011/1*KUSK-_baK01EpzcrHDJZLg.png" /></figure><p>Hands-on implementation:</p><p>Let us start with library imports</p><blockquote>import pandas as pd<br>import matplotlib as mpl<br>import matplotlib.pyplot as plt</blockquote><blockquote>import tensorflow as tf<br>from tensorflow.keras.models import Model<br>import numpy as np</blockquote><blockquote>from sklearn.model_selection import train_test_split<br>from sklearn.preprocessing import MinMaxScaler, StandardScaler</blockquote><blockquote>df = pd.read_csv(“Dataset_anomaly.csv”) train_data, test_data, _, _ = train_test_split(df.values, df.values[:,0:1], test_size=0.3, random_state=42)</blockquote><p>Using scaling to perform the data transformation. Filtering the data with and without anomalies after scaling. We will use normal data to train the model.</p><blockquote>scaler = MinMaxScaler()<br>data_scaled = scaler.fit(train_data)</blockquote><blockquote>train_data_scaled = pd.DataFrame(data_scaled.transform(train_data))<br>test_data_scaled = pd.DataFrame(data_scaled.transform(test_data))</blockquote><blockquote>normal_train_data= train_data_scaled[train_data_scaled[‘col_0’] == 0]]<br>anomaly_train_data = train_data_scaled[train_data_scaled[‘col_0’] == 1]]</blockquote><blockquote>normal_test_data= test_data_scaled[test_data_scaled[‘col_0’] == 1]]<br>anomaly_test_data = test_data_scaled[test_data_scaled[‘col_0’] == 1&#39;]]</blockquote><p>Here below we are implementing the autoencoder architecture. The initial layers are for the encoder that is used for data downsampling and followed by the decoder layer that is used to upsample the data. The last layer shall match the dimension of actual data, here in our example 224.</p><blockquote>class AutoEncoder(Model):<br> def __init__(self):<br> super(AutoEncoder, self).__init__()<br> self.encoder = tf.keras.Sequential([<br> tf.keras.layers.Dense(120, activation=”relu”),<br> tf.keras.layers.Dense(60, activation=”relu”),<br> tf.keras.layers.Dense(30, activation=”relu”),<br> tf.keras.layers.Dense(8, activation=”relu”)])<br> <br> self.decoder = tf.keras.Sequential([<br> tf.keras.layers.Dense(30, activation=”relu”),<br> tf.keras.layers.Dense(60, activation=”relu”),<br> tf.keras.layers.Dense(120, activation=”relu”),<br> tf.keras.layers.Dense(224, activation=”sigmoid”)])<br> <br> def call(self, x):<br> encoded = self.encoder(x)<br> decoded = self.decoder(encoded)<br> return decoded</blockquote><blockquote>model= AutoEncoder()<br>model.compile(optimizer=’adam’, loss=’mae’)</blockquote><blockquote>decoder_out_normal = model.predict(normal_test_data)</blockquote><blockquote>decoder_out_anomaly = model.decoder(anomaly_test_data)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tsLtT9FEzgcan5leHk3KTA.png" /></figure><p>The values from the decoder and actual data are compared to analyze the reconstruction for both abnormal and normal data. We can clearly see anomaly data has a large reconstruction loss</p><blockquote>reconstructions = model.predict(normal_test_data)<br>train_loss = tf.keras.losses.mae(reconstructions, normal_test_data)</blockquote><p><strong>Detecting anomalies: Using reconstruction loss to detect anomalies</strong></p><p>We will detect anomalies by determining how well our model can reconstruct the input data.</p><p>The reconstruction errors are considered to be anomaly scores. The threshold is then calculated by adding the np.mean and np.std of the reconstruction errors. The reconstruction errors above this threshold are considered to be anomalies</p><blockquote>threshold = np.mean(train_loss) + 1.5*np.std(train_loss)</blockquote><ul><li>Find MAE loss on training samples.</li><li>If the reconstruction loss for a sample is greater than this threshold value, we can infer that the model sees a pattern that it isn’t familiar with. We will label this sample as an anomaly</li></ul><p>Thanks for reading!!!</p><p>Credits:</p><p><a href="http://www.timeseriesclassification.com/">http://www.timeseriesclassification.com/</a></p><p><a href="https://towardsdatascience.com/anomaly-detection-for-dummies-15f148e559c1">https://towardsdatascience.com/anomaly-detection-for-dummies-15f148e559c1</a></p><p><a href="https://github.com/srivatsan88/">https://github.com/srivatsan88/</a></p><p><a href="https://keras.io/examples/timeseries/timeseries_anomaly_detection/">https://keras.io/examples/timeseries/timeseries_anomaly_detection/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad92aa0493d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using MLFLOW in Azure for Machine learning]]></title>
            <link>https://medium.com/@Sanath_raj/using-mlflow-in-azure-for-machine-learning-19be327fc3c4?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/19be327fc3c4</guid>
            <category><![CDATA[hyperopt]]></category>
            <category><![CDATA[mlflow]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Wed, 14 Oct 2020 10:24:35 GMT</pubDate>
            <atom:updated>2020-10-15T09:16:27.680Z</atom:updated>
            <content:encoded><![CDATA[<h3>Using MLFLOW in Azure for Machine learning with HyperOpt</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/280/1*psKbpp2GbTSAXdLVhioo7g.png" /></figure><p>Managing machine learning life cycle using MLFLOW</p><p>MLflow is an open source platform for managing the end-to-end machine learning lifecycle. It has following components:</p><p>Tracking: For tracking experiments and log records</p><p>Models: Used in managing and deploying various ML models</p><p>Projects: Helps is project and team management.</p><p>Model Registry: Stores version control details which is useful for internal audits/ reviews and PCA</p><p>Model Serving: Which is used for hosting and to connect with APIs</p><p>We will in this blog, cover the use of MLflow in 3 main areas : EDA , Model building and Tracking</p><p><strong>Getting started with MLFLOW for EDA</strong></p><p>We can save the results of EDA in MLflow as tables. These results may include univariate analysis , outlier values , capping and capped tables. We can instantiate the mlflow by mlflow.start_run():</p><p>We just have to do three things:</p><ol><li>Importing library</li><li>Writing the tables as csv to required path</li><li>Loading the path as part of MLflow log</li></ol><pre>import mlflow<br>dfPandas.to_csv(&quot;/dbfs/FileStore/tables/folder/filename.csv&quot;)<br><br>with mlflow.start_run():<br>mlflow.log_artifact(&quot;/dbfs/FileStore/tables/folder/filename.csv&quot;)</pre><p><strong>Using MLFLOW for model building and storing as experiments in sklearn</strong></p><p>For building models we will use mlflow.start_run() like in EDA and then we will define/ write the model building code inside the run() method.</p><pre>import mlflow.sklearn<br>from sklearn.metrics import mean_squared_error, r2_score<br>mlflow.set_experiment(&quot;/Databricks/user/MLFLOW/Linearmodel/iter1&quot;)<br><br>with mlflow.start_run():<br>  dataset_X_train = dftrain.Xcolumns<br>  dataset_X_test = dftest.Xcolumns<br>  dataset_Y_train = dftrain.YColumn<br>  dataset_Y_test = dftest.YColumn<br><br>  model= linear_model.LinearRegression() <br>  model.fit(dataset_X_train , dataset_Y_train)<br><br>  predY = model.predict(dataset_X_test )<br>  print(&quot;mse&quot;, mean_squared_error(dataset_Y_test, predY)<br>  mlflow.log_metric(&quot;mse&quot;, mean_squared_error(dataset_Y_test, predY)<br>  mlflow.log_metric(&quot;r2&quot;, r2_score(dataset_Y_test, predY))<br>  mlflow.log_artifact(&quot;linearmodel.csv&quot;)</pre><p><strong>Using Hyperopt with MLflow for model optimization with tracking in spark</strong></p><p>Hyperopt is an alternate way of hyperparameter tuning which is faster when compared to Randomsearch or Gridsearch method as this method uses Bayesian optimization. We have seen this method to be at least 10 times faster than gridsearch. Random search is more efficient than grid search with high dimensions, but it is an uniformed way.</p><p>Intuitively how Bayesian optimizer works is, based on earlier iteration and value of cost function, it will now try to converge more around the point that gave higher accuracy or lower loss. The usually used choice in Bayesian optimizer is Tree Parzen Estimator (TPE).</p><pre>import mlflow<br>from hyperopt import fmin, tpe, hp, anneal, Trials<br>from pyspark.ml.classification import RandomForestClassifier<br>from pyspark.ml.evaluation import BinaryClassificationEvaluator, MulticlassClassificationEvaluator<br>from pyspark.mllib.evaluation import MulticlassMetrics<br><br>mlflow.set_experiment(&quot;/Databricks/user/MLFLOW/Linearmodel/iter1&quot;)<br><br>model2= RandomForestClassifier(featuresCol = &#39;features&#39;, labelCol = &#39;ycol&#39;,  maxDepth=params[&#39;maxDepth&#39;],  numTrees=params[&#39;numTrees&#39;],           <br>impurity=params[&#39;impurity&#39;])<br><br>rfModel = model2.fit(train)<br>PredY = rfModel.transform(train)<br>evaluator1 = MulticlassClassificationEvaluator().setMetricName(&quot;accuracy&quot;)                             .setPredictionCol(&quot;prediction&quot;).setLabelCol(&quot;ycol&quot;)<br><br>Accuracy=evaluator1.evaluate(PredY)</pre><p>Defining the search space for hyperopt</p><pre>search_space = {<br>    &#39;maxDepth&#39;: hp.choice(&#39;maxDepth&#39;, range(8,12)),<br>    &#39;numTrees&#39;: hp.choice(&#39;numTrees&#39;, np.arange(450,650,50)),<br>    &#39;impurity&#39;: hp.choice(&#39;impurity&#39;, [&quot;gini&quot;, &quot;entropy&quot;])<br>}</pre><p>Running the experiment in MLFlow using trials() for spark model. <em>We can use SparkTrials() if we are running a sklearn model, however this will work only on drivernode, will not use any worker node.</em></p><pre>trials=Trials()<br>bestHyperParam = fmin(f,<br>            search_space,<br>            algo=tpe.suggest,<br>            max_evals=10,<br>            trials=trials,<br>            show_progressbar= True)<br>mlflow.log_params(bestHyperParam)</pre><p><em>*Suggestive reading — algos i.e tpe.suggest. Other options are tpe.rand.suggest &amp; anneal.suggest</em></p><p>We can check the results in experiment tab with the experiment name given. The experiment tab will have the experiment details captured with all metrics that are relevant for a classification model. We can even download the results as .csv file</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ZlDuKd47zU-DGm3Hmb7Zw.jpeg" /></figure><p>We can relevant all the runs in the experiment and click on compare to get a visual parallel plot which is dynamic in nature as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H1Hh2g0pXWjRl5_cZL2WsA.jpeg" /></figure><p>MLflow library is highly recommended for building machine learning models in your project, if you are using Azure stack. Please let us know if you liked mlflow + hyperopt</p><p>Thanks a lot for reading!!</p><p>Credits</p><p><a href="https://docs.databricks.com/applications/mlflow/index.html">https://docs.databricks.com/applications/mlflow/index.html</a></p><p><a href="https://databricks.com/product/managed-mlflow">https://databricks.com/product/managed-mlflow</a></p><p><a href="https://towardsdatascience.com/getting-started-with-mlflow-52eff8c09c61">https://towardsdatascience.com/getting-started-with-mlflow-52eff8c09c61</a></p><p><a href="https://towardsdatascience.com/hyperparameter-optimization-in-python-part-2-hyperopt-5f661db91324">https://towardsdatascience.com/hyperparameter-optimization-in-python-part-2-hyperopt-5f661db91324</a></p><p><em>Originally published at </em><a href="https://www.thinkai.org/post/using-mlflow-in-azure-for-machine-learning"><em>https://www.thinkai.org</em></a><em> on October 14, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19be327fc3c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting started with GCP for running ML models on cloud]]></title>
            <link>https://medium.com/@Sanath_raj/getting-started-with-gcp-for-running-ml-models-on-cloud-9e01cf6b7a62?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e01cf6b7a62</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Wed, 08 Jul 2020 06:26:31 GMT</pubDate>
            <atom:updated>2020-10-13T06:33:37.458Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/286/0*0NGxxalpRbilzZTN.png" /></figure><p>Deploying Tensorflow model in GCP</p><p>Sooner or later we are all going to use cloud computing for deploying various ML models and productionize it. One popular option we see is Google Cloud Platform (GCP) . GCP offered by Google, is a suite of cloud computing services that runs on the same infrastructure that Google uses, which provides a series of modular cloud services including computing, data storage, data analytics and machine learning.</p><p>Lets see how we can navigate within the GCP console and deploy our code or model. Assumption here is you already have created the account in GCP and you have active subscription. To know about how to create an account and start subscription, you can check here</p><p><strong>Step 1</strong></p><p>Get the project ID from the homepage. If project is not created please create a new project as explained in this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*DTVxXVi0WEygiyQ-l1HLKw.jpeg" /></figure><p><strong>Step 2</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/72/1*JR0NI-U0lTnXaOtRiD0SeA.png" /></figure><p>Use the navigation option</p><p>Create bucket in GCP using create storage option. Choose a Regional bucket and set a unique name and Please keep the <em>bucket </em>name and the <em>region </em>of the storage handy, which we will use in step 6.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/375/1*0f75W78TQhW5jzUu6Bq-ZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*xU36gGmzHEBrh4BdT8VKYQ.jpeg" /></figure><p><strong>Step 3</strong></p><p>From navigation option, Using AI platforms we can click on Notebooks options</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/921/1*x3I0OHLifHr5hY8RURM9Kw.jpeg" /></figure><p>On the Notebook instances page, click <strong>NEW INSTANCE</strong> .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/124/1*OJOkjxrEf1fWh674SXUqZQ.png" /></figure><p>Select TensorFlow 2.x with/without GPUs if you wish run a TF model. The new VM will take 2–3 minutes to start.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*kf4947BtyCcbx-JLfQlNRg.jpeg" /></figure><p>Once the VM is up, click on Open JupyterLAB</p><p>Once opened, you will be able to see the UI as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*WmBYMS7ffS_ou1vPjYVZVw.jpeg" /></figure><p><strong>Step 5</strong></p><p>Clone your repository from github or other location option using Terminal option</p><p>git clone <a href="https://github.com/GoogleCloudPlatform/your_repo_name">https://github.com/GoogleCloudPlatform/your_repo_name</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/93/1*16Z9Z83t7zwvDbuqjWjF4Q.jpeg" /><figcaption>Go to terminal , copy paste the repository path and hit enter</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DM-NXhVzj55ibmDB_t1_Tg.jpeg" /></figure><p>At the command-line prompt, type in the command and press Enter.</p><p>You will able be to see the progress and status as Done once completed.</p><p>Confirm that you have cloned the repository by double clicking on the folder and ensure that you can see its contents.</p><p><strong>Step 6</strong></p><p>Inside the repository click open the jupyter notebook you wish you to excute. Create a new cell. Please key in the project ID, bucket name and region name and enter. Execute rest of the code which is present as-is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HFH70BWEL0TasoOQRz4nmQ.jpeg" /></figure><p>Hope these steps helps you to deploy your first notebook on GCP. Thanks for reading!!</p><p><strong>Credits:</strong></p><p><a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects">https://cloud.google.com/resource-manager/docs/creating-managing-projects</a></p><p><a href="https://www.coursera.org/learn/gcp-fundamentals">https://www.coursera.org/learn/gcp-fundamentals</a></p><p><a href="https://en.wikipedia.org/wiki/Google_Cloud_Platform">https://en.wikipedia.org/wiki/Google_Cloud_Platform</a></p><p><em>Originally published at </em><a href="https://www.thinkai.org/post/getting-started-with-gcp-for-running-ml-models-on-cloud"><em>https://www.thinkai.org</em></a><em> on July 8, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e01cf6b7a62" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Download free copy of DEEP LEARNING WITH PYTORCH]]></title>
            <link>https://medium.com/@Sanath_raj/download-free-copy-of-deep-learning-with-pytorch-b716bbd68ae6?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/b716bbd68ae6</guid>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Wed, 08 Jul 2020 04:19:14 GMT</pubDate>
            <atom:updated>2020-07-08T04:28:51.462Z</atom:updated>
            <content:encoded><![CDATA[<p>Download the free copy of “Deep Learning with PyTorch” book by PyTorch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*DUfIwyHT-FFhGdIg.png" /></figure><p>Get a free copy of the full book and learn how to get started with AI / ML development using PyTorch</p><p>Deep Learning with PyTorch provides a detailed, hands-on introduction to building and training neural networks with PyTorch, a popular open source machine learning framework. This full book includes:</p><ul><li>Introduction to deep learning and the PyTorch library</li><li>Pre-trained networks</li><li>The mechanics of learning</li><li>Using a neural network to fit data</li><li>Using convolutions to generalize</li><li>Real-world examples: Building a neural network designed for cancer detection</li><li>Deploying to production</li></ul><p>Link to download : <a href="https://pytorch.org/deep-learning-with-pytorch">https://pytorch.org/deep-learning-with-pytorch</a></p><p>Step 1:</p><p>Fill in the two fields and submit</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*XQG8Rg64_OCBuMC5mBGZ0w.jpeg" /></figure><p>Step2:</p><p>Here you go, click on the link to grab your copy!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ozTG0Pu4ININDYtfYe6CDQ.jpeg" /></figure><p>Thanks for reading. Sit back and enjoy the book!!</p><p><em>Originally published at </em><a href="https://www.thinkai.org/post/download-free-copy-of-deep-learning-with-pytorch"><em>https://www.thinkai.org</em></a><em> on July 8, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b716bbd68ae6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Machine Learning Pipelines using Pyspark]]></title>
            <link>https://medium.com/@Sanath_raj/building-machine-learning-pipelines-using-pyspark-4547d30796eb?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/4547d30796eb</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[spark]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[pipeline]]></category>
            <category><![CDATA[introduction-to-pyspark]]></category>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Sat, 04 Jul 2020 07:43:43 GMT</pubDate>
            <atom:updated>2020-07-04T07:43:43.642Z</atom:updated>
            <content:encoded><![CDATA[<p>Using pipelines in Pyspark for Machine Learning</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K-2GDktS3fDMKMDgdBBnIQ.jpeg" /></figure><p>Let’s understand the concept of using Pipelines in spark using a simple dataset. We have to create a session.</p><pre>from pyspark.sql import SparkSession<br>spark = SparkSession.builder.appName(&#39;Session&#39;).getOrCreate()</pre><p>Lets read the dataset which is in csv format, using spark.read.csv</p><pre>titanicdata = spark.read.csv(&quot;titanic.csv&quot;, inferSchema=True, header=True)</pre><p>We can understand how the dataset looks like using display</p><pre>display(titanicdata)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gvb5S8KK3qOp-Y2QuAb75g.jpeg" /></figure><p>We can select the columns that are relevant for building a classification model. Using dataframe.select([cols]) to create a subset of dataset.</p><pre>Data2 = titanicdata.select([ &#39;Survived&#39;, &#39;Pclass&#39;,  &#39;Sex&#39;,  &#39;Age&#39;, &#39;SibSp&#39;,  &#39;Parch&#39;,  &#39;Fare&#39;,  &#39;Embarked&#39;])</pre><p>Using dataframe.na.drop() to filter the datasets for NAs or missing values</p><pre>Data3 = Data2.na.drop()</pre><p>Importing the required library for creating the objects for Pipeline.</p><pre>from pyspark.ml.feature import (VectorAssembler, VectorIndexer, OneHotEncoder, StringIndexer)</pre><p>We will convert ‘<em>sex</em>’ and ‘<em>embark</em>’ columns which are categorical in nature, to onehot encoded dummy variables. We will use string encoder function to convert the Male Female to index, where it simply converts the value to 0 or 1. After that we can use onehot encoder to get the dummy variable column created.</p><p>For ex: Male, Female,Male will be 0,1,0 after string encoding and next 0,1,0 will be [1,0],[0,1],[0,1] after onehot encoding</p><pre>gender_indexer = StringIndexer(inputCol=&#39;Sex&#39;,outputCol=&#39;genderindex&#39;)<br>gender_encoded =  OneHotEncoder (inputCol=&#39;genderindex&#39;, outputCol=&#39;genderVec&#39;)</pre><p>We will be using same procedure to convert the column <em>embark</em>. But on thing we must understand here, we have not transformed the data, just that we are creating these transformation objects for Pipeline</p><pre>embark_indexer = StringIndexer(inputCol=&#39;Embarked&#39;, outputCol=&#39;Embarkedindex&#39;)<br>embark_encoded = OneHotEncoder(inputCol=&#39;Embarkedindex&#39;, outputCol=&#39;EmbarkedVec&#39;)</pre><p>We will use VectorAssembler function to convert all the features into one column, For this we will be only using the columns that are in expected format like one-hot encoded or scaled features etc. Output of VectorAssembler function is combination of all selected column values.</p><pre>assembler = VectorAssembler(inputCols=[&#39;Pclass&#39;, &#39;genderVec&#39;, &#39;Age&#39;, &#39;SibSp&#39;, &#39;Parch&#39;, &#39;Fare&#39;, &#39;EmbarkedVec&#39;], outputCol = &#39;features&#39;)</pre><p>Lets create a DecisionTreeClassifier object which has 2 main attributes. ‘featuresCol’ , which is all x features combined into one column and ‘labelCol’ which is the target/y/label column in our dataset.</p><pre>from pyspark.ml.classification import DecisionTreeClassifier<br>dtc = DecisionTreeClassifier(featuresCol=&#39;features&#39;, labelCol=&#39;Survived&#39;)</pre><p>We can now use dataframe.randomSplit to split the data into train and test, with split ratio of 70% and 30 % in train and test respectively</p><pre>train_df,test_df = Data3.randomSplit([0.7,0.3])</pre><p>Now we can pass the train data into the Pipeline that has objects sequentially passed.</p><p>We can define the pipeline as below where stages are in accordance to the sequence of the transformation that are required to be done on the dataset. They are</p><ul><li>We shall be first doing String indexing of columns : Gender and Embark</li><li>Then we shall be doing the onehot encoding of columns that are string indexed</li><li>All the columns that are onehotencoded and other features, are now converted into one column using vector assembler.</li><li>Output of vector assembler is then passed on to a decision tree model</li></ul><pre>pipeline = Pipeline(stages= [gender_indexer, embark_indexer, gender_encoded, embark_encoded,assembler,dtc])</pre><p>Using fit and transform to transform the data and to create the model</p><pre>fit_model = pipeline.fit(train_df)</pre><p>We can use transform to get the predicted values of the data that is in test dataset</p><pre>test_results = fit_model.transform(test_df)</pre><p>test_results will have the predicted values. Prediction will contain both class prediction and probability values.</p><p>Using evaluation metrics to analyse the model performance.</p><pre>from pyspark.ml.evaluation import MulticlassClassificationEvaluator, BinaryClassificationEvaluator</pre><p>We can use MulticlassClassificationEvaluator function to get the accuracy measures</p><pre>acc_eval = MulticlassClassificationEvaluator(metricName=&#39;accuracy&#39;, predictionCol=&quot;prediction&quot;, labelCol=&quot;Survived&quot;)</pre><p>We can display the results by passing the test_data</p><pre>acc_eval.evaluate(test_results)</pre><p>We can see in the other post how to use gridsearch feature in Pyspark. Thanks for reading!!</p><p>Credits:</p><p><a href="https://www.udemy.com/user/joseportilla/">https://www.udemy.com/user/joseportilla/</a></p><p><a href="https://en.wikipedia.org/wiki/Apache_Spark">https://en.wikipedia.org/wiki/Apache_Spark</a></p><p><a href="http://spark.apache.org/docs/latest/quick-start.html">http://spark.apache.org/docs/latest/quick-start.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4547d30796eb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using PyTorch for building a Convolutional Neural Network (CNN) model]]></title>
            <link>https://medium.com/@Sanath_raj/using-pytorch-for-building-a-convolutional-neural-network-cnn-model-6e47ffb61b88?source=rss-72310a12e867------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e47ffb61b88</guid>
            <category><![CDATA[convolution-neural-net]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[cifar-10]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[pytorch]]></category>
            <dc:creator><![CDATA[Sanath Raj]]></dc:creator>
            <pubDate>Mon, 29 Jun 2020 05:10:47 GMT</pubDate>
            <atom:updated>2020-06-29T05:10:47.309Z</atom:updated>
            <content:encoded><![CDATA[<p>Lets see how do we use PyTorch library for building a simple CNN model for CIFAR dataset</p><p><strong>Import the libraries required in PyTorch</strong></p><pre>import torch<br>import torch.nn as nn<br>import torch.nn.functional as F<br>from torch.utils.data import DataLoader<br>from torchvision import datasets, transforms<br>from torchvision.utils import make_grid</pre><p>Loading the data from PyTorch repo using datasets function which is part of the utils helper function. And only transformation we wish to do on this data set is .ToTensor()</p><pre>transform =  transforms.ToTensor()</pre><p>Loading the train and test data</p><pre>train_data = datasets.CIFAR10( root = r&quot;c:\path&quot;, download=True, train= True, transform= transform)</pre><pre>test_data = datasets.CIFAR10( root = r&quot;c:\path&quot;, download=True, train= False, transform= transform)</pre><p>Loading the data into batches to optimize the memory and to avoid running into out-of-memory error. Here we are using the batch size of 100 images for both test and train</p><pre>train_loader = DataLoader(train_data, batch_size= 100, shuffle=True)<br>test_loader = DataLoader(test_data, batch_size = 100, shuffle=True)</pre><p>We can visualize the images that are loaded already using make_grid</p><pre>for images,labels in train_loader:<br>    break<br>im = make_grid(images[0:10], nrow= 5) <br>plt.figure(figsize=(10,4))<br>plt.imshow(np.transpose(im.numpy(),(1,2,0)))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*6AFUnbL-hip44ZxnpOLWcw.jpeg" /></figure><p>Lets check the dimension of each batch. As you can see below each batch has 100 images and each image is of the dimension [3, 32, 32], where 3 indicates 3 color channel and 32*32 is the image width and height</p><pre>images.shape</pre><p>torch.Size([100, 3, 32, 32])</p><p>We are going to build a CNN model which has architecture as given below.</p><p>The initial layers of CNN is usually referred to as a <em>sequential</em> layer, as you can see that sequences of convolution layer + relu + pooling is placed in sequence. Second part of the model is referred to as <em>classification </em>layer which actually does the job of classifying the images into its respective classes.</p><p>Features learning has happen in the initial layers of CNN model which finds patterns in an image. We can do that by convoluting over an image and looking for patterns. In the first few layers of CNNs the network can identify lines and corners, but we can then pass these patterns down through our neural net and start recognizing more complex features as we get deeper. It simply does convolve with a multiplication hence the name “convolutional neural network” indicates that the network employs a mathematical operation called convolution. This property makes CNNs really good at identifying objects in images.</p><p>In terms of architecture the image is converted to feature maps. The number of feature maps we want from an image is an arbitrary ( hyperparameter ). We use <em>conv2d </em>function of pytorch to get the image converted to feature maps. These feature maps are then downsampled using a maxpooling function. This step is performed multiple times depending upon the depth of the model we choose.</p><p>We use stride, filtersize and padding to decide the traversal of image or graph by kernels. By default the stride =1, which means on step size at a time. Filtersize the is dimension of the square filter , for example when filter is 3 , it means we are 3*3 square filter. Padding usually = 0, when set to any +ve value other 0, it add that many rows of values in the border of the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/0*OwwwESfNsuck--Lc.gif" /></figure><p>Below you can see how the data set when convoluted with a 3*3 square filter is converted to a feature map.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*UDajiVdbGO0ZriMN.gif" /></figure><p>Below illustration shows how the padding = 1, helps the filter to traverse corner pixels of the image more number of time than the earlier state where padding = 0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VCH2n-tZbyeiq7vF.gif" /></figure><p>What follows a conv2d is a pooling layer which is used for downsampling the image dimension. We usually use two types of pooling — Avg and Max pooling. Sharing the illustration below which is using the Maxpooling with filter size 2*2. Filter size 2*2 reduces the dimension of matrix or feature map to half,i.e. from m*m to [m/2]*[m/2]</p><p>Implementing the CNN architecture using PyTorch. We will use the nn module.</p><pre>class convCIFAR(nn.Module):<br>    def __init__ (self):<br>      super().__init__()<br>      <br>        self.conv1 = nn.Conv2d(3,5,5,1)  <br>        self.conv2 = nn.Conv2d(5,20,3,1)         <br><br>        self.fc1 = nn.Linear(6*6*20,120)<br>        self.fc2 = nn.Linear(120,80)<br>        self.fc3 = nn.Linear(80,10)<br><br> def forward(self,x):<br>        x = F.relu(self.conv1(x))<br>        x = F.max_pool2d(x,2,2)<br>        x = F.relu(self.conv2(x))  <br>        x = F.max_pool2d(x,2,2)<br>        <br>        x = x.view(-1,6*6*20)        <br>           <br>        x = F.relu(self.fc1(x))        <br>        x = F.relu(self.fc2(x))  <br>        x = F.log_softmax(self.fc3(x),dim = 1)    <br>        return(x)</pre><p>Lets now lets understand the sections of the code above</p><p>The each image inside batch is fed into model of batch size 100 each. So each batch will have dimension as <em>100*3*32*32</em></p><pre>for i , (X_train,y_train) in enumerate(train_data):     <br>    break<br>x = X_train.view(-1,3,32,32)</pre><pre>x.shape</pre><p>torch.Size([1, 3, 32, 32])</p><p>This signifies each image is having 3 color channel and is of the dim 32*32</p><pre>self.conv1 = nn.Conv2d(3,5,5,1)</pre><p>This convert the image which is 3 color channel to 5 feature maps with a filter of 5*5 and stride as 1</p><pre>self.conv2 = nn.Conv2d(5,20,3,1)</pre><p>In the next step we convert the already converted 5 layer feature map to feature maps of depth 20</p><p>When passed inside first conv layer</p><pre>X  = conv1(x)<br>X.shape</pre><p>torch.Size([1, 5, 28, 28])</p><p>The image is getting converted from 32*32 to 28*28 as 5*5 filter. And we are passing the same X to next layer</p><pre>X = F.max_pool2d(X,2,2)<br>X.shape</pre><p>torch.Size([1, 5, 14, 14])</p><p>We can clearly see 2*2 max pooling down-sampling the 28*28 to 14*14</p><p>Lets now understand the other part of model which used for classification. For classification we will be using linear layer of nn.module. For this layer it expects the data to be flattened, we will first flatten the dataset and then pass it to subsequent layers of self.fc1, self.fc2 and self.fc3. All the layers will passed to ReLU activation and last layer of the model activation used is soft-max which gives the probability distribution of the predicted class.</p><pre>x = x.view(-1,6*6*20) <br>self.fc1 = nn.Linear(6*6*20,120) <br>self.fc2 = nn.Linear(120,80) <br>self.fc3 = nn.Linear(80,10)</pre><p>The input for first linear layer , i.e. will e the output from the last layer of conv2d. Hence the output F.max_pool2d(X,2,2) of torch.Size([1, 5, 14, 14]) will be torch.Size([1, 20, 6, 6])</p><pre>X = F.max_pool2d(X,2,2)<br>X.shape</pre><p>torch.Size([1, 20, 6, 6])</p><p><strong>Forward and backward propagation</strong></p><pre>criterion = nn.CrossEntropyLoss() <br>optimizer = torch.optim.Adam(model.parameters(), lr = 0.001)</pre><p>Lets iterate it over 10 epochs. Lets create place holder for train accuracy and train loss</p><p>Lets now use model.forward to to do the forward pass. Then compare the predicted output with actual label and measure loss. We are using <em>CrossEntropyLoss </em>to measure the model performance.</p><pre>for i in range(epochs):<br>    trn_corr = 0<br>    tst_corr = 0<br>    <br>    # Run the training batches<br>    for b, (X_train, y_train) in enumerate(train_loader):<br>        b+=1<br>        <br>        # Apply the model<br>        y_pred = model(X_train)<br>        loss = criterion(y_pred, y_train)<br> <br>        # Tally the number of correct predictions<br>        predicted = torch.max(y_pred.data, 1)[1]<br>        batch_corr = (predicted == y_train).sum()<br>        trn_corr += batch_corr<br>        <br>        # Update parameters<br>        optimizer.zero_grad()<br>        loss.backward()<br>        optimizer.step()<br>        <br>        # Print interim results<br>        if b%100 == 0:<br>            print(f&#39;epoch: {i}    loss: {loss.item()}  accuracy: {trn_corr.item()*100/(100*b)}%&#39;)<br>        <br>    train_losses.append(loss)<br>    train_correct.append(trn_corr)</pre><p><strong>Credits:</strong></p><p><a href="https://en.wikipedia.org/wiki/Convolutional_neural_network">https://en.wikipedia.org/wiki/Convolutional_neural_network</a></p><p><a href="https://towardsdatascience.com/applied-deep-learning-part-4-convolutional-neural-networks-584bc134c1e2">Applied Deep Learning - Part 4: Convolutional Neural Networks</a></p><p><a href="https://www.tutorialspoint.com/signals_and_systems/convolution_and_correlation.htm">https://www.tutorialspoint.com/signals_and_systems/convolution_and_correlation.htm</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e47ffb61b88" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>