<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ilan Kushnir on Medium]]></title>
        <description><![CDATA[Stories by Ilan Kushnir on Medium]]></description>
        <link>https://medium.com/@ilankushnir?source=rss-5203293aeda0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xm8Cl4Jbc5LW6PSCxnSIvQ.png</url>
            <title>Stories by Ilan Kushnir on Medium</title>
            <link>https://medium.com/@ilankushnir?source=rss-5203293aeda0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 01:45:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ilankushnir/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[KWS Homelab v.2: the home infrastructure I live with (and the 3D‑modeled rack that went viral)]]></title>
            <link>https://medium.com/@ilankushnir/kws-homelab-v-2-the-home-infrastructure-i-live-with-and-the-3d-modeled-rack-that-went-viral-dbcd5d852340?source=rss-5203293aeda0------2</link>
            <guid isPermaLink="false">https://medium.com/p/dbcd5d852340</guid>
            <dc:creator><![CDATA[Ilan Kushnir]]></dc:creator>
            <pubDate>Tue, 27 Jan 2026 15:58:10 GMT</pubDate>
            <atom:updated>2026-01-27T17:03:04.575Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-d8u4ODnOkCdi4UFQLGi1A.png" /></figure><p>I’m <strong>Ilan Kushnir</strong>, a <strong>Tech Lead @ Dynamic Yield</strong>. Outside of work, I can’t resist turning curiosity into a real system: a place to learn, to tinker, to run the services my family and I actually use and ofcourse — over-engineering everything.</p><p>This post is a snapshot of my current homelab — what it’s built from, how it’s routed, and how I keep it <strong>powerful without turning it into a second full‑time job</strong>.</p><p>If you want the origin story (my previous iteration — a Kubernetes-based setup where I learned a ton), that chapter is here: <a href="https://medium.com/@ilankushnir/kws-cluster-e7e079cf103b">https://medium.com/@ilankushnir/kws-cluster-e7e079cf103b</a></p><p>🤙 <a href="https://makerworld.com/en/models/2139130-kws-rack-v-2-heavy-duty-10-inch-homelab-rack">The new KWS Rack v.2 model can be found here</a></p><h3>Where this iteration starts: “keep the learning, reduce the babysitting”</h3><p>My first serious homelab iteration went all‑in on <strong>Kubernetes</strong> and “production-style” tooling — <strong>Flux GitOps</strong>, <strong>Ansible, Terraform</strong>, and a lot of automation. It was an incredible learning experience… and it also turned me into a free‑time, amateur DevOps who had to keep the platform healthy.</p><p>This time, the design goal is different. I want to spend more time <strong>building and using</strong> things than <strong>maintaining a platform</strong> — while still keeping grown‑up ergonomics like observability, backups, and consistent routing. In other words: boring in the best way — predictable, recoverable, and easy to reason about.</p><p>That’s the theme of everything below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*iKw45I3o1jDqp2thNsLIgg.png" /></figure><h3>The rack: KWS Rack v2 (and the maker community surprise)</h3><p>One of my favorite parts of this journey isn’t software — it’s the physical side.</p><p>I designed <strong>KWS Rack v2</strong> end‑to‑end in <strong>Fusion 360</strong> and then printed + iterated it in the real world. It started as a simple itch (“why do most 10-inch options feel either flimsy or overcomplicated?”) and turned into a full modular rack system: I measured my gear, modeled the frame, tested tolerances, reworked joints, and repeated that loop until it felt like something you’d <strong>actually trust</strong> to hold dense homelab equipment.</p><p>What makes it special to me isn’t just that it’s 3D printed — it’s that it’s engineered to feel <strong>solid</strong>, <strong>expandable</strong>, and <strong>pleasant to build</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X1XWL5gbZqpn0NO4XFq7SQ.png" /></figure><h3>What it is (in plain terms)</h3><p>KWS Rack is a modular, heavy‑duty <strong>10-inch mini rack</strong> designed to hold real equipment — NAS units, power supplies, Raspberry Pi clusters, screens — while staying clean and maintainable.</p><p>The core idea: <strong>don’t print a “final rack”</strong>. Print a starting size that fits today, and then grow it as your homelab grows.</p><h3>Modular by design (print now, scale later)</h3><p>You can start small and extend without redesigning anything:</p><ul><li>Minimum size starts at <strong>3U</strong></li><li>It scales up to <strong>21U+ </strong>(and stays stable!)</li><li>A great starting point is the <strong>6U cube base</strong>, then you can add <strong>3U / 6U extensions</strong> whenever you want</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ur3xn4BOChR5ERUuc9nTLw.png" /></figure><h3>The “why” behind the design choices</h3><p>A few features I kept coming back to while designing it:</p><ul><li>A <strong>magnetic frame</strong> so small screws/tools can literally stick to the rack while you work (and it opens the door for magnetic add-ons in the near future)</li><li>A <strong>heavy-duty structure</strong> that doesn’t feel like a display prop — it’s stable enough for dense builds</li><li>A <strong>true 10-inch standard</strong> approach so it plays nicely with common 10-inch gear and accessories</li><li>Optional “make it look finished” upgrades: <strong>side + top panels</strong>, either printed or <strong>laser-cut 3mm acrylic</strong></li><li>Designed from day one to scale vertically <strong>without compromises</strong></li></ul><p>I’m also building an ecosystem around it — LED and cable management are part of that (some pieces are in progress / published separately).</p><h3>Upcoming modules (the ecosystem direction)</h3><p>This is the part that keeps it fun: the rack isn’t a single print, it’s a platform. Some modules I’m actively building toward:</p><ul><li><a href="https://makerworld.com/en/models/2233953-2u-rack-control-display-module#profileId-2521672">Screen + Raspberry Pi system monitor/controller module</a></li><li><a href="https://makerworld.com/en/models/2154991-patch-keystones-panel-2-3u-for-10-inch-kws-rack#profileId-2335937">Keystone patch panel with magnetic labels</a></li><li><a href="https://makerworld.com/en/models/2233529-server-rack-cable-management-mounts#profileId-2430416">Rack cable management mounts</a></li><li><a href="https://makerworld.com/en/models/2254100-synology-4bay-nas-module-for-10-inch-kws-rack-4u#profileId-2455107">Synology 4‑bay base module (remix)</a></li><li>2U snap‑in Raspberry Pi cluster (up to 8 Pis)</li><li>Power‑supply shelf with cable management</li><li>Magnetic removable back plate</li><li>UPS + PDU back‑panel holder</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9NEvwwUNmf1dWfaj8Acjhw.png" /></figure><h3>The community surprise</h3><p>I released the model for free because I love open maker building blocks — and I genuinely didn’t expect how much it would resonate.</p><p>It’s been showing up across self-hosting / homelab / maker communities (and getting remixes, prints, and new modules), which has been one of the most unexpectedly rewarding parts of this whole project.</p><ul><li><a href="https://makerworld.com/en/models/2139130-kws-rack-v-2-heavy-duty-10-inch-homelab-rack#profileId-2317125">MakerWorld</a></li><li><a href="https://www.reddit.com/r/minilab/comments/1prou2h/kws_rack_is_live_modular_heavyduty_10inch_mini/">Reddit</a></li></ul><p>The most satisfying part is that people have already started printing their own KWS Racks! you can see one <a href="https://makerworld.com/en/models/2139130-kws-rack-v-2-heavy-duty-10-inch-homelab-rack#profileId-2317125">leading Rack here on Reddit</a> and also some more makes on <a href="https://www.instagram.com/kws.rack/">KWS Rack instagram profile</a></p><h3>The architecture</h3><p>The core idea is simple: this setup isn’t “simple” in the sense of “tiny” — it’s a real home stack with a control plane, observability, backups, and services I use. What’s different this time is the goal: <strong>maximum maintainability per unit of capability</strong>.</p><p>I keep a few non‑negotiables:</p><ul><li><strong>One front door</strong> for HTTP services.</li><li><strong>Hostnames everywhere</strong> so I don’t live in a world of random ports.</li><li><strong>Only nodes (and a few stable endpoints) get IPs</strong> — apps stay behind the proxy.</li><li><strong>Git as the source of truth</strong>, because future‑me deserves documentation, not archaeology.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/959/1*-Uvn4YJI-X6a1TOCSM4Sbw.png" /></figure><h3>Nodes and roles (small fleet, clear boundaries)</h3><p>Everything lives on a DHCP‑reserved LAN (192.168.1.0/24) with a clear split of responsibilities:</p><p><strong>Raspberry Pi 5</strong> - the control plane</p><ul><li>Traefik (ingress)</li><li>Cloudflared (Cloudflare Tunnel connector)</li><li>Authentik (SSO / forwardAuth)</li><li>Komodo (Git-driven control plane for Compose stacks)</li><li>Prometheus + Grafana (+ exporters)</li><li>Kopia (backup UI + policies)</li></ul><p><strong>Raspberry Pi 4 </strong>- smart home lane</p><ul><li>Home Assistant (host networking)</li><li>Mosquitto (MQTT)</li></ul><p><strong>Raspberry Pi 4 </strong>- media automation + reading</p><ul><li>Plex ecosystem + arr stack</li><li>Audiobooks + reading services</li><li>Invite tooling</li></ul><p><strong>Raspberry Pi 4</strong> - general apps + automation + AI tooling</p><p><strong>Lenovo ThinkCentre M70q Gen 5</strong>, Proxmox - heavier workloads</p><ul><li>Video streaming services</li><li>Audio streaming services</li><li>Local lite LLMs</li></ul><p><strong>Synology DS923+</strong> - storage + backups</p><h3>Routing model: Cloudflare Tunnel → Traefik → service</h3><p>Externally, I avoid opening a dozen ports. Everything goes through a single pattern:</p><p><strong>Cloudflare Tunnel (single) → cloudflared → Traefik → service</strong></p><p>Internally, it stays consistent: I hit services by hostname and Traefik routes them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b6y9d-IpFLS4Ip-m7R5I7g.png" /></figure><h3>Authentication - “admin UIs shouldn’t be public by accident”</h3><p>A lot of homelab services ship with admin panels that were never meant to be internet-facing. My rule is simple:</p><ul><li><strong>Authentik</strong> provides SSO and acts as <strong>Traefik forwardAuth</strong> middleware.</li><li>Most admin UIs sit behind Authentik.</li><li>A few services intentionally <strong>don’t</strong> (e.g., Plex client compatibility; apps that use their own trusted login).</li><li>Some services are <strong>mixed</strong>: public invite paths, private admin UI (path‑based routing).</li></ul><p>This gives me a consistent “trust boundary” without complicated per-app networking.</p><h3>What I’m running (the stuff I actually use)</h3><p>I try not to think about this as “containers I deployed” — it’s closer to a home platform that supports a bunch of small, useful workflows. I keep responsibilities separated by node (so failures are contained), and I keep the public write‑up intentionally vague where it should be (especially around media automation).</p><h3>Control plane / management</h3><p>At the center is rpi00-control: it’s the practical “control plane” that gives everything else a consistent front door and a consistent way to manage and secure admin UIs.</p><p>Traefik handles ingress and hostname-based routing, Cloudflare Tunnel gives me external access without opening ports, Komodo keeps deployments GitOps-driven, and Authentik gives me a single authentication boundary.</p><h3>Observability</h3><p>I like knowing what’s happening without SSH’ing into five machines. Prometheus scrapes metrics (including node/container stats, Synology via SNMP, and Proxmox visibility), and Grafana turns it into dashboards I can glance at.</p><p>There’s also a small rack screen running a kiosk rotator. It’s not “necessary”… but it makes the rack feel like a living system, and it turns troubleshooting into a calmer, faster loop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fUC0hn9T5RD0kbGpehik4w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*axNTUbHG1sy3gqrG3pQQPQ.png" /></figure><h3>Backups (3‑2‑1 mindset, built into the routine)</h3><p>Backups are the difference between a hobby and a weekend-consuming incident. Proxmox runs vzdump backups to the NAS for VMs/LXCs, and Kopia handles app-data snapshots per node (so a failed SSD or a bad upgrade is an inconvenience, not a disaster). The “1” from the “3–2–1” is not implemented yet but I definitely will implement some offsite backup for my most important data in the near future.</p><h3>Smart home</h3><p>Smart home lives on its own node (rpi01-smarthome). Home Assistant is noisy by nature (lots of integrations, lots of events), so I keep it isolated. Mosquitto (MQTT) acts as the message bus for devices and automations.</p><h3>Media ecosystem (kept intentionally high‑level)</h3><p>On the media side, I run <strong>Plex</strong> (in a dedicated Proxmox LXC) and an <strong>arr stack</strong> for “media automation” and library workflows.</p><p>I’m intentionally not going into how content arrives here — this post is about architecture, reliability, and the joy of self‑hosting, not the mechanics behind it.</p><h3>Books, audiobooks, and reading tools</h3><p>This started as “let me host a couple of services” and turned into a surprisingly pleasant home library. Audiobookshelf and Kavita are the user-facing part, and Calibre-Web Automated help me organize every book/audiobook I buy.</p><p>I use:</p><ul><li><strong>Audiobookshelf</strong></li><li><strong>Kavita</strong></li><li><strong>Calibre-Web Automated</strong></li></ul><h3>Useful tools (tiny apps that remove friction)</h3><p>This is the category I keep expanding, because it quietly improves daily life. I’m running a self-hosted markdown editor for quick notes, Stirling PDF when I need a “Swiss army knife” for documents, a lightweight journaling app, IT‑Tools, and BookStack as the documentation home where things don’t rot.</p><p>I also have website change detection running (for those “tell me when this page changes” moments), Tube Archivist for organizing video content I care about, and Postiz for managing my social posting workflow.</p><p>In other words: small, boring utilities — the best kind.</p><h3>Automation + AI tooling (kept pragmatic)</h3><p>I love AI — but I try to keep it in the “useful assistant” bucket, not “new platform I have to babysit”. For automation I’m using <strong>n8n</strong>, I have an Open WebUI to unify local workflows, and I’m running a <strong>medium-sized local model</strong> on the Lenovo Proxmox box for experiments and practical home tasks — planning to add sometime in the future a Nvidia DGX Spark 😍 to run more serious LLMs.</p><h3>Personal Finance</h3><p>One newer addition I’m genuinely excited about is <strong>Nudlers</strong> (<a href="https://github.com/enudler/nudlers">https://github.com/enudler/nudlers</a>) - a privacy-first personal finance manager that feels especially promising — currently Israeli banks only, (and it’s extra fun knowing it’s built by people with serious fintech chops).</p><h3>The repo as living documentation (and a sanity tool)</h3><p>One of the best maintenance decisions I made is treating configuration like a product:</p><ul><li>inventory/ is the source of truth for nodes</li><li>stacks/&lt;node&gt;/ holds Compose stacks per node (clear ownership and boundaries)</li><li>stacks/rpi00-control/edge/dynamic/ is the centralized routing map (hostnames → IP:port)</li></ul><p>It means I can rebuild, migrate, or recover with much less guesswork. Also my AI coding assistant may benefit from the documentation, suggesting good deployment solutions and maintaining it easily.</p><h3>Closing thoughts</h3><p>As a dev, I spend my days thinking about systems that scale with teams. In the homelab, it’s the opposite: systems that scale with <strong>one person’s attention</strong>.</p><p>This setup is where I currently try to balance:</p><ul><li>enthusiast-grade capabilities,</li><li>day‑to‑day usability,</li><li>and maintainability that keeps it fun.</li></ul><p>If you’re building your own homelab and you’ve hit the “Kubernetes was fun… but now I want sleep” stage — I’d love to hear what you simplified and what you kept.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbcd5d852340" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Astra: PayU GPO’s Cutting-Edge AI Assistant]]></title>
            <link>https://medium.com/payu-engineering/astra-payu-gpos-cutting-edge-ai-assistant-829ce95dd890?source=rss-5203293aeda0------2</link>
            <guid isPermaLink="false">https://medium.com/p/829ce95dd890</guid>
            <dc:creator><![CDATA[Ilan Kushnir]]></dc:creator>
            <pubDate>Fri, 04 Jul 2025 12:41:45 GMT</pubDate>
            <atom:updated>2025-07-06T08:31:48.731Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f3q6mRB1tnHqcZoMIOsliA.jpeg" /></figure><p><em>By </em><a href="https://www.linkedin.com/in/kobicarmeli"><em>Kobi Carmeli</em></a><em>, </em><a href="https://www.linkedin.com/in/lonnieberry"><em>Lonnie Berry</em></a><em> &amp; </em><a href="https://www.linkedin.com/in/eyal-bahar"><em>Eyal Bahar</em></a></p><h3>Hackathon Highs: 1000 Minds, 18 Prizes, One Crazy Weekend</h3><p>When our company announced an internal GenAI hackathon supported by AWS back in 2023, excitement quickly spread through the office. With 1000 employees joining, everyone was eager to win the great prizes: nine PlayStations and nine iPhones. Our team took this challenge very seriously, combined our creativity, and ended up winning. The prize wasn’t just cool gadgets — we also got the amazing opportunity to turn our idea into a real product.</p><p>However, moving from the excitement of winning to actually building the product was not simple. We realized we needed a strong internal knowledge base to support products like RFP Master (an agent that should help us with handling RFPs quickly and accurately) and PayU Connect (a tool that closing the gap between engineers and account managers and helps them updating existing and new clients with our upcoming and new features).</p><p>To quickly provide value, we put together a small team to build an internal portal. At first, we decided to use Amazon Q, since it promised easy integration with knowledge bases without any coding, along with smooth SSO and permission management. We named our portal GPTo (as a montage to gepetto who created pincoiio) to show our goal of intuitive and easy knowledge management.</p><h3>Turning the hackathon win into real impact</h3><p>We pivoted fast. We switched from Amazon Q (which turned out to be a black box) to Amazon Bedrock’s RAG and LLM’s routing services while keeping all the agentic workflows and assitant logic on our side. With 1000 employees (250 of them being engineers), we decided that <strong>exposing Confluence</strong> would be our quickest win.</p><p>Why? Confluence was already full of gold-engineering docs, onboarding material, and runbooks for production issues-and nobody was enforcing strict page-level permissions. Better still, we wouldn’t need to build new habits or tools-just improve access.</p><p>We also made a critical decision: <strong>pay per token, not per seat</strong>. A subscription-based model meant high upfront costs and limited reach. With token billing, we made it accessible to everyone, without wasting budget on underused seats.</p><p>To make this work, we had to clean up our confluence spaces. We tagged stale or irrelevant pages with no-rag, eliminated duplicates - The result? An AI assistant that could finally answer internal questions accurately-and at scale.</p><p>Our team also handled all <strong>UX/UI design</strong> internally. We built a smooth experience for search and chat, added shortcuts for common questions, and designed clear feedback mechanisms. People started using the tool. We saw adoption from developers, product managers, even legal and support teams. The feedback was encouraging: finally, an assistant that feels like part of the team.</p><h3>A deeper dive into Astra project</h3><h4>The Business Need</h4><p>Teams frequently face challenges when trying to quickly access internal documentation, whether stored on platforms like Confluence or SharePoint. Astra addresses this issue by automating information retrieval and answering clarifications, allowing employees to redirect their efforts toward more strategic and value-added tasks. Moreover, Astra provides contextual responses specifically adapted to PayU’s unique business environment, ensuring the information delivered aligns accurately with internal practices, terminology, and data. Recognizing the sensitivity of corporate information, Astra incorporates rigorous security protocols and strict access control measures to ensure data remains protected throughout the entire interaction process.</p><h4>Core Capabilities</h4><p>Astra leverages Retrieval-Augmented Generation (RAG) technology, integrating pertinent internal documents directly into generated responses to provide precise and well-informed answers. Its dynamic conversation management capability enhances user experience by maintaining a coherent dialogue flow, tracking conversation history, and creating a natural, human-like interaction. Astra further refines responses through advanced prompt engineering, skillfully blending domain-specific context into both system and user-generated prompts. Additionally, it offers real-time streaming, delivering responses incrementally, token-by-token, which ensures a seamless and engaging conversational experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j87GgCgjV2nU08uhFTUykQ.png" /></figure><h3>Architectural Overview</h3><p>Astra’s backend is built on <strong>Node.js</strong> and <strong>Fastify</strong> with <strong>TypeScript</strong> for a performant, type-safe environment. This platform choice allows for rapid development and straightforward maintenance.</p><h4>Multi-Model LLM Integration</h4><p>Astra connects to various Large Language Models (LLMs), including:</p><ul><li><strong>OpenAI</strong> (e.g., GPT-4 and O3 for complex, in-depth responses)</li><li><strong>Amazon Bedrock</strong> (for specialized foundation models like Claude, Cohere)<br><em>we are also using a client for Fake LLM — for local testing</em></li></ul><p>This multi-model setup lets Astra match the right AI model to each query’s complexity or intent and it can be easily expanded to support newer models that being released on a monthly basis.</p><h4>Model Selection Logic</h4><p>Tasks are routed to specific models using an internal AIClientsManager abstraction:</p><ul><li><strong>GPT-4</strong> is selected for deep reasoning and document-based responses.</li><li><strong>Faster models</strong> are chosen for summarization or utility tasks like title generation.</li><li><strong>Configurations</strong> define model-to-task mappings dynamically.</li></ul><p>Currently, we have specific models preconfigured for different types of tasks. One of the features we’re working on is <strong>Auto Mode</strong>, which will enable Astra to automatically select the most suitable model based on the user’s intent. This will eliminate the need for users to manually choose a model or understand the underlying differences. At the same time, it will help us optimize costs by reserving more advanced, resource-intensive models for complex tasks — only when truly necessary.</p><h3>How Astra Retrieves Accurate, Contextual Answers</h3><p>At the heart of Astra’s accuracy is a process called Retrieval-Augmented Generation (RAG). Instead of solely depending on its LLM knowledge, Astra fetches the latest information directly from our company’s internal documentation. This ensures every answer is reliable, up-to-date, and tailored to our organization’s unique data.</p><p>When a user asks Astra a question, the system first decides if it needs to look into internal knowledge sources. If so, Astra identifies the right knowledge base based on factors like domain (e.g., engineering or security) and geographic platform (e.g., Enterprise or LATAM). It then reformulates the query for optimal retrieval and searches through documents using Amazon Bedrock’s powerful vector search.</p><p>Once documents are fetched, Astra processes and evaluates each one, considering factors like semantic relevance and how recently the document was updated. The most relevant documents are then included directly in the prompt to the AI model, clearly citing each source. Finally, Astra generates the response, highlighting whether the information came from internal documentation, general AI knowledge, or a combination of both — ensuring you always understand exactly where the information originated.</p><h4>Expanding Astra’s Reach: Integrated Web Search with SearxNG</h4><p>While internal documentation provides immense value, sometimes the most current or relevant information exists externally. Recognizing this, we integrated a powerful web search capability into Astra using SearxNG, an open-source, privacy-focused metasearch engine. Deployed directly within our Kubernetes clusters, SearxNG enriches Astra’s responses by seamlessly combining internal knowledge with real-time external data.</p><p>When a user asks a question that benefits from external sources-like recent technology trends, third-party integrations, or evolving compliance standards-Astra automatically queries our self-hosted SearxNG instance. This ensures users receive comprehensive, accurate, and timely answers without needing to switch tools or leave Astra’s interface.</p><p>This integration doesn’t just deliver external data; it intelligently enriches the responses based on the user’s context and query specifics. SearxNG respects privacy by anonymizing all searches, preventing user data exposure, and offers configurable search engines, ensuring results are relevant to PayU’s industry and operational context. Early feedback has been enthusiastic, with teams appreciating how effortlessly Astra blends internal precision with external insights-empowering quicker, better-informed decisions.</p><h3>Conversation Flow Management with LangChain’s StateGraph</h3><p>Astra’s conversations are managed with LangChain’s StateGraph, enabling structured, dynamic, and adaptive interaction flows.</p><h3>Key Nodes</h3><ul><li>title: Generates a clean, searchable title for each conversation.</li><li>retrieveFromKnowledgeBase: Executes the RAG workflow.</li><li>evaluateDocumentsRelevance: Applies scoring to determine if documents should be used.</li><li>completion: Generates the final AI response.</li><li>regularFlow: The fallback path using general LLM knowledge.</li><li>retrieveWorkFlow: Full document-based workflow.</li></ul><h4>Conditional Routing</h4><pre>graph.addConditionalEdges(&#39;shouldRetrieve&#39;, [<br>  [&#39;true&#39;, &#39;retrieveWorkFlow&#39;],<br>  [&#39;false&#39;, &#39;regularFlow&#39;]<br>]);</pre><h3>Example Flow</h3><p><strong>User:</strong> <em>“What’s the process for requesting a new IAM role in GECAD?”</em></p><p>Here’s how Astra processes this using its LangChain-powered StateGraph workflow:</p><ol><li>title node<br>Generates a descriptive title for conversation history: <br>➝ <em>“GECAD IAM Role Requests”</em></li><li>shouldRetrieve decision function<br>We retreive when a user select the internal knowledgebase as a data source - for this question he would probbably select it because it is internal data and this resolves to true.</li><li><strong>Route to </strong>retrieveWorkFlow<br>Activates the RAG flow. Astra connects to the Amazon Bedrock knowledge base for the GECAD and Engineering domains (the filters we got from the user).</li><li>retrieveFromKnowledgeBase node<br>Performs a filtered semantic search using the user query across indexed documentation.</li><li>evaluateDocumentsRelevance node <br>Applies a multi-factor scoring mechanism to rank the retrieved documents:<br>•<strong> Semantic similarity<br></strong>•<strong> Recency boost<br></strong>•<strong> LLM-generated relevance score</strong></li><li><strong>Decision: Relevant documents found</strong> <br>Top results include:<br>• <em>”GECAD IAM Role Creation Guide”<br></em>•<em> ”Security Workflow Overview for GECAD”</em></li></ol><p>The final step would be to pass them along with the user’s question as context to the selected model to generated a well structured and tailored response.</p><h3>Advanced Prompt Engineering &amp; Contextual Intelligence</h3><p>One of Astra’s core differentiators is its ability to generate highly contextual prompts on the fly. Every prompt is constructed dynamically using multiple context layers and formatting standards.</p><h3>Prompt Inputs</h3><ul><li><strong>System Role &amp; Domain Definition</strong> <br>Example: <em>“You are Astra, an AI assistant specialized in PayU’s </em>engineering<em> and </em>security<em> documentation.” </em>(user filters are being used here).</li><li><strong>Document Injection</strong> <br>Retrieved documents are added into the prompt with metadata:</li></ul><pre>📄 Document 1:  <br>Title: \\&quot;IAM Role Creation - GECAD\\&quot;  <br>Relevance: 0.92  <br>Source: Confluence  <br>Content Snippet: “Users must complete the JIRA request linked at…”</pre><h3>Conversation Summaries</h3><p>Both lowDetailSummary and highDetailSummary are included in every prompt to give the model an understanding of:</p><ul><li>The broader context (low-detail)</li><li>The most recent, fine-grained interactions (high-detail)</li></ul><p>This dual-summary strategy ensures Astra has enough clarity to generate highly contextual responses — even across long conversations — <strong>without exceeding token limitations</strong>.</p><p>more about this topic in the next sections…</p><h3>Document Relevance Scoring &amp; Ambiguity Handling</h3><h4>Multi-Factor Relevance Scoring</h4><p>To deliver accurate and trustworthy results, Astra doesn’t just retrieve documents — it evaluates them through a layered scoring system:</p><ol><li><strong>LLM-Based Relevance Grading</strong> <br>A dedicated, lightweight model assesses whether a document:<br>• Directly answers the user’s question<br>• Contains useful supporting context<br>• Is current and trustworthy<br>The result is a float score between 0.0 and 1.0.</li><li><strong>Temporal Recency Boost</strong> <br>Documents are boosted based on how recently they were updated:</li><li><strong>Weighted Final Score</strong> <br>Astra blends semantic similarity and recency into one cohesive score using a configurable weight (alpha, defaulted to 0.8):<br>• <strong>similarity</strong>: The vector or semantic match between the document and the query.<br>• <strong>recencyBoost</strong>: A calculated multiplier that slightly increases the score of more recently updated documents.<br>• <strong>Final Score</strong>: A weighted combination of the two, allowing Astra to prefer documents that are both contextually relevant and fresh.<br>This scoring model ensures that, for example, a slightly less relevant document from last week may still rank above a highly relevant document from two years ago-striking a critical balance between quality and recency.</li><li><strong>Sorting &amp; Filtering</strong><br>Once all documents have been scored, Astra performs a final selection step:<br>• <strong>Sort</strong> the documents from highest to lowest score.<br>• <strong>Filter out</strong> any documents below a minimum relevance threshold.<br>• <strong>Limit</strong> the number of documents injected into the prompt.<br>Only this final curated set is passed to the language model to avoid polluting the response with irrelevant or outdated information.</li></ol><h4>Code examples:</h4><pre>function logRecencyBoost(updatedAt: string, scale = 10): number {<br>  const deltaDays = (Date.now() - new Date(updatedAt).getTime()) / (1000 * 60 * 60 * 24);<br>  return 1 + Math.log(1 + scale / (1 + deltaDays));<br>}<br><br><br>function calculateFinalScore(similarity: number, updatedAt: string, alpha = 0.8): number {<br>  const recencyBoost = logRecencyBoost(updatedAt);<br>  return alpha * similarity + (1 - alpha) * recencyBoost;<br>}</pre><h3>Handling Ambiguous or Incomplete Queries</h3><p>In real usage, queries are rarely perfect. They may lack clarity, specificity, or context. Astra tackles this through a smart pre-retrieval decision engine.</p><h4>Retrieval Decision via LLM</h4><p>Before performing document retrieval, Astra invokes a small LLM to decide whether it’s even necessary. This helps prevent wasting compute on questions the model can confidently answer from its own training.</p><h4>LLM Evaluation Criteria:</h4><ul><li>Is the question related to internal processes, tools, or domains?</li><li>Is it unclear, vague, or missing key terms?</li><li>Could internal documentation meaningfully improve the answer?</li><li>Is the conversation history insufficient for context?</li></ul><p>If yes, retrieval proceeds. If no, Astra uses its general knowledge and may prompt for clarification.</p><h4>Query Reformulation</h4><p>When a query is deemed too vague but still retrieval-worthy, Astra reformulates it before sending it to the knowledge base.</p><h4>Reformulation Guidelines:</h4><ul><li>Add platform, product, and department names if known</li><li>Clarify intent: transform “IAM access” into “request IAM access”</li><li>Preserve the user’s original meaning</li></ul><h4>Example</h4><p><strong>User’s Original Input:</strong></p><blockquote><em>“IAM access”</em></blockquote><p><strong>Reformulated Query:</strong></p><blockquote><em>“What is the process for requesting IAM access in the GECAD platform, and where can I find the relevant JIRA form?”</em></blockquote><p>This refined version helps the vector search engine find significantly more relevant and higher-quality documents.</p><h3>Summarization Strategy &amp; Token Window Management</h3><p>One of the main technical challenges when working with large language models is the <strong>token limit</strong> — the total number of words the model can consider at once. Astra solves this with a multi-layered, progressive summarization system that balances context and brevity.</p><h4>Dual-Level Summarization</h4><p>Astra maintains two parallel summaries for each conversation:</p><ol><li><strong>Low-Detail Summary<br></strong>• High-level overview of the entire conversation so far<br>• Updated infrequently<br>• Formatted in concise bullet points</li><li><strong>High-Detail Summary<br></strong>• Detailed summary of the most recent N messages (our N is 5)<br>• Preserves nuance, intent, and clarifications<br>• Regenerated more frequently to reflect current focus</li></ol><p>These summaries are injected into every LLM prompt to ensure consistent understanding and context history awareness, without exceeding token limits.</p><h4>Example:</h4><pre>&lt;low_detailed_summary&gt;<br>- The user is working on access control policies in GECAD<br>- Earlier questions covered SSO, account creation, and IAM requests<br>&lt;/low_detailed_summary&gt;<br><br>&lt;high_detailed_summary&gt;<br>- The user now needs the link to the JIRA form for new IAM roles<br>- They&#39;ve already reviewed the Confluence access guide<br>&lt;/high_detailed_summary&gt;</pre><h3>Summary Merging</h3><p>To ensure continuity and preserve key ideas from longer threads, Astra merges existing summaries into a compact new version when necessary.</p><h4>Merge Prompt Template:</h4><pre>You are an expert summarizer. You will receive two summaries:<br>- A low-detail summary with a concise overview.<br>- A high-detail summary with in-depth insights.<br><br>Your task is to merge these two inputs into one comprehensive summary that captures all the key points from the conversation history from the beginning.<br><br>Instructions:<br>1. Combine both summaries to cover all essential insights from the entire conversation.<br>2. Clearly indicate that the output is a summarization of the conversation history from the beginning.<br>3. Preserve the brevity of the low-detail summary while incorporating important details from the high-detail summary.<br>4. Eliminate redundancy by merging similar points and rephrasing as necessary.<br>5. Format the final summary in clear, well-structured bullet points.<br><br>&lt;HighDetailSummary&gt;<br>{highDetailSummary}<br>&lt;/HighDetailSummary&gt;<br><br>&lt;LowDetailSummary&gt;<br>{lowDetailSummary}<br>&lt;/LowDetailSummary&gt;</pre><p>The merged summary is then stored in the database and injected into future prompts, maintaining context efficiently across long-running conversations.</p><p>This allows Astra to maintain awareness of:</p><ul><li>The user’s overarching goal</li><li>Prior clarifications and decisions</li><li>Shifts in topic or focus</li></ul><p>…all without overwhelming the token window.</p><h3>Real-Time Streaming with SSE (Server-Sent Events)</h3><p>To create a smooth and engaging user experience, Astra streams responses to the frontend in real time using <strong>Server-Sent Events (SSE)</strong> — a unidirectional, lightweight alternative to WebSockets.</p><h4>Why SSE?</h4><ul><li>✅ <strong>Ideal for one-way data streams</strong> (server ➝ client)</li><li>✅ <strong>Simpler and lighter</strong> than WebSockets</li><li>✅ <strong>Native support</strong> in most browsers</li><li>✅ <strong>Built-in retry and reconnection logic</strong></li></ul><h4>Astra’s Custom SSE Implementation</h4><p>Astra uses a fully custom implementation built on top of Fastify:</p><h4>Our Custom Event Types:</h4><pre>export const ConversationEventTypes = {<br>  ERROR: &#39;error&#39;,<br>  METADATA: &#39;metadata&#39;,<br>  MESSAGE_START: &#39;message_start&#39;,<br>  TITLE_GENERATION: &#39;title_generation&#39;,<br>  MESSAGE_COMPLETE: &#39;message_complete&#39;,<br>  MESSAGE_CONTENT_DELTA: &#39;message_content_delta&#39;,<br>} as const;</pre><p>Each type represents a different phase of the streaming experience:</p><ul><li>message_start – Indicates that Astra has received the query and is beginning to generate a response.</li><li>message_content_delta – The core of the streaming process: sends chunks of the response as they’re generated (token by token).</li><li>message_complete – Signifies that the full response has been sent.</li><li>metadata – Includes performance-related data like elapsed time, model name, and any debug flags.</li><li>title_generation – Sent when Astra auto-generates a title for the conversation thread inside the workflow state graph.</li><li>error – If an error occurs mid-stream (e.g. LLM failure, retrieval error), this sends a graceful structured message without killing the connection - which means, the client will receive the error as an event.</li></ul><h4>Backend Implementation Highlights</h4><p>Astra’s Fastify-based server defines a custom EventsStream class to manage SSE:</p><pre>export class EventsStream implements EventsStreamer {<br>  constructor(private readonly reply: FastifyReply) {}<br><br>  start(): void {<br>    this.reply.raw.writeHead(200, {<br>      Connection: &#39;keep-alive&#39;,<br>      &#39;Content-Type&#39;: &#39;text/event-stream&#39;,<br>      &#39;Cache-Control&#39;: &#39;no-cache&#39;,<br>    });<br>    this.reply.isSseActive = true;<br>  }<br><br>  sendEvent(event: string, data: object): void {<br>    const jsonData = JSON.stringify(data);<br>    const escapedData = jsonData<br>      .replace(/\n/g, &#39;\ndata: &#39;)<br>      .replace(/\r/g, &#39;&#39;)<br>      .trim();<br>    const sseMessage = `event: ${event}\ndata: ${escapedData}\n\n`;<br>    this.reply.raw.write(sseMessage);<br>  }<br><br>  end(): void {<br>    this.reply.raw.end();<br>    this.reply.isSseActive = false;<br>  }<br>}</pre><h3>Error Handling and Recovery</h3><p>Astra’s streaming implementation is designed to be robust and user-friendly, even when things go wrong mid-response.</p><p>Instead of abruptly closing the stream or returning a generic HTTP 500 error, Astra emits a <strong>structured </strong>error event within the SSE channel:</p><h4>Error Event Example:</h4><pre>sendError(messageId: string, error: Error): void {<br>  this.sendEvent(&#39;error&#39;, {<br>    id: messageId,<br>    error: error.message,<br>  });<br>}</pre><p>This allows the frontend to:</p><ul><li>✅ <strong>Display user-friendly error messages</strong> without disrupting the conversation view.</li><li>🔁 <strong>Retry gracefully</strong> or offer fallback suggestions based on the nature of the error.</li><li>🧑🏻‍💻 <strong>Log debug metadata</strong> such as messageId, error type, and timing to assist in issue tracking.</li></ul><p>By emitting structured error events as part of the same stream, Astra avoids breaking the conversational experience and empowers the frontend to handle issues more intelligently.</p><h4>Dual Error Handling Approach</h4><p>Astra distinguishes between streaming and non-streaming flows for robust error handling:</p><ol><li><strong>Standard API Requests (non-streaming):<br></strong>• Use conventional HTTP status codes (e.g., 400, 403, 500)<br>• Return error messages in a JSON payload</li><li><strong>Streaming Requests (SSE):<br></strong>• Errors are <strong>streamed</strong> as structured SSE events (not returned as full responses)<br>• The connection stays alive, so the UI doesn’t freeze or crash<br>• Enables partial completion or fallback messaging</li></ol><h4>Key Benefits of Astra’s SSE Layer</h4><p>🚀 <strong>Fast UX</strong> — Users begin reading responses almost immediately <br>🛡️ <strong>Resilient by design</strong> — Errors don’t crash sessions, the stream continues <br>⚡ <strong>Lightweight and scalable</strong> — Uses a single persistent HTTP connection<br>🎯 <strong>Built for structured interaction</strong> — Every piece of the conversation (title, deltas, metadata) has its own dedicated event</p><h3>Security &amp; Safety Guardrails</h3><p>Building an AI assistant that operates on internal company knowledge requires a rock-solid foundation of <strong>data security, access control</strong>, and <strong>ethical guardrails</strong>. Astra was designed from day one to uphold these standards.</p><h4>Prompt-Level Guardrails</h4><p>Astra embeds safety directly into its system prompts. These instructions shape the assistant’s behavior, telling it what <strong>not</strong> to say — even if technically possible.</p><h4>Example: Employee Privacy Rules</h4><pre>Employee Privacy Guardrails:<br>- PROHIBITED: Sharing performance reviews or feedback about employees<br>- PROHIBITED: Speculative or evaluative language<br>- RULE: When in doubt, exclude the information entirely</pre><h4>Example: Internal Info Protection</h4><pre>Internal Information Guardrails:<br>- PROHIBITED: Login credentials, security keys, or infrastructure details<br>- PROHIBITED: Unreleased product features or sensitive strategy documents<br>- PROHIBITED: Speculative statements or anything that could harm internal trust<br>- RULE: Only share factual, published, and context-appropriate documentation</pre><p>These embedded instructions ensure that Astra operates as a safe, reliable assistant — never a source of internal leaks, rumors, or unvetted content.</p><h4>Domain &amp; Platform Scoping</h4><p>To ensure that Astra provides only the most relevant and appropriate answers, it supports granular filtering by both <strong>domain</strong> and <strong>platform</strong>.</p><p>This is essential for organizations like PayU GPO that span multiple business units, geographies, and technical ecosystems.</p><h4>Domain Examples:</h4><ul><li>engineering</li><li>security</li><li>product</li><li>data</li></ul><h4>Platform Examples:</h4><ul><li>global</li><li>latam</li><li>gecad</li><li>enterprise</li></ul><p>Each assistant instance — or even individual query — can be scoped to a specific slice of the organization.</p><h4>Example:</h4><p>An Engineer from the GECAD platform asking about onboarding flow will receive documents tagged with:</p><pre>platforms: [&#39;gecad&#39;]<br>domains: [&#39;engineering&#39;]</pre><p>but he also can easily change the filters according to his needs!</p><p>This ensures:</p><ul><li>✅ <strong>No cross-regional confusion</strong> — Users receive answers tailored to their target region or business unit.</li><li>✅ <strong>High relevance</strong> — The assistant pulls from only the most relevant documents (to be more specific — from the relevant confluence spaces).</li><li>✅ <strong>Cleaner, faster prompts</strong> — Smaller, more focused context windows improve LLM performance and reduce token cost.</li></ul><p>By enforcing platform and domain scoping, Astra behaves more like a knowledgeable internal expert — not a generalist guessing across departments.</p><h4>Attribution &amp; Transparency</h4><p>Astra doesn’t just provide answers — it <strong>shows its work</strong>.</p><p>Every informative response begins with a <strong>source attribution header</strong> based on what knowledge was used to generate the answer:</p><ul><li>📚 <strong>Based on the Internal Knowledge Base</strong> — When the response is grounded in internal documentation.</li><li>🧠 <strong>Based on General Knowledge</strong> — When no relevant documents are found, and the answer is generated from the model’s internal knowledge.</li><li>📚🧠 <strong>Based on the Internal Knowledge Base and General Knowledge</strong> — When both are combined in the response.</li></ul><p>This source banner appears <strong>at the top of every response</strong>, creating instant clarity and building trust with users.</p><h4>Example:</h4><blockquote>📚 <strong>Based on the Internal Knowledge Base</strong> <br>You can request a new IAM role in GECAD by submitting a JIRA form found <a href="https://confluence/iam-request">here</a>…</blockquote><h4>Inline Document Citations</h4><p>Astra also includes <strong>inline markdown links</strong> to every document used in its response. Each citation includes:</p><ul><li>The document title</li><li>A source emoji 📄</li><li>A clickable link to the original source (e.g., Confluence)</li></ul><h4>Example:</h4><blockquote><em>You should complete the onboarding checklist described in 📄 </em><a href="https://confluence/onboarding-checklist"><em>Employee Onboarding Guide</em></a><em>.</em></blockquote><p>These links serve two key purposes:</p><ul><li>✅ <strong>Transparency</strong> — Users can validate the source</li><li>✅ <strong>Productivity</strong> — Direct access to full documents when needed</li></ul><h3>Real-World Feedback &amp; Impact</h3><p>Since its internal rollout, Astra has already started to prove its value — helping teams work faster, find answers quicker, and rely less on scattered documentation or Slack threads.</p><p>One particularly powerful endorsement came from <strong>Tomer Guttman</strong>, our SRE Team Lead at PayU GPO.</p><h4>🛠️ Real Problem, Real Impact</h4><p>Tomer encountered a complex infrastructure issue that required understanding multiple internal systems and historical decisions — something traditional documentation search tools couldn’t handle effectively.</p><p>He turned to Astra and asked a question phrased in natural language.</p><blockquote><em>“Astra helped me find exactly what I needed — including documents I didn’t even know existed. I tried asking ChatGPT and Claude, but only Astra understood the internal structure well enough to give me a usable answer.”</em></blockquote><blockquote><em>- </em><strong><em>PayU’s SRE Team Lead</em></strong></blockquote><p>Why did Astra succeed where others failed?</p><ul><li>✅ It pulled from <strong>PayU-specific Confluence pages</strong></li><li>✅ It understood internal acronyms and context</li><li>✅ It filtered out outdated or irrelevant documents</li><li>✅ It cited exact locations in the docs</li></ul><p>This kind of real-world use is exactly what Astra was built for.</p><h3>Future Roadmap</h3><p>Astra is just getting started. Here’s what’s coming next in our roadmap:</p><h4>📦 More Integrations</h4><ul><li>SharePoint, Google Drive, and internal ticketing systems</li><li>Snowflake Semantic Layer for structured data access</li></ul><h4>🧠 Enhanced AI Capabilities</h4><ul><li>Multi-agent workflows (e.g., researcher + summarizer agents)</li><li>Long-term user memory for personalization</li></ul><h4>🔐 Private RAG Domains and Agents</h4><ul><li>Isolated knowledge bases for specific departments</li><li>Custom assistants/agents per business use case</li></ul><h4>🪄 Smarter Prompts</h4><ul><li>Domain-specific prompt tuning</li><li>Contextual memory improvements with learned habits</li></ul><h3>Final Thoughts</h3><p>Astra is more than just a tool — it’s a <strong>co-pilot for the PayU ecosystem</strong>, designed to understand our language, systems, and workflows better than any off-the-shelf AI could.</p><p>It was architected for:</p><ul><li>🧠 Knowledge clarity</li><li>💬 Conversational fluidity</li><li>🔐 Enterprise-grade safety</li><li>⚡ Developer performance</li></ul><p>If you’re working at a company that struggles with internal knowledge chaos — and you’re tired of endless Confluence digging, Slack messages, or “just ask Bob” culture — Astra is a blueprint for how internal AI assistants can work at scale.</p><p>Stay tuned. This is just the beginning. 🚀</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VarjO9SqHUe3VKBlozuWYA.png" /></figure><p><em>Want to learn more or build your own internal assistant? Reach out to the Beyond team or follow future technical deep dives on our engineering blog.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=829ce95dd890" width="1" height="1" alt=""><hr><p><a href="https://medium.com/payu-engineering/astra-payu-gpos-cutting-edge-ai-assistant-829ce95dd890">Astra: PayU GPO’s Cutting-Edge AI Assistant</a> was originally published in <a href="https://medium.com/payu-engineering">payu-engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[KWS Cluster]]></title>
            <link>https://medium.com/@ilankushnir/kws-cluster-e7e079cf103b?source=rss-5203293aeda0------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7e079cf103b</guid>
            <category><![CDATA[homelab]]></category>
            <category><![CDATA[hobby]]></category>
            <category><![CDATA[self-hosted]]></category>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Ilan Kushnir]]></dc:creator>
            <pubDate>Fri, 29 Apr 2022 22:57:15 GMT</pubDate>
            <atom:updated>2022-05-26T21:15:46.388Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*bnYnLEDMeUahrsuK2MJg5Q.jpeg" /></figure><h3>My Kubernetes Homelab project</h3><h3>TL;DR</h3><p>I got into the world of Homelabs and built my home K8S cluster out of 4 Raspberry PIs and another RPI as a NAS server. It allowed me to practice K8S, GitOps concepts, monitoring, some networking, and also to deploy A LOT OF VERY COOL SELF-HOSTED APPS! — like my own Spotify, media server, network-level ad-blocker, some dev tools, and many more!</p><p><strong>Check out my source code in my GitHub repo: </strong><a href="https://github.com/ilanKushnir/kws-cluster"><strong>KWS-Cluster</strong></a><strong> and ⭐️ it!</strong></p><ul><li>Some Buzzwords: Kubernetes, K3S, Gitops, Flux, Grafana, Prometheus, Plex, Longhorn</li><li>Don’t miss the useful links I’ve shared at the end of the post.</li><li>Continue reading if you want to get instructions on how to build one of your own!</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*425okj0IpzccY3RSEIXJeg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p9u7esYEWnC1MNNUSdz1Rw.png" /></figure><h3>The beginning</h3><p>It all started at a Node.JS conference in a lecture about Raspberry PIs (RPIs) and node-red by Ran Bar-Zyk, the idea of developing some small side projects with card-sized computers blew my mind. After a few weeks, I’ve stumbled upon some very cool <a href="https://github.com/dFurman/wolt-checker-bot">Wolt Restaurants Telegram Bot</a> repository that the developer was running on a small RPI home server. These two events led me to the idea of doing a side project that will include RPIs, be accessible, and give me a very cool basic home infrastructure for implementing my own ideas and practicing the tech stack that I’m using on a daily basis in my workplace as a backend engineer.</p><p>I thought to myself, what if I combine a few RPI nodes into one Kubernetes cluster that will be maintained by me… it will get me a bit into the ops world and will give me the possibility to create stuff and use it for my own purposes. I immediately started looking for some inspiration, and fortunately, there were many people who have already done that and created their own clusters.</p><p>First, I’ve bought two RPIs and started playing around by setting up a K3S cluster with two nodes. K3S is a lightweight Kubernetes distribution that suits best ARM architecture nodes such as Raspberry PIs, it was very natural and easy to setup and to start playing around with my basic cluster. In the next part, I will describe my learning process and how you can build your own Homelab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T3Kx4nfSH7lDAhwBzZevrw.png" /></figure><h3>What is a Homelab???</h3><p>Homelab is a home infra that you are building and maintaining as a hobby, it can serve you for various purposes like deploying your own stuff and also some ready-made self-hosted very useful apps. Here are some of the services that I’m using on my cluster:</p><ul><li><strong>Navidrome</strong> — self-hosted Spotify-like service that streams my own music collection wherever I go</li><li><strong>Media stack</strong> — Plex, Sonarr, Radarr, and more…</li><li><strong>Monitoring stack</strong> — Grafana, Prometheus &amp; AlertManager</li><li><strong>Longhorn</strong> — Distributed block storage for persistent storage</li><li><strong>NAS server</strong> — Network Attached Server (mainly for media and large data files)</li><li><strong>Dev tools</strong> — self-hosted VS Code, echo-server, and there is even a possibility to deploy an echo system with the basic AWS stack!</li><li>and many many more…</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*02Hnbom5EWaUFId9OeednQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ZZIe9jOosR-Gp3cXCbt_w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YByBL-VS8q6Y9RUWa7qhQg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P-vxjyorK2W_jkcD_KU5Ag.png" /><figcaption>Some of my self-hosted dashboards and services</figcaption></figure><p>In general, homelabs can be very compact, useful, fun, and educative!</p><h3>One tip to success!</h3><p>Don’t be afraid to troubleshoot! — always have a goal in mind and don’t stop debugging until you get to the desired result! you will always get there sooner or later!</p><h3>Where should you start from?</h3><p>First of all, start by getting some RPIs and connecting them to your home network. There is a very cool YT tutorial (by NetworkChuck) for setting up a K3S cluster, just follow it, bake the PIs, setup K3S and you should be OK!<br>Check out the links section at the bottom.</p><p>The next steps in the project were split into two phases.</p><h3>Phase 1 — learn the basics and practice manually</h3><p>In order to use some automation tools in the future, first, you’ll need to learn the basics. Without the basics, you won&#39;t be able to understand what&#39;s going on in the cluster under the hood and you will struggle to troubleshoot issues.</p><p>Here are some steps that I would recommend taking next:</p><ul><li>Setup your first K3S cluster on 2 RPIs (at least)</li><li>Explore Kubernetes’ basic resources (deployments, services, ingress, secrets and etc.)</li><li>Deploy your own docker image in the cluster and test it with the postman</li><li>Buy some PoE HATs for your RPIs and maybe even a cluster case!</li><li>Deploy monitoring stack</li><li>Deploy Longhorn as your storage class</li><li>Deploy your own website! you can use a dockerized static HTML website or even your own self-hosted WordPress instance!</li><li>Deploy Pi-Hole — a network-level ad-blocker! one of the coolest but its a more advanced deployment</li><li>Deploy media stack</li><li>Secure your cluster’s ingresses with SSL certificates and some SSO authentication where needed</li><li>Deploy Heimdall to list all your services</li><li>Deploy Monero crypto-miner! — an 8GB RPI with no GPU can mine you 0.8$ a year! 😂 — it will be just a good story to tell your friends</li></ul><blockquote><em>You can find all the related guides and links at the bottom</em></blockquote><p>Regarding the hardware, you can go in two directions. You can buy RPIs and a cluster case or you can run VMs locally that will do the same job. What I like about my RPI cluster is that it is always up and running, not related at all to my workstation, and allows me to manage my resources in a live environment. It&#39;s just always there, accessible, expandable, and very good-looking!<br>One problem is that RPIs doubled their prices because of the chips shortage, I hope they will get back to their original prices soon.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*_Jcxvq7HNeAdQB3LIUZkfQ.png" /><figcaption>My Home cluster’s hardware — expanded gradually</figcaption></figure><blockquote><em>I’ll update this article in the near future with some recommended shopping list — stay tuned or contact me</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uTKuiLGe-0tGr6NdGjTE5w.png" /><figcaption>Assembling the cluster case</figcaption></figure><h3>Phase 2 — join the community and start building your dream cluster!</h3><p>In this phase, after learning the basics and doing stuff manually I wanted to find some more automated solutions and I got the chance to join the awesome <a href="https://k8s-at-home.com/"><strong>K8S@Home</strong></a> community. I can’t describe enough how supportive, friendly and informative it is!</p><p>First of all check out the <a href="https://github.com/k8s-at-home">Github repo</a>, where you can find:</p><ul><li>An automated Gitops operated awesome cluster template! you just follow the instructions and an awesome cluster is deployed with many cool Gitops tools that you really would love to learn and use!</li><li>Maintained Helm charts of many cool self-hosted services</li><li>A collection of Awesome Gitops clusters that were built by the community members can inspire you a lot and even help you to deploy stuff in your own cluster — many of them are based on the template mentioned above.</li><li>A discord channel with a lot of inspiring topics and also don’t be shy to ask for some help! there will always be someone who would love to help you!</li><li>Don’t miss <a href="https://github.com/onedr0p/home-ops/">this GREAT repo</a> that was created by <strong>onedr0p</strong> - one of the community members that helped me and many others!</li></ul><h3>Keep maintaining and expanding</h3><p>This template and community will make it easier to expand and stabilize your cluster, maintain it, and learn a lot of best practices. Dive into Gitops, deploy the apps you wanna use, expand your hardware, invite some friends to use and test it, and deploy your own developed ideas.<br>The cluster is your world and there are no limits to what you can do with it!</p><h3>Future ideas</h3><ul><li>Another dev/staging cluster built from RPI zeroes 2</li><li>Develop micro-services closed environment</li><li>Improve cluster security</li><li>Deploy home surveillance system</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yKOxtp1-sDFenWJuzTTD1w.png" /></figure><h3>Useful resources, tutorials, and ideas to explore:</h3><h3>Phase 1:</h3><p><strong>Network Chuck’s Youtube guides:</strong></p><ul><li><a href="https://www.youtube.com/watch?v=X9fSMGkjtug">Build your first raspberry pi cluster</a></li><li><a href="https://www.youtube.com/watch?v=gyMpI8csWis">Build an RPI NAS server</a></li><li><a href="https://www.youtube.com/watch?v=hHtGN_JzoP8">Deploy Monero crypto-miner </a>— just for the practice</li></ul><p><strong>DOCs to explore:</strong></p><ul><li><a href="https://kubernetes.io/docs/home/">Kubernetes</a></li><li><a href="https://rancher.com/docs/k3s/latest/en/">K3S</a> — the lightweight K8S</li><li><a href="https://doc.traefik.io/traefik/">Traefik</a> (proxy / ingress controller)</li><li><a href="https://longhorn.io/">Longhorn</a></li></ul><p><strong>Cool automated deployments:</strong></p><ul><li><a href="https://github.com/carlosedp/cluster-monitoring">Monitoring stack</a></li><li><a href="https://github.com/kubealex/k8s-mediaserver-operator">Media server stack</a></li><li>Many more docker images on <a href="https://hub.docker.com/u/linuxserver">Linuxserver</a></li></ul><h3>Phase 2:</h3><ul><li>Dive into the <a href="https://discord.gg/Yv2gzFy">K8S@Home</a> community</li><li>Expolre their <a href="https://github.com/k8s-at-home">Gihub repos</a></li><li>Deploy the apps you like from this <a href="https://github.com/awesome-selfhosted/awesome-selfhosted">huge self-hosted apps collections</a>!</li><li>Deploy some of the <a href="https://github.com/k8s-at-home/charts/tree/master/charts">K8S@Home charts</a></li></ul><h3>🙏 Gratitude</h3><p>Thanks to the great supportive community at <a href="https://github.com/k8s-at-home/">Kubernetes @Home</a>. <br>Most of the support and inspiration for my cluster came from <a href="https://github.com/onedr0p/"><strong>onedr0p</strong></a> and from his <a href="https://github.com/onedr0p/home-ops"><strong>super cluster</strong></a>.</p><h3><strong>Keep expanding, deploying, maintaining, and developing your cluster with your own ideas!</strong></h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7e079cf103b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>