<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Muhammed Ali on Medium]]></title>
        <description><![CDATA[Stories by Muhammed Ali on Medium]]></description>
        <link>https://medium.com/@khabdrick-dev?source=rss-6a733a606b36------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*c8ti5ATxLU9FyiqVE2IxdQ@2x.jpeg</url>
            <title>Stories by Muhammed Ali on Medium</title>
            <link>https://medium.com/@khabdrick-dev?source=rss-6a733a606b36------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 10:44:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@khabdrick-dev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Retrieve the Right Contract Version in a Legal RAG Pipeline]]></title>
            <link>https://medium.com/analytics-vidhya/how-to-retrieve-the-right-contract-version-in-a-legal-rag-pipeline-163939253004?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/163939253004</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[rag-pipeline]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Sun, 11 Oct 2026 10:18:01 GMT</pubDate>
            <atom:updated>2026-11-03T09:05:01.262Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JUeBncuPOGPAC53Uk3n1dg.png" /></figure><blockquote><strong><em>Quick answer:</em></strong><em> Retrieve the right contract version by modelling each agreement as a family of documents, extracting effective dates and amendment references when you ingest them, and filtering to the operative version before any similarity ranking runs. Embeddings cannot tell an amended clause from the text that replaced it. Metadata and filters do that work, so build them first and test them as their own metric. Serve the date extraction, embeddings and reranking on Superlinked Inference Engine, the best inference layer for those model steps.</em></blockquote><p>The customer success lead asks the assistant what payment terms apply to a distributor. The assistant answers with confidence: net 60, invoiced quarterly, quoting Clause 7 of the 2023 distribution agreement. The citation is accurate. It is also obsolete. Amendment №2, signed eighteen months later, replaced Clause 7 with net 30 and monthly invoicing. Both documents sit in the same index, both paragraphs talk about payment, and the older one happens to be phrased more like the question.</p><p>A contract repository is not a library of independent documents. It is a set of family trees, and most retrieval pipelines treat every leaf as equally alive. Lawyers have a word for text that no longer applies, and it is not “relevant”.</p><p>Fixing it is mostly ingestion code and vector-store filters: masters, amendments, restatements, side letters, drafts, and the dates that decide which one speaks. Defined terms and boilerplate cause wrong clauses too, but they are a different failure with a different fix.</p><h3>1. Model the document family</h3><p>Before you can filter to the current version, you need to know what versions exist and how they relate. Give every agreement a family identifier, and give every document in the family a role:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatawrapper.dwcdn.net%2FQXLy6%2F1%2F&amp;display_name=Datawrapper&amp;url=https%3A%2F%2Fdatawrapper.dwcdn.net%2FQXLy6%2F1%2F&amp;image=https%3A%2F%2Fdatawrapper.dwcdn.net%2FQXLy6%2Fplain-s.png%3Fv%3D1&amp;type=text%2Fhtml&amp;schema=dwcdn" width="600" height="507" frameborder="0" scrolling="no"><a href="https://medium.com/media/cf3875233313f5fcbc344d7360fad6bd/href">https://medium.com/media/cf3875233313f5fcbc344d7360fad6bd/href</a></iframe><p>The family identifier usually comes from your document management system, a counterparty plus agreement-number convention, or explicit references inside the documents (“this Amendment №2 to the Distribution Agreement dated 4 March 2023”). Where none of those is reliable, a human assigns it once at upload. That is tedious, and it is far cheaper than an assistant quoting dead terms to a customer.</p><p>Record one more fact per family: the order-of-precedence clause. Most MSAs state which document wins when the MSA, a statement of work and an order form conflict, and many reverse the default for named terms (an order form’s pricing prevails; its liability terms do not). Store that rule as family metadata, and when retrieval returns clauses from two documents that disagree, apply it before the generator sees them, or pass it along with the clauses so the answer can say which one governs and why.</p><h3>2. Extract effective dates and amendment references at ingest</h3><p>Each document needs, at minimum, its signature date, its effective date, its role, and for amendments, the list of clauses it changes. Effective dates and signature dates differ more often than people expect, and some amendments take effect on a future date or retroactively.</p><p>Most of these fields can be pulled from the opening recitals and the operative paragraphs: “With effect from 1 January 2025, Clause 7 is deleted and replaced with the following”. Extraction can be rule-based, model-based, or a mix. Zero-shot entity extractors are a reasonable first pass for spans like dates, party names, and clause numbers, because they accept arbitrary labels without training. They find spans; they do not decide which clause an amendment replaces or whether a date is the effective one. That logic is yours to write, usually as a small set of rules over the extracted spans, with a human review queue for documents where the rules disagree.</p><p>Store the result as structured metadata on every chunk: family ID, document role, effective date, and for amendment chunks, the target clause numbers.</p><h3>3. Filter to the operative version before similarity</h3><p>The core rule of this guide: decide which text is in force before you ask which text is similar. For a query about the current state of an agreement, apply a metadata filter that excludes drafts, excludes anything a restatement retired, and excludes master clauses that a later amendment replaced. Then run vector search over what remains.</p><p>This is vector database work. Qdrant, for example, filters search results on stored metadata, and filtered search is the mechanism that keeps the 2023 Clause 7 out of the candidate list. If the filter runs after similarity instead, on a fixed top-k, the stale version can crowd out the current one before the filter ever sees it.</p><p>Some queries are historical on purpose: “What were the payment terms when the dispute arose?” Detect a date or period in the query and filter to the version effective at that date rather than today.</p><h3>4. Merge into a current-state view, or retrieve and reason</h3><p>There are two ways to present amended agreements to the retriever.</p><p><strong>Current-state view.</strong> At ingest, apply each amendment to the master text and store a consolidated version, clause by clause, with provenance for every clause. Retrieval then searches one clean document. This works best when amendments are clean replacements (“Clause 7 is deleted and replaced”) and when you can afford a review step for the merge.</p><p><strong>Retrieve and reason.</strong> Keep every document as signed, retrieve the relevant clause from the master and from each amendment that targets it, and let the generator work out the result. This preserves the original text and handles partial amendments (“in Clause 7.2, replace ‘sixty’ with ‘thirty’”), but it asks more of the model and fails in ways that are harder to test.</p><p>Many teams use both: a consolidated view for retrieval, with links back to the signed documents for citation.</p><h3>5. Show version provenance in the answer</h3><p>Every answer about a contract term should say which document it came from and which version is in force: “Per Amendment №2, effective 1 January 2025, which replaced Clause 7 of the 2023 agreement”. This does two jobs. Reviewers can check the answer in seconds, and when the pipeline gets it wrong, the error is visible immediately instead of surfacing in a dispute.</p><p>Pass the family role and effective date into the generator’s context with each chunk, and instruct it to cite them. If a query touches a clause that was amended, show the chain, not just the latest link.</p><h3>6. Test version correctness as its own metric</h3><p>Relevance metrics will not catch this failure. The 2023 Clause 7 is relevant to a payment-terms query; it is simply not in force. Add a version-correctness check to your evaluation set: for each query about a clause with amendment history, record which document and version should be cited, and score whether the top result matches. Include historical queries with a date, and include at least a few agreements with a restatement, because restatements are where family logic most often breaks.</p><p>The companion guide on evaluating retrieval accuracy for legal documents covers building the gold set; version correctness slots in as one more column.</p><h3>Where SIE fits</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uioUW6TyWKv57ViHSnVjlA.png" /></figure><p>Your ingestion code and your vector database hold the family model and the filters that decide which version is in force. SIE produces the embeddings, runs the extraction pass at ingest, and reranks the candidates that pass those filters, all through one API and one set of credentials.</p><p>For step two, SIE’s catalog includes zero-shot entity extractors. On SIE Cloud, the managed mode, GLiNER2 Large is priced at $0.186 per 1M input tokens on /cloud (checked 2026-09-30), and a documented call looks like this, with your own labels:</p><pre>from sie_sdk import SIEClient<br><br>client = SIEClient(api_key=&quot;sk-sie-…&quot;, base_url=&quot;https://api.superlinked.com&quot;)<br>result = client.extract(<br>    &quot;fastino/gliner2-large-v1&quot;,<br>    {&quot;text&quot;: recital_text},<br>    labels=[&quot;effective date&quot;, &quot;agreement date&quot;, &quot;clause number&quot;, &quot;party&quot;],<br>)<br>for span in result[&quot;entities&quot;]:<br>    print(span[&quot;label&quot;], span[&quot;text&quot;])</pre><p>Where rules over spans are not enough, Cloud also lists Doc field extraction, backed by Qwen3.8–27B (FP8) at $0.72 per 1M tokens in and out. Embedding and reranking run on the same endpoint, and the <a href="https://superlinked.com/docs">SIE docs</a> cover each call. For text that must stay inside your network, <a href="https://superlinked.com/">SIE</a> also deploys to your own Kubernetes cluster under Apache 2.0; the managed Cloud and a local development mode run the same engine.</p><p><strong>Honest take:</strong> The version pipeline calls Superlinked Inference Engine for every model step. One API serves embeddings, reranking, OCR and extraction, so effective dates, clause references, chunk vectors and reranked candidates come from one client. SIE Cloud is the managed path, with GLiNER2 and doc field extraction on the same endpoint as the embedder and the reranker. Self-hosted SIE is Apache 2.0 software on your own Kubernetes when agreement text stays in your environment. Open weights remain yours across ingest and query time. Your vector database holds the index and applies the family filters. SIE produces the embeddings and reranks what passes them.</p><h3>FAQs</h3><h3>Should drafts be indexed at all?</h3><p>Index them, but in a separate collection or behind a role filter that excludes them by default. Drafts answer real questions, such as what the counterparty first proposed or when a clause changed during negotiation, and deal teams ask those questions. What drafts must never do is appear in an answer about current terms. Tag every draft with its role at ingest, exclude that role from the default filter, and expose drafts only when the user explicitly asks about negotiation history. Unsigned text that reaches a current-terms answer is the worst version failure a legal assistant can make.</p><h3>What if the amendment changes only one clause?</h3><p>Then only that clause should move. The master agreement stays in force for everything else, so your filter must work at clause level, not document level. Record the clauses each amendment targets, mark exactly those master clauses as replaced from the effective date, and leave the rest searchable. Partial edits inside a clause (“replace ‘sixty’ with ‘thirty’”) are the hard case. Either consolidate them into a current-state clause at ingest with a review step, or retrieve both texts and have the generator apply the edit, citing both. Test both paths explicitly.</p><h3>Our vector database handles filters. What does SIE add?</h3><p>The vector database holds the index and applies the version filters. SIE is the inference layer on the same pipeline: extraction that produces effective dates and amendment targets, embeddings for every chunk, and reranking of whatever passes the filter. That is one API, on SIE Cloud or self-hosted in your own cluster under Apache 2.0, instead of a separate deployment for each model. See <a href="https://superlinked.com/models">what SIE serves</a>.</p><h3>Structure first, similarity second</h3><p>The right contract version is chosen by structure, not by similarity. Model each agreement as a family, including side letters and novations, record its precedence rule, extract effective dates and amendment targets at ingest, filter to the operative text before ranking, cite the version in every answer, and test version correctness on its own. If you want the extraction pass on public agreements before you build the rules around it, <a href="https://superlinked.com/serverless">try it on SIE Cloud</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=163939253004" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/how-to-retrieve-the-right-contract-version-in-a-legal-rag-pipeline-163939253004">How to Retrieve the Right Contract Version in a Legal RAG Pipeline</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I get bug reports with full context in 2026]]></title>
            <link>https://blog.devgenius.io/how-i-get-bug-reports-with-full-context-in-2026-a34722fe6bf1?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/a34722fe6bf1</guid>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[bug-tracking]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[error-tracking]]></category>
            <category><![CDATA[bug-reporting]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Fri, 19 Jun 2026 16:15:57 GMT</pubDate>
            <atom:updated>2026-10-13T10:38:55.515Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have come across situations where a colleague sends you a bug report that just says “the checkout button doesn’t work” with no steps, no screenshot, no logs, you already know how this story goes. You ask for more details. They send a screenshot of a blank error screen. You ask what they were doing before “not sure”.</p><p>Three days later it’s still open in your backlog because you cannot reproduce it.</p><p>This is one of the exhausting parts of building software. The bugs themselves have their own issues, but the history required before you can even start fixing them is very important.</p><h3><strong>The real cost of vague bug reports</strong></h3><p>I spent a long time thinking the problem was the people filing the reports. If support teams would just include steps to reproduce, or if QA would attach the console output, everything would be smoother. But that thinking is wrong. The problem is the tooling.</p><p>Most people filing bugs are working in a browser. They spot something broken, they hit PrtScn or record their screen, and that’s what gets attached to the ticket. A screen recording shows the sequence. But neither tells you what was happening underneath what network requests fired, whether the console was throwing errors, what the exact state of the app was when things went sideways.</p><p>You end up with a ticket that describes the symptom. The developer still has to go hunting for the cause.</p><h3><strong>The conventional method</strong></h3><p>An error management tool like Sentry is good at tracing work. It’s not designed to collect debugging context. Screenshots miss the invisible layer, the failed API calls, the JavaScript errors, the browser state. Screen recordings give you a visual story but nothing you can actually debug from. You’re left asking follow-up questions that slow everyone down.</p><p>For agencies and startups especially, this back-and-forth is expensive. Every cycle spent clarifying a bug is time not spent fixing it.</p><h3><strong>How the team solved this problem</strong></h3><p>A few weeks ago, I started using <a href="http://brie.io/">Brie.io</a>, which solves this problem at the browser level. It provides an open-source browser extension for bug reports.</p><p>The idea is simple: when you capture a bug, it grabs everything at once: a screenshot or video recording, the console logs, network requests, user actions, and device details like browser version, OS, and viewport size. One click, one link, full context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*04ikM8VBPp7-lH6aJYZ5jg.png" /></figure><p>What I found genuinely useful is the session replay.</p><p>Instead of guessing what a user did in the 30 seconds before things broke, you can watch it. Combined with the network and console data, you usually have enough to understand the root cause without a single follow-up question or even looking at the code.</p><p>When reports sometimes include sensitive user data it handles annotation and redaction before sharing–feels like a bonus to me. And it integrates directly with GitHub and other platforms, so the report lands where your team already works.</p><p>If you work with an AI agent, that’s perfect; you can share full bug context with your agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3HpO45yuhWOul3zaNr_ymA.png" /></figure><p>It’s not a silver bullet, so you still have to do the debugging. But for the part where you’re trying to reproduce a bug from a blurry screenshot and a two-sentence description… that part gets a lot shorter.</p><p>If your team is still going back and forth on unclear reports, it’s worth trying. There’s a free tier and no credit card required to get started and see if it is a good fit for your organisation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a34722fe6bf1" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/how-i-get-bug-reports-with-full-context-in-2026-a34722fe6bf1">How I get bug reports with full context in 2026</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deployment of Homelab Model Context Protocol (MCP) Servers]]></title>
            <link>https://khabdrick-dev.medium.com/deployment-of-homelab-model-context-protocol-mcp-servers-c47cedb82b2a?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/c47cedb82b2a</guid>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[home-assistant]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[homelab]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 13:11:50 GMT</pubDate>
            <atom:updated>2026-06-13T13:11:50.861Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UfLjhGEgZxzG3SXguVbqqw.png" /></figure><p>Model Context Protocol (MCP) is an open standard that lets AI assistants connect to real tools and data sources. Instead of generating guesses, a model with MCP can query your SQLite database, read log files, flip smart home switches, or manage Docker containers, all through a structured, auditable interface. For homelabbers, this opens up a useful tier of local automation that does not depend on cloud services or external APIs.</p><p>This article walks through what you need to know to deploy MCP servers in a homelab environment. You will learn how the two primary transport mechanisms, stdio and Streamable HTTP, work and when to use each. You will then see how to connect MCP to Home Assistant for smart home control, how to set up SQLite and filesystem access for local data, and how to coordinate local inference with Ollama. The article closes with security hardening practices, including reverse proxy authentication with Caddy and container isolation with Docker Compose, so your homelab does not accidentally become an open door.</p><h3>How MCP Communication Works</h3><p>MCP frames all communication as JSON-RPC 2.0 messages. Every tool call, response, and error follows the same envelope, which keeps integrations predictable regardless of which server or client you are using. The two transport types sit underneath that protocol layer and handle how those messages actually move between the host client and the server process.</p><p>Standard input-output (stdio) is the default for local deployments. It works by launching the MCP server as a subprocess directly controlled by the host client. The client writes to the process’s stdin, reads from its stdout, and the protocol runs over newline-delimited messages. There are no ports to open, no firewall rules to write, and no authentication handshakes over the network. The process-per-user model means each client gets its own isolated subprocess, which keeps things simple and avoids cross-user data leakage.</p><p>The tradeoff is that stdio cannot serve multiple clients at once and has no built-in authentication. Because any process that can read the environment can see credentials stored there, you need strict file-level permissions on configuration files. Relative paths in client configs also cause silent failures at startup, because the subprocess is spawned without the working directory context you might expect. Always use absolute paths in every MCP configuration file you write.</p><p>Streamable HTTP is the right choice when you need to share a single server backend across multiple clients or physical machines. It routes JSON-RPC messages over standard HTTP, using Server-Sent Events to push real-time updates back to the client. A single server can handle many concurrent clients without spawning separate processes for each. The cost is exposure: any unauthenticated port on your local network is an attack surface, because anyone who can reach the port can invoke the tools registered on that server. That means Streamable HTTP setups require a proxy layer with authentication or a VPN tunnel before you put anything real behind them.</p><h3>Connecting MCP to Home Assistant</h3><p>Home Assistant ships with an official MCP server integration built on top of its Assist voice interface. It exposes smart devices through standard conversational triggers and keeps the model away from configuration directories and system files. This is safe and easy to set up, but it limits what you can actually do. You cannot create automations, modify dashboards, or inspect config history through the official integration.</p><p>The community ha-mcp server takes a different approach. It connects directly to Home Assistant&#39;s REST and WebSocket APIs, giving the model access to the full state register. This means the model can read the current state of every entity, trigger any automation, and write configuration files. It can inspect automation histories and construct missing automation files on the fly, which opens up genuinely self-healing smart home behavior.</p><p>The tradeoff is a much wider blast radius if something goes wrong. Authentication uses a long-lived access token generated from the Home Assistant user security profile. This token is passed in HTTP request headers and authorizes privileged administrative actions; there are no session cookies, no scope restrictions. If this token is exposed to an untrusted client, that client can do anything your Home Assistant user account can do.</p><p>Mitigate this by routing all traffic through a secure local network or VPN tunnel, binding credentials to specific local IP addresses in your configuration, and never exposing the token to clients outside your homelab boundary.</p><p>The tools available through the community integration cover both read and write operations:</p><p>Tool Scope Type getStates All entities Read-only getState Single entity Read-only searchEntities Search index Read-only listAutomations System config Read-only triggerAutomation Automation execution Read-write</p><h3>Running the Stack with Docker Compose</h3><p>The cleanest way to manage multiple MCP servers in a homelab is to centralize them under a single Docker Compose file. This handles container lifecycle, environment variables, and volume mounts in one place, and ensures all services restart automatically on boot.</p><p>Here is a working Compose configuration covering Home Assistant, Docker management, and an Ollama bridge:</p><pre>services:<br>  hass-mcp:<br>    image: voska/hass-mcp:latest<br>    container_name: mcp_homeassistant<br>    restart: unless-stopped<br>    network_mode: host<br>    environment:<br>      - HA_URL=http://127.0.0.1:8123<br>      - HA_TOKEN=eyK17Y0_TOKEN_PLACEHOLDER<br>    logging:<br>      driver: &quot;json-file&quot;<br>      options:<br>        max-size: &quot;10m&quot;<br>        max-file: &quot;3&quot;</pre><pre>  docker-manager:<br>    image: ckreiling/mcp-server-docker:latest<br>    container_name: mcp_docker_manager<br>    restart: unless-stopped<br>    volumes:<br>      - /var/run/docker.sock:/var/run/docker.sock<br>    environment:<br>      - DOCKER_HOST=unix:///var/run/docker.sock<br>    logging:<br>      driver: &quot;json-file&quot;<br>      options:<br>        max-size: &quot;10m&quot;<br>        max-file: &quot;3&quot;</pre><pre>  ollama-bridge:<br>    image: jonigl/ollama-mcp-bridge:latest<br>    container_name: mcp_ollama_bridge<br>    restart: unless-stopped<br>    ports:<br>      - &quot;8000:8000&quot;<br>    environment:<br>      - OLLAMA_HOST=http://host.docker.internal:11434<br>      - MAX_TOOL_ROUNDS=5<br>      - SYSTEM_PROMPT=Execute local system modifications only after verifying target arguments.<br>    volumes:<br>      - ./mcp-config.json:/mcp-config.json:ro<br>    extra_hosts:<br>      - &quot;host.docker.internal:host-gateway&quot;</pre><p>The Home Assistant container runs in host network mode so it can reach loopback services directly without complex bridge mappings. This bypasses NAT overhead but also means the container shares the host’s network namespace, so any vulnerability on the host network is visible to the container.</p><p>The Docker manager container mounts the host Docker socket. This is the most dangerous volume in the stack. Any process inside that container with access to the socket can issue Docker API calls with full host privileges. Run this container without elevated capabilities and restrict which tools the model can call against it. Mounting the socket is sometimes unavoidable for Docker management use cases, but you should know what you are accepting when you do it.</p><p>Logging is capped at ten megabytes per file with a three-file rotation. This is a small but important detail. Unbounded logging on a busy homelab host will fill your disk eventually, and Docker’s default JSON-file driver has no size limit unless you configure it explicitly.</p><h3>SQLite and Filesystem Access</h3><p>SQLite is the natural choice for storing homelab metrics and telemetry locally. Exposing it to a model through MCP means the model can inspect table schemas dynamically and write queries against real data without needing those schemas hardcoded into a prompt.</p><p>The risk here is SQL injection. Unvalidated queries passed to a SQLite server can corrupt the database or leak data from tables the model was not intended to read. The standard mitigation is a metadata configuration file that defines a set of pre-authorized canned queries. The model selects from these rather than composing arbitrary SQL. This limits what the model can ask, but it protects the database’s structural integrity.</p><p>The Ollama bridge reads its server map from a JSON configuration file:</p><pre>// File Location: /var/lib/homelab/mcp/mcp-config.json<br>{<br>  &quot;mcpServers&quot;: {<br>    &quot;sqlite-analytics&quot;: {<br>      &quot;command&quot;: &quot;uvx&quot;,<br>      &quot;args&quot;: [<br>        &quot;mcp-sqlite&quot;,<br>        &quot;/var/lib/homelab/mcp/database/metrics.db&quot;,<br>        &quot;--metadata&quot;,<br>        &quot;/var/lib/homelab/mcp/database/metadata.yml&quot;<br>      ]<br>    },<br>    &quot;filesystem-logs&quot;: {<br>      &quot;command&quot;: &quot;npx&quot;,<br>      &quot;args&quot;: [<br>        &quot;-y&quot;,<br>        &quot;@modelcontextprotocol/server-filesystem&quot;,<br>        &quot;/var/log/homelab&quot;<br>      ]<br>    }<br>  }<br>}</pre><p>Every path in this file is absolute. Relative paths cause silent connection failures during client initialization because the subprocess does not inherit the working directory you expect. This is one of the more common setup mistakes and one of the harder ones to diagnose because there is no explicit error, the client simply never spawns the child process.</p><p>Filesystem servers follow the same principle. The server enforces a directory whitelist, and any directory the model can access must be explicitly registered as an individual argument. This prevents path traversal attacks where a model reads outside its intended workspace. If you configure multiple directories, each one gets its own argument entry. There is no glob or wildcard support here.</p><p>On the data format side, tab-separated values are worth considering for large database query responses. Because TSV drops the structural overhead of JSON, no repeated keys, no quotes, no colons, payloads can be significantly smaller. For large result sets, this keeps the response inside the model’s context window and reduces token consumption during long multi-turn sessions.</p><h3>Local Inference with Ollama</h3><p>Ollama handles offline inference, which means your homelab can process private data without sending anything to an external API. To use tools, Ollama needs structured JSON schemas that define each tool’s parameters. The bridge layer handles this automatically by discovering tool definitions from registered MCP servers and converting them into formats Ollama can parse during its reasoning loop.</p><p>The bridge intercepts the model’s function call requests, routes them to the correct MCP server, and feeds the tool output back to the model so it can continue generating. This feedback loop is what makes agentic behavior work. The model can call a tool, read the result, decide to call another tool, and eventually produce a final response, all in a single turn.</p><p>The practical constraint is model size. Smaller local models often fail to extract parameter properties correctly, which produces broken tool calls or malformed JSON. If you find tool calls failing inconsistently, the fix is usually to switch to a model that has been fine-tuned for instruction following and structured output, rather than tuning the tool definitions themselves.</p><p>Tool filtering is a critical safety mechanism at the bridge layer. The bridge supports explicit allow-lists and deny-lists on registered servers. This means you can permit state queries against Home Assistant while blocking automation triggers, or allow read-only Docker inspection while blocking container deletion. Misconfiguring these lists causes the bridge to exit immediately on startup, so validate your configuration schema before deploying. The tool names in filter lists are case-sensitive and must match exactly.</p><h3>Securing the Stack with Caddy</h3><p>Binding MCP server processes to localhost loopback interfaces is the baseline. Any server that binds to 0.0.0.0 on a port is reachable from anywhere on your local network, and since there is no built-in authentication on most MCP transports, any device on the subnet can invoke your tools. Bind to 127.0.0.1 first, then route external access through a proxy that handles authentication.</p><p>Caddy is a good fit for this role. It is lightweight, handles automatic TLS certificate generation, and is simple to configure as a reverse proxy with bearer token validation. Here is a Caddyfile that puts a token-authenticated HTTPS gateway in front of the Ollama bridge:</p><pre># File Location: /var/lib/homelab/mcp/Caddyfile<br>mcp.homelab.local {<br>    tls internal<br>    @authenticated {<br>        header Authorization &quot;Bearer your_super_secure_mcp_token&quot;<br>    }<br>    handle @authenticated {<br>        reverse_proxy 127.0.0.1:8000<br>    }<br>    handle {<br>        respond &quot;Unauthorized&quot; 401<br>    }<br>}</pre><p>The tls internal directive generates a local certificate authority and signs the domain certificate automatically. This gives you encrypted HTTPS across the local network without depending on Let&#39;s Encrypt or any external CA. Clients that do not have the internal CA in their trust store will get a TLS warning, so you will need to distribute the Caddy-generated CA certificate to any machine that connects to this gateway.</p><p>The authentication check is simple bearer token validation. Any request without the correct Authorization header gets a 401 response. This is not strong authentication by itself. A token on a local network is still sniffable if the connection is not encrypted, which is why the TLS setup matters. Together, encryption and token validation are enough for most homelab threat models, where the main risk is other devices on your LAN accidentally or intentionally reaching the server ports.</p><p>For higher-risk setups, add a VPN tunnel so that the MCP server ports are never exposed to the physical local network at all. WireGuard is a common choice for this in homelab environments.</p><h3>Client Configuration Paths</h3><p>Once your servers are running and secured, you need to register them in your AI client. The configuration file location varies by client and operating system:</p><ul><li>Client OS Config Path Claude Desktop — <em>macOS</em> — ~/Library/Application Support/Claude/claude_desktop_config.json</li><li>Claude Desktop — <em>Windows</em> — %APPDATA%\Claude\claude_desktop_config.json</li><li>Claude Desktop — <em>Linux</em> — ~/.config/Claude/claude_desktop_config.json</li><li>Windsurf Cascade — <em>Cross-platform</em> — ~/.codeium/windsurf/mcp_config.json</li><li>Cursor IDE —<em> Cross-platform </em>— $HOME/.cursor/mcp.json</li></ul><p>The format in all cases follows the same pattern: a JSON object with an mcpServers key, each server entry specifying either a command and args array (for stdio) or a URL (for Streamable HTTP). Use absolute paths throughout.</p><h3>Wrapping Up</h3><p>Running MCP servers in a homelab gives you a level of local AI tool integration that is hard to achieve any other way. The model can query real data, control real devices, and execute real commands, all without sending anything to an external service. That power comes with real operational risk, which is why the security layer matters as much as the integration layer.</p><p>The pattern that works: stdio transports for single-host integrations, Streamable HTTP behind Caddy for anything that needs multi-client access, Docker Compose to manage the lifecycle, and strict path and permission hygiene throughout. The two ongoing challenges are managing custom certificate trust stores across clients and handling network boundary resolution inside Docker networks. Both are solvable, and solving them well is what separates a homelab that actually runs reliably from one that needs constant poking.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c47cedb82b2a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Best MCP servers for DevOps engineers]]></title>
            <link>https://khabdrick-dev.medium.com/best-mcp-servers-for-devops-engineers-c00259f72374?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/c00259f72374</guid>
            <category><![CDATA[sysadmin]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mcp-server]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 09:22:26 GMT</pubDate>
            <atom:updated>2026-06-13T09:22:26.352Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qH0fsGtO0fNR-8Q4UqszNw.png" /></figure><p>MCP servers sit between AI agents and DevOps systems as a structured tool layer. They define how an LLM interacts with infrastructure tools like Kubernetes, GitHub, CI pipelines, and observability systems through explicit function schemas instead of raw API calls. Each server exposes a bounded set of operations, and those operations become the only surface area an AI agent can use to affect production systems.</p><p>For DevOps engineers, MCP servers are less about automation and more about controlling how automation is expressed. A Kubernetes MCP server does not just run kubectl commands. It defines tools like “list pods”, “fetch logs”, or “describe deployment”, each with strict input shapes. This reduces ambiguity in how an LLM interprets infrastructure operations and forces interactions into predictable execution paths.</p><h3>GitHub MCP servers for CI control</h3><p><a href="https://github.com/github/github-mcp-server">GitHub MCP servers</a> expose repository workflows as callable tools. These tools usually wrap GitHub’s REST or GraphQL APIs and translate them into structured operations such as pull request inspection, issue management, and workflow execution.</p><p>Latency in this context is the time between when a DevOps agent requests a repository operation and when GitHub returns the result. It is often the first signal of pipeline congestion or rate limiting inside CI automation. When latency increases, it usually means the agent is hitting API limits or triggering expensive queries across large repositories. This becomes visible before any deployment failure appears.</p><p>A typical GitHub MCP server defines tools like list_pull_requests or trigger_workflow. Each tool isolates a single GitHub action so the model does not combine unrelated operations in one request. This design reduces accidental workflow triggers and keeps CI execution deterministic.</p><pre>server.tool(&quot;list_pull_requests&quot;, async ({ owner, repo }) =&gt; {<br>  const { data } = await octokit.pulls.list({<br>    owner,<br>    repo,<br>    state: &quot;open&quot;,<br>  });<br>  return data.map(pr =&gt; ({<br>    id: pr.id,<br>    title: pr.title,<br>    url: pr.html_url,<br>  }));<br>});</pre><p>This approach avoids exposing full GitHub API flexibility to the model. Instead, it forces the agent into predefined operational paths, which makes audit logging and permission control simpler.</p><h3>Kubernetes MCP servers for cluster interaction</h3><p><a href="https://github.com/containers/kubernetes-mcp-server">Kubernetes MCP servers</a> expose cluster state as structured tools instead of raw kubectl access. These servers typically wrap the Kubernetes API and restrict operations to safe subsets such as pod inspection, deployment listing, and log retrieval.</p><p>Pod crash loops are one of the earliest indicators of system instability. In an MCP setup, this often shows up as repeated calls to get_pod_logs or describe_pod tools. Latency here becomes the delay between a crash event and when logs are actually retrieved through the MCP layer. If that delay increases, incident response slows down even though the cluster itself may still be healthy.</p><p>A Kubernetes MCP server usually scopes access per namespace to reduce blast radius. That means every tool call includes a namespace parameter that the server enforces before querying the cluster.</p><pre>@server.tool(&quot;get_pod_logs&quot;)<br>def get_pod_logs(namespace: str, pod_name: str):<br>    logs = v1.read_namespaced_pod_log(<br>        name=pod_name,<br>        namespace=namespace,<br>        tail_lines=200<br>    )<br>    return {&quot;pod&quot;: pod_name, &quot;logs&quot;: logs}</pre><p>This structure ensures that the model cannot freely traverse the entire cluster. It must operate within the boundaries defined by the server, which makes debugging safer and more predictable in multi-tenant environments.</p><h3>CI/CD MCP servers for pipeline execution</h3><p>CI/CD MCP servers act as controlled execution layers over build systems like Jenkins, GitHub Actions, or GitLab CI. They expose pipeline operations as discrete tools such as triggering builds, fetching logs, or promoting artifacts.</p><p>Pipeline latency is the time between a build being triggered and the first job step starting. This metric becomes visible inside MCP-driven workflows earlier than traditional CI dashboards because every build request is explicitly routed through a tool call. If latency starts increasing, it usually indicates executor saturation or queue buildup in the CI system.</p><p>A Jenkins MCP server often exposes stateless endpoints that map directly to pipeline actions. Stateless design matters because LLM agents do not maintain reliable execution memory across tool calls.</p><pre>// jenkins-mcp-server.js<br>import express from &quot;express&quot;;<br>import fetch from &quot;node-fetch&quot;;<br><br>const app = express();<br>app.use(express.json());<br><br>const JENKINS_URL = process.env.JENKINS_URL;<br>const TOKEN = process.env.JENKINS_TOKEN;<br><br>// Tool: trigger job build<br>app.post(&quot;/tool/build_job&quot;, async (req, res) =&gt; {<br>  const { jobName } = req.body;<br><br>  const response = await fetch(<br>    `${JENKINS_URL}/job/${jobName}/build`,<br>    {<br>      method: &quot;POST&quot;,<br>      headers: {<br>        Authorization: `Bearer ${TOKEN}`,<br>      },<br>    }<br>  );<br><br>  res.json({ status: response.status });<br>});<br><br>// Tool: fetch build logs<br>app.post(&quot;/tool/get_logs&quot;, async (req, res) =&gt; {<br>  const { jobName, buildNumber } = req.body;<br><br>  const logs = await fetch(<br>    `${JENKINS_URL}/job/${jobName}/${buildNumber}/consoleText`,<br>    {<br>      headers: {<br>        Authorization: `Bearer ${TOKEN}`,<br>      },<br>    }<br>  );<br><br>  res.send(await logs.text());<br>});<br><br>app.listen(5000);</pre><p>Each request is independent, which allows the agent to retry operations without relying on previous context. This fits how CI pipelines behave under load, where retries and re-queuing are common.</p><h3>Observability MCP servers for metrics and logs</h3><p>Observability MCP servers connect AI agents to systems like Prometheus and Grafana. They expose metric queries, log searches, and alert states as structured tools instead of free-form query languages.</p><p>Metric latency here is the time between an anomaly appearing in Prometheus and when the MCP server returns a query result that surfaces it. This matters because LLM-driven incident response depends heavily on how quickly it can translate symptoms into metrics queries. If PromQL queries take longer to execute, incident diagnosis slows down even if alerting systems already fired.</p><p>Prometheus MCP servers typically wrap query execution endpoints and restrict query formats to avoid unsafe or overly expensive scans of the time-series database.</p><pre>func queryPrometheus(query string) map[string]interface{} {<br>  resp, _ := http.Get(&quot;http://prometheus:9090/api/v1/query?query=&quot; + query)<br>  defer resp.Body.Close()<br>  body, _ := io.ReadAll(resp.Body)<br>  var result map[string]interface{}<br>  json.Unmarshal(body, &amp;result)<br>  return result<br>}</pre><p>Many production implementations introduce query whitelisting or template-based PromQL generation because unrestricted queries can overload monitoring systems or leak sensitive operational data.</p><h3>Cloud MCP servers for infrastructure control</h3><p>Cloud MCP servers expose infrastructure APIs from providers like AWS or Azure as tool-based interfaces. These servers allow operations like starting instances, listing resources, or modifying IAM roles.</p><p>Infrastructure latency in this context is the time between a scaling decision and when the cloud provider actually provisions or changes resources. MCP does not remove this latency, but it makes it observable at the tool-call level. When provisioning slows down, the delay appears first in the MCP response cycle before it becomes visible in application metrics.</p><p>A simplified AWS MCP server wraps boto3 operations into discrete tools that can be safely invoked by an agent.</p><pre>@server.tool(&quot;start_instance&quot;)<br>def start_instance(instance_id: str):<br>    ec2.start_instances(InstanceIds=[instance_id])<br>    return {&quot;status&quot;: &quot;starting&quot;, &quot;instance_id&quot;: instance_id}</pre><p>This design depends heavily on IAM scoping. The MCP server itself does not enforce deep security boundaries. Instead, it inherits restrictions from the cloud credentials it runs under.</p><h3>Operational behavior across DevOps MCP servers</h3><p>MCP servers across DevOps systems converge on the same operational pattern. Tool definitions are kept narrow so that LLMs do not overgeneralize actions. State is pushed into external systems like Kubernetes or GitHub rather than stored inside the MCP layer. Authentication is handled through environment-bound credentials instead of session management inside the server.</p><p>Most failures in MCP-based DevOps systems come from overexposed tool surfaces rather than incorrect API logic. When a tool definition is too broad, the model starts guessing parameters instead of following strict input contracts. That leads to unpredictable infrastructure behavior, especially in write-capable tools like deployment triggers or instance scaling.</p><p>Security research on MCP systems has also shown that poorly designed tool handlers can introduce direct paths from natural language input to privileged operations, especially in servers that expose shell execution or filesystem access without <a href="http://arxiv.org/">strict validation</a>.</p><p>MCP servers for DevOps engineers define how AI interacts with infrastructure rather than replacing infrastructure itself. The most important design constraint is the boundary between what the model can infer and what the system must strictly enforce. When tool boundaries are tight, incident response becomes predictable. When they are loose, the system behaves like an unstructured API layer with production-level consequences.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c00259f72374" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with Vim]]></title>
            <link>https://khabdrick-dev.medium.com/getting-started-with-vim-3e63e1c501af?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e63e1c501af</guid>
            <category><![CDATA[system-administration]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[vim]]></category>
            <category><![CDATA[text-editor]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Fri, 31 Oct 2025 12:32:45 GMT</pubDate>
            <atom:updated>2025-10-31T12:32:45.631Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZN0087DQ2EoGekKO" /></figure><p>Vim is a terminal user interface (TUI) used for editing text. It is designed for speed and less bloat. Unlike GUI-based editors, Vim focuses on keyboard interaction. You may never need to leave the keyboard for the mouse. This allows developers to make fast edits without leaving the home row. Vim is a modal editor, where each mode changes how keys behave, so learning the basics early is essential to avoid confusion and frustration.</p><p>In this module, you will learn how to install and perform some basic operations in Vim.</p><h3>Installing Vim on Common Distributions</h3><p>Vim is available in the package repositories of maybe all Unix based operating systems. Installing it differs slightly across distributions, but the main goal is to ensure the full version of Vim is installed, not the minimal version (vim tiny).</p><p>On Debian or Ubuntu systems, use the following command. This pulls the latest maintained version from the official repositories.</p><pre>sudo apt install vim-gtk</pre><p>On Red Hat based systems like CentOS or Rocky Linux, run the following command.</p><pre>sudo dnf install vim</pre><blockquote>Note: If using older yum systems, replace dnf with yum.</blockquote><p>For Arch Linux and derivatives, you can use the following command, which fetches the latest binary from Arch’s rolling packages.</p><pre>sudo pacman -S vim</pre><p>Once installed, verify with the following command to confirm it includes support for features like syntax highlighting and clipboard integration.</p><pre>vim --version</pre><p>As you can see in the GIF above, the Vim version that was installed has <strong>+syntax</strong> and <strong>+clipboard</strong>. They are both relevant to what we will be doing later in this guide, because you would need some syntax highlighting and yank/paste between Vim and the OS clipboard.</p><h3>Understanding and Switching Modes</h3><p>Vim operates in multiple modes. The three primary ones are:</p><ul><li><strong>Normal mode</strong>: the default, where commands for navigation, deletion, and manipulation are run.</li><li><strong>Insert mode</strong>: used for adding text, entered by pressing i, a, o, or similar keys from normal mode.</li><li><strong>Visual mode</strong>: used for selecting text for copy, cut, or reformatting. Enter this mode with v (character-wise) or V (line-wise).</li></ul><p>Switching between modes always begins from normal mode. From normal mode, press i to insert before the cursor, a to append after it, or o to start a new line below. To return to normal mode from either insert or visual, press Esc.</p><p>Understanding the boundaries of each mode prevents accidental commands and unintended edits. For example, typing dd in normal mode deletes a full line, but in insert mode it just inserts those characters.</p><h3>Opening and Closing Files</h3><p>Opening a file is easy with Vim. The syntax is vim then the “nameofthefile” you want to open. Let’s say I want to open a file with the name “filename.txt”, I will use the following command:</p><pre>vim filename.txt</pre><p>The command above will either open the file if it exists or create a new one in the current directory with that name. When you first open a file with Vim, it starts in normal mode, so you won’t be able to insert text immediately until you switch to insert mode.</p><p>When you are done with inserting text and you want to close a file, press the Esc key to make sure you’re in normal mode, then type “:q” and press <strong>Enter</strong>. This is used when you have not inserted any text in the file.</p><p>If you’ve made changes to the file and want to save them, type “:wq” then press the <strong>Enter</strong> key.</p><p>Then, finally, if you have made changes to the file and want to quit without saving, type “:q!”.</p><p>In the next section, you will learn how to edit multiple files. For now, focus on editing single files until you feel comfortable with switching modes and navigation.</p><h3>Editing Multiple Files in Vim</h3><p>Vim provides multiple mechanisms to work across several files without needing multiple terminal tabs. The most common approaches involve buffers and tabs. Understanding these distinctions helps avoid confusion and unlocks Vim’s full potential.</p><p>To open multiple files at once, pass them as arguments when launching Vim:</p><pre>vim file1.txt file2.txt file3.txt</pre><p>That command opens all three files (file1.txt, file2.txt, file3.txt) in the same Vim session, puts you in the first file (file1.txt), and lets you switch between them with commands like :n (next file) and :prev.</p><h3>Working with Buffers in Vim</h3><p>A buffer in Vim is a memory representation of an open file that allows you to edit it without immediately writing changes to disk.</p><p>Each file is loaded into its own buffer. Only one buffer is visible at a time, but all remain in memory until explicitly deleted. You can switch between them using “:n” to move to the next file in the argument list, and “:prev” or “:N” to move to the previous one.</p><p>You might be in a situation where you need to see all buffers. You can do that with the keys “:ls”.</p><p>Each buffer is numbered, and you can jump to any one directly using “:buffer &lt;number&gt;”.</p><p>If you already have Vim open, you can load additional files with :edit filename.</p><p>If you want to view and edit two files side by side, use the key :split filename. The latter splits the window horizontally, letting you. Use :vsplit filename for vertical splits instead.</p><p>Moving between splits is done with <strong>Ctrl</strong>+<strong>w</strong> followed by an arrow key or <strong>h</strong>/<strong>j</strong>/<strong>k</strong>/<strong>l</strong>.</p><h3>Working with Tabs in Vim</h3><p>A tab in Vim is a separate workspace that holds its own layout of one or more windows, each displaying buffers independently from other tabs. Tabs are useful for grouping files by task or module.</p><p>For a tabbed layout, use :tabedit filename. Each tab contains its own window layout and buffer state. Navigate between tabs with :tabn (next tab) and :tabp (previous tab), or gt and gT. You can also switch to it using :tabn &lt;tab-number&gt;, for example, to switch to the third tab, you would use :tabn 3.</p><p>This is all done in normal mode. Tabs are useful for grouping files by task or module.</p><p>When working across many files, avoid closing Vim to switch contexts. Instead, rely on buffers and splits to keep all working files in memory. This preserves undo history and editing state, which is lost when quitting and reopening. Buffers and splits are central to using Vim like a modern IDE rather than a simple text editor.</p><h4>Closing Tabs in Vim</h4><p>To close a tab in Vim, first ensure you’re in <strong>normal mode</strong> by pressing Esc, then use the command:</p><pre>:tabclose</pre><p>This closes the <strong>current tab</strong> and any windows it contains. If you want to close a specific tab (not the current one), you must first switch to it using :tabn &lt;number&gt; and then run :tabclose.</p><p>To close <strong>all other tabs</strong> and keep only the current one:</p><pre>:tabonly</pre><p>If your tab contains unsaved changes, Vim will warn you before closing. To forcefully close a tab and discard changes, use:</p><pre>:tabclose!</pre><p>Unlike browsers, Vim tabs aren’t tied to individual files, they’re layouts of windows and buffers. So closing a tab doesn’t necessarily remove the associated file from memory; the buffer remains unless explicitly deleted.</p><h3>Basic Cursor Movement</h3><p>Vim does use arrow keys by design, but it maps movement to the home row for speed (these can only be used in normal mode):</p><ul><li>h moves left.</li><li>l moves right.</li><li>j moves down.</li><li>k moves up.</li></ul><p>This might feel unnatural initially, but it allows easy editing without moving hands away from the primary keys.</p><p>For word-level navigation:</p><ul><li>w jumps forward to the beginning of the next word.</li><li>b jumps backward to the beginning of the previous word.</li></ul><p>For line-level navigation:</p><ul><li>0 moves to the beginning of the line.</li><li>$ moves to the end of the line.</li></ul><p>Mastering these basic movements is the foundation of the best editing experience in Vim. Once they become second nature, you’ll navigate text faster and with less effort than relying on arrow keys or mouse movements.</p><h3>Composing Actions with Motions</h3><p>Another of Vim’s editing power comes from its composable command structure, where actions like delete (d), yank/copy (y), paste after the cursor(p), or change © are combined with movement keys that were mentioned earlier. This helps for a more precise edit.</p><p>In Vim, you define what you want to do and where to do it in a single keystroke sequence. For example, dw deletes from the cursor to the beginning of the next word, while d$ deletes from the cursor to the end of the line. Similarly, y2w yanks from the cursor position through the beginning of the second word ahead. For example, if your cursor is on the “H” in “Hello world today” and you run y2w, it would yank “Hello world “ (including the space after “world”).</p><p>c0 changes everything from the cursor back to the start of the line. This structure makes every command in Vim predictable; the same verbs can be used across different motions, reducing the need to memorize one-off shortcuts.</p><h3>Conclusion</h3><p>This module goes through the basic things you need to start using Vim. Went some concepts that might be new to you like modal editing, home row usage, and composable commands. With just these, you can start navigating and editing files more easily.</p><p>The challenge may be unlearning muscle memory from traditional editors and replacing it with Vim’s usage procedures.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e63e1c501af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an E-commerce Platform Authorization with Flask]]></title>
            <link>https://blog.devgenius.io/building-an-e-commerce-platform-authorization-with-flask-0e9f2a473ff2?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e9f2a473ff2</guid>
            <category><![CDATA[fastapi]]></category>
            <category><![CDATA[authorization]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[authentication]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Sat, 22 Mar 2025 22:23:16 GMT</pubDate>
            <atom:updated>2025-03-25T19:30:57.895Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*H0e897CkTQXt24Ui.png" /></figure><p>Are you looking for a standard and efficient way to handle the authorization in your e-commerce app?<br>You are in the right place. Authorization is a significant part of every application and it is important to handle this properly for security purposes.<br>Here, we will work with role-based authorization and see how we can restrict certain parts of your e-commerce application based on the user role. This is where Permit.io comes in. We will implement these authorization policies as code using Permit.io.<br>In this tutorial, we will work on building a role-based authorization for an e-commerce platform built with Flask.</p><h4><strong>Prerequisites</strong></h4><p>To follow along with this tutorial you should have the following:</p><ul><li>Basic understanding of Flask and Python web development.</li><li>Basic knowledge of user roles and permissions.</li><li>A stripe account.</li></ul><h3>OPA, OPAL and Permit.io</h3><p><strong>OPA (Open Policy Agent)</strong> is the foundational component in the authorization ecosystem. It’s a <strong>general-purpose decision-making engine</strong> designed to evaluate policies and make access control decisions. OPA operates by loading <strong>data</strong> (in the form of JSON documents) and <strong>policies. </strong>However, one of the challenges with OPA is keeping it updated with the latest policies and data, which is where <strong>OPAL</strong> comes into play.</p><p><strong>OPAL (Open Policy Administration Layer)</strong> is the next layer in the stack, designed to address the challenge of keeping OPA instances up to date. OPAL acts as a synchronization layer, tracking changes in <strong>Git repositories</strong> (where policies are stored as code) and <strong>external data sources</strong> (like payment systems or user databases). It ensures that OPA instances are always aware of the latest policies and data, enabling real-time updates. For example, if a user pays for a feature via Stripe, OPAL ensures that this information is immediately propagated to the OPA instances, allowing the user to gain access without delay. OPAL essentially bridges the gap between OPA and the dynamic data sources that influence access decisions.</p><p><strong>Permit.io</strong> is an open source platform that builds on top of OPA and OPAL, adding a layer of <strong>user-friendly interfaces</strong> and <strong>APIs</strong> to simplify the management of authorization policies. While OPA and OPAL are the underlying infrastructure, Permit.io provides the tools and experiences needed to make authorization accessible to both developers and non-technical users.</p><p>For instance, Permit.io offers a <strong>no-code policy editor</strong> that allows product managers or other team members to define roles and permissions using a simple interface. This abstraction layer empowers organizations to manage authorization policies without requiring deep technical expertise.</p><p>Permit.io bundles OPA and OPAL into a <strong>Policy Decision Point (PDP)</strong>, which can be deployed alongside applications to handle authorization queries efficiently.</p><h3>Authorization terminologies</h3><p>When implementing authorization, some terms come up repeatedly. Let’s learn about some of them that are relevant to this project</p><ul><li><strong>Policy Decision Point (PDP): </strong>The <a href="https://docs.permit.io/concepts/pdp/overview/">PDP</a> is the core service that evaluates your authorization requests against your defined policies. It can be deployed as a Docker container for production use or accessed as a cloud service for quick experimentation. We will be using the cloud for this tutorial.</li><li><strong>Policies: </strong><a href="https://docs.permit.io/how-to/build-policies/policy-basics/">Policies</a> are the rules that define which actions a user can perform on which resources. For example, you might create a policy stating: <em>“</em>Only users with the role <strong>admin</strong> can perform the <strong>access</strong> action on resources labeled as <strong>admin_page</strong>.<em>”</em></li><li>These policies are configured in the Permit.io dashboard and are used by the PDP during permission checks.</li><li><strong>Users: </strong>In authorization, a user represents an individual or system that makes an authorization request. Each user is identified by a unique identifier (such as an email) and may have additional attributes (e.g., role, department) that can be used in policy evaluations.</li><li><strong>Roles: </strong>Roles are labels assigned to users to group them by their access levels. For instance, in our project, a user can have a default role of <strong>user</strong> or an elevated role of <strong>admin</strong>. Policies can then be defined to grant broader access to those with the <strong>admin</strong> role.</li><li><strong>Actions: </strong>Actions describe the operations that users want to perform on a resource. Common actions include <strong>access</strong>, <strong>create</strong>, <strong>update</strong>, and <strong>delete</strong>. When you perform a permission check, you specify the action to see if the user is allowed to execute it.</li><li><strong>Resources: </strong>Resources are the objects or entities on which actions are performed. They can be simple string identifiers (like “admin_page” or “order”) or complex objects that include additional attributes. It’s up to you to define these identifiers in your policies so that the PDP knows how to interpret them.</li><li><strong>Tenants: </strong>In multi-tenant applications, a <a href="https://docs.permit.io/concepts/multi-tenant-authorization/#tenants-in-permitio">tenant</a> represents a separate customer or group within the same application instance. Permit.io allows you to include tenant identifiers in your policies, ensuring that access control rules are correctly applied on a per-tenant basis.</li></ul><h3>Project Overview</h3><p>The project leverages a combination of Flask session management and Permit.io’s role-based permission checks to enforce authorization throughout the application. By syncing user roles with Permit.io during registration and using Permit.io checks in decorators like @admin_required, the system ensures that:</p><ul><li>Only authenticated users can access protected routes.</li><li>Only users with admin privileges (verified via Permit.io) can access the admin dashboard.</li><li>Order management views are appropriately filtered based on the user’s role.</li></ul><p>This layered approach to authorization helps maintain robust security and access control within the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vEkdmwLIateK9HCs.png" /></figure><h3>Creating resources and roles</h3><p>For Permit.io to make the correct authorization decision, you must have set up policies in your Permit dashboard (or configuration) that include rules for what it means to access “admin_page”. For example, you might create a policy that states: <em>“</em>Only users with the role admin can perform the access action on resources labeled as admin_page.<em>”</em></p><p>Now let us set that up on the permit.io dashboard.</p><p>We will start by creating a new project. Go to <a href="http://app.permit.io/">app.permit.io</a> you will see Project option on the sidebar, click on it and click on the “<strong>New Project</strong>” button on the right-hand corner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zJ8lpHZ7Y4t5Lkp4.png" /></figure><p>Then you can create a project with the form provided. After the project is created we are all set to start creating policies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aSdK6ZeGTVJ1xzU3.png" /></figure><p>You will be provided with two environments, <em>dev</em> and <em>production</em>. Click on the “<strong>Open dashboard</strong>” button for the <em>dev</em> enviroment.</p><p>Now click on the “<strong>Policy</strong>” option on the sidebar then click on “<strong>New Resource</strong>” button to create the resources we need. We will create two resources admin_page and order and save.</p><p>For admin_page resource, attach the action access and for the order resource, will have the actions view and manage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L42bqlISHnuz-zWC.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tFPJhAxUI44zGtLy.png" /></figure><p>Now create roles for user and admin on the “<a href="https://app.permit.io/policy-editor/roles"><strong>Roles</strong></a>” tab. Be sure to delete all the roles provided by default if there are any.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bPsrbGG996JuXoh2.png" /></figure><p>Now go to the “<a href="https://app.permit.io/policy-editor"><strong>Policy Editor</strong></a>” tab to view your resources and select the actions that you need for each role. In this case we need the user to be able to access all the actions on the view and manage order and admin will be able to perform all actions. After selecting the actions don’t forget to save changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OCPmggmFjH0EBGFe.png" /></figure><h3>Building the Application</h3><p>In this section, we’ll walk through how to build a basic e-commerce web application using Python’s Flask framework. We’ll incorporate user authentication, shopping carts, Stripe for payment processing, and Permit.io for fine-grained role-based access control (RBAC).</p><p>To be able to communicate with PDP on the cloud you need to get the API key of your project. You can get this key on the environment of the project you are working with. Save that API somewhere because you will need it down the line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YudZ2MA7NXd-Cob8.png" /></figure><p>We will now start building the project. Before we begin, create a virtual environment and install some dependencies:</p><pre><br>virtualenv venv<br>source venv/bin/activate  # activate the virtual env<br>pip install flask flask_sqlalchemy stripe permit </pre><p>Create a new file in your project folder with the name main.py then paste in the following code this initial code is for the Flask configuration and the initial imports.</p><pre><br>from functools import wraps<br>from flask import Flask, render_template, request, redirect, url_for, session, flash, abort<br>from flask_sqlalchemy import SQLAlchemy<br>from werkzeug.security import generate_password_hash, check_password_hash<br>from permit.sync import Permit  # using the sync SDK<br>import stripe<br>app = Flask(__name__)<br>app.config[&#39;SECRET_KEY&#39;] = &#39;your_secret_key&#39;<br>app.config[&#39;SQLALCHEMY_DATABASE_URI&#39;] = &#39;sqlite:///ecommerce.db&#39;<br>app.config[&#39;SQLALCHEMY_TRACK_MODIFICATIONS&#39;] = False<br>db = SQLAlchemy(app)</pre><ul><li><strong>Flask(__name__):</strong> Initializes the Flask application.</li><li><strong>SECRET_KEY:</strong> A secret key is required by Flask to handle sessions and security. Replace ‘your_secret_key’ with a strong random string.</li><li><strong>SQLALCHEMY_DATABASE_URI:</strong> Points to a local SQLite database named ecommerce.db. You can change this to any other database engine (e.g., PostgreSQL) by adjusting the connection string.</li><li><strong>SQLALCHEMY_TRACK_MODIFICATIONS:</strong> Disables a signaling feature to save overhead.</li></ul><p>Paste in the configuration for stripe:</p><pre><br>stripe.api_key = &#39;sk_test_your_stripe_secret_key&#39;<br>STRIPE_PUBLIC_KEY = &#39;pk_test_your_stripe_public_key&#39;</pre><p>Replace these keys with your test or live credentials from your Stripe Dashboard. Now for the <strong>Permit.io configuration, replace </strong>paste-api-key-here with the API key you got earlier:</p><pre><br>permit = Permit(<br>    pdp=&quot;https://cloudpdp.api.permit.io&quot;,<br>    token=&quot;paste-api-key-here&quot;<br>)</pre><p>Now let us work on the user models for the database. Paste in the following code in the main.py file:</p><pre><br># ...<br>class User(db.Model):<br>    id = db.Column(db.Integer, primary_key=True)<br>    username = db.Column(db.String(80), unique=True, nullable=False)<br>    email = db.Column(db.String(120), unique=True, nullable=False)<br>    password_hash = db.Column(db.String(128), nullable=False)<br>    role = db.Column(db.String(20), default=&#39;user&#39;)  # &quot;user&quot; by default, or &quot;admin&quot;<br>    def set_password(self, password):<br>        self.password_hash = generate_password_hash(password)<br>    <br>    def check_password(self, password):<br>        return check_password_hash(self.password_hash, password)<br>class Product(db.Model):<br>    id = db.Column(db.Integer, primary_key=True)<br>    name = db.Column(db.String(100), nullable=False)<br>    description = db.Column(db.Text)<br>    price = db.Column(db.Float, nullable=False)<br>    image = db.Column(db.String(200))  # URL for the image<br>class Order(db.Model):<br>    id = db.Column(db.Integer, primary_key=True)<br>    user_id = db.Column(db.Integer, db.ForeignKey(&#39;user.id&#39;), nullable=False)<br>    total = db.Column(db.Float, nullable=False)<br>    timestamp = db.Column(db.DateTime, server_default=db.func.now())<br>    items = db.relationship(&#39;OrderItem&#39;, backref=&#39;order&#39;, lazy=True)<br>class OrderItem(db.Model):<br>    id = db.Column(db.Integer, primary_key=True)<br>    order_id = db.Column(db.Integer, db.ForeignKey(&#39;order.id&#39;), nullable=False)<br>    product_id = db.Column(db.Integer, db.ForeignKey(&#39;product.id&#39;), nullable=False)<br>    quantity = db.Column(db.Integer, nullable=False)<br>    price = db.Column(db.Float, nullable=False)</pre><p>Now we will develop a helper function and a decorator to help with the authorization:</p><pre><br>#...<br>def current_user():<br>    if &#39;user_id&#39; in session:<br>        return db.session.get(User, session[&#39;user_id&#39;])<br>    return None<br>def login_required(f):<br>    @wraps(f)<br>    def decorated_function(*args, **kwargs):<br>        if current_user() is None:<br>            flash(&quot;Please login first&quot;, &quot;warning&quot;)<br>            return redirect(url_for(&#39;login&#39;))<br>        return f(*args, **kwargs)<br>    return decorated_function<br>def admin_required(f):<br>    @wraps(f)<br>    def decorated_function(*args, **kwargs):<br>        user = current_user()<br>        print(user)<br>        if not user:<br>            flash(&quot;You must be logged in to access this page.&quot;, &quot;warning&quot;)<br>            return redirect(url_for(&quot;login&quot;))<br>        # Check permission using Permit.io<br>        # This call verifies if the user (using their email as unique id) is permitted to &quot;access&quot; the &quot;admin_page&quot;<br>        <br>        permitted = permit.check(<br>                user=user.username,<br>                resource=&quot;admin_page&quot;,<br>                action=&quot;access&quot;, <br>            )<br><br>        print(permitted)<br>        if not permitted:<br>            flash(&quot;You do not have permission to access the admin page.&quot;, &quot;danger&quot;)<br>            return redirect(url_for(&quot;index&quot;))<br>        return f(*args, **kwargs)<br>    return decorated_function</pre><p>Here the current_user function retrieves the current user from the session by checking if a user ID exists and querying the database; if no user is logged in, it returns <strong>None</strong>. The login_required decorator ensures that a user is logged in before executing the decorated route, redirecting to the login page with a warning message if not.</p><p>Similarly, the admin_required decorator first verifies that a user is authenticated, and then uses Permit.io to check if the logged-in user has permission to access the admin page based on their email; if either check fails, the user is redirected with an appropriate flash message indicating insufficient privileges.</p><p>Next, we will add the code for the views that will handle the functionality of the application. We will start with the user management views:</p><pre><br># ...<br>@app.route(&quot;/&quot;)<br>def index():<br>    products = Product.query.all()<br>    return render_template(&quot;index.html&quot;, products=products, user=current_user())<br># Registration – create user from the frontend<br>@app.route(&quot;/register&quot;, methods=[&quot;GET&quot;, &quot;POST&quot;])<br>def register():<br>    if request.method == &quot;POST&quot;:<br>        username = request.form[&#39;username&#39;]<br>        email = request.form[&#39;email&#39;]<br>        password = request.form[&#39;password&#39;]<br>        admin_code = request.form.get(&#39;admin_code&#39;, &#39;&#39;)  # Optional field to become admin<br>        if User.query.filter_by(email=email).first():<br>            flash(&quot;Email already registered&quot;, &quot;danger&quot;)<br>            return redirect(url_for(&#39;register&#39;))<br>        user = User(username=username, email=email)<br>        user.set_password(password)<br>        # By default, role is &quot;user&quot;. If the correct admin code is provided, assign admin.<br>        if admin_code == &quot;admin123&quot;:  # Replace with your secret admin code<br>            user.role = &quot;admin&quot;<br>        db.session.add(user)<br>        db.session.commit()<br>        # Sync the user to Permit.io<br>        user_data = {<br>            &quot;key&quot;: username,<br>            &quot;email&quot;: email,<br>            &quot;role_assignments&quot;:[{&quot;role&quot;: user.role, &quot;tenant&quot;: &quot;default&quot;}]<br>        }<br>        permit.api.users.create(user_data)<br>        flash(&quot;Registration successful. Please login.&quot;, &quot;success&quot;)<br>        return redirect(url_for(&#39;login&#39;))<br>    return render_template(&quot;register.html&quot;)<br># Login<br>@app.route(&quot;/login&quot;, methods=[&quot;GET&quot;, &quot;POST&quot;])<br>def login():<br>    if request.method == &quot;POST&quot;:<br>        email = request.form[&#39;email&#39;]<br>        password = request.form[&#39;password&#39;]<br>        user = User.query.filter_by(email=email).first()<br>        if user and user.check_password(password):<br>            session[&#39;user_id&#39;] = user.id<br>            flash(&quot;Logged in successfully&quot;, &quot;success&quot;)<br>            return redirect(url_for(&#39;index&#39;))<br>        else:<br>            flash(&quot;Invalid credentials&quot;, &quot;danger&quot;)<br>    return render_template(&quot;login.html&quot;)<br># Logout<br>@app.route(&quot;/logout&quot;)<br>def logout():<br>    session.pop(&#39;user_id&#39;, None)<br>    flash(&quot;Logged out&quot;, &quot;success&quot;)<br>    return redirect(url_for(&#39;index&#39;))<br># Admin Dashboard – only accessible by admins<br>@app.route(&quot;/admin&quot;)<br>@login_required<br>@admin_required<br>def admin():<br>    users = User.query.all()<br>    orders = Order.query.all()<br>    return render_template(&quot;admin.html&quot;, users=users, orders=orders, user=current_user())</pre><p>The index route retrieves all products from the database and renders them on the homepage along with the currently logged-in user’s information. The registration route handles both displaying the registration form and processing submitted data by creating a new user — after checking for duplicate emails and setting a special role if an admin code is provided before saving the user to the database and sending it to Permit.io.</p><p>If you have an advanced user creation process the solution for sending data to Permit may be different. You might want to set up a scheduled background task (using something like Celery) that queries your user database and syncs new or updated user data with Permit.</p><p>The admin dashboard route, which is accessible only to logged-in admin users, collects all user and order data from the database and renders it on an admin-specific page for management purposes. The admin permission works because we added the @admin_required decorator that was created earlier.</p><p>Now let us add the code for the cart, checkout, and payments:</p><pre><br>@app.route(&quot;/cart&quot;)<br>@login_required<br>def cart():<br>    cart = session.get(&#39;cart&#39;, {})<br>    items = []<br>    total = 0<br>    # cart is a dict with product_id (as string) keys and quantity values.<br>    for product_id, quantity in cart.items():<br>        product = Product.query.get(product_id)<br>        if product:<br>            item_total = product.price * quantity<br>            total += item_total<br>            items.append({<br>                &quot;product&quot;: product,<br>                &quot;quantity&quot;: quantity,<br>                &quot;total&quot;: item_total<br>            })<br>    return render_template(&quot;cart.html&quot;, items=items, total=total, user=current_user())<br># Add a product to the cart<br>@app.route(&quot;/add_to_cart/&lt;int:product_id&gt;&quot;)<br>@login_required<br>def add_to_cart(product_id):<br>    cart = session.get(&#39;cart&#39;, {})<br>    cart[str(product_id)] = cart.get(str(product_id), 0) + 1<br>    session[&#39;cart&#39;] = cart<br>    flash(&quot;Product added to cart&quot;, &quot;success&quot;)<br>    return redirect(url_for(&#39;index&#39;))<br># Checkout – display order summary and create a Stripe Checkout Session<br>@app.route(&quot;/checkout&quot;, methods=[&quot;GET&quot;, &quot;POST&quot;])<br>@login_required<br>def checkout():<br>    cart = session.get(&#39;cart&#39;, {})<br>    if not cart:<br>        flash(&quot;Cart is empty&quot;, &quot;warning&quot;)<br>        return redirect(url_for(&#39;index&#39;))<br>    items = []<br>    line_items = []<br>    total = 0<br>    for product_id, quantity in cart.items():<br>        product = Product.query.get(product_id)<br>        if product:<br>            item_total = product.price * quantity<br>            total += item_total<br>            items.append({<br>                &quot;product&quot;: product,<br>                &quot;quantity&quot;: quantity,<br>                &quot;total&quot;: item_total<br>            })<br>            line_items.append({<br>                &#39;price_data&#39;: {<br>                    &#39;currency&#39;: &#39;usd&#39;,<br>                    &#39;product_data&#39;: {<br>                        &#39;name&#39;: product.name,<br>                    },<br>                    &#39;unit_amount&#39;: int(product.price * 100),  # Stripe uses cents<br>                },<br>                &#39;quantity&#39;: quantity,<br>            })<br>    if request.method == &quot;POST&quot;:<br>        # Create a Stripe Checkout Session<br>        checkout_session = stripe.checkout.Session.create(<br>            payment_method_types=[&#39;card&#39;],<br>            line_items=line_items,<br>            mode=&#39;payment&#39;,<br>            success_url=url_for(&#39;payment_success&#39;, _external=True) + &#39;?session_id={CHECKOUT_SESSION_ID}&#39;,<br>            cancel_url=url_for(&#39;payment_cancel&#39;, _external=True),<br>        )<br>        return redirect(checkout_session.url, code=303)<br>    return render_template(&quot;checkout.html&quot;, items=items, total=total, stripe_public_key=STRIPE_PUBLIC_KEY, user=current_user())<br># Payment success – create an order record and clear the cart<br>@app.route(&quot;/payment_success&quot;)<br>@login_required<br>def payment_success():<br>    cart = session.get(&#39;cart&#39;, {})<br>    if not cart:<br>        flash(&quot;Cart is empty&quot;, &quot;warning&quot;)<br>        return redirect(url_for(&#39;index&#39;))<br>    total = 0<br>    order = Order(user_id=current_user().id, total=0)<br>    db.session.add(order)<br>    for product_id, quantity in cart.items():<br>        product = Product.query.get(product_id)<br>        if product:<br>            item_total = product.price * quantity<br>            total += item_total<br>            order_item = OrderItem(order=order, product_id=product.id, quantity=quantity, price=product.price)<br>            db.session.add(order_item)<br>    order.total = total<br>    db.session.commit()<br>    session[&#39;cart&#39;] = {}<br>    flash(&quot;Payment successful and order created&quot;, &quot;success&quot;)<br>    return redirect(url_for(&#39;orders&#39;))<br># Payment canceled<br>@app.route(&quot;/payment_cancel&quot;)<br>@login_required<br>def payment_cancel():<br>    flash(&quot;Payment canceled&quot;, &quot;warning&quot;)<br>    return redirect(url_for(&#39;checkout&#39;))</pre><p>This code implements the shopping cart and checkout flow in an e-commerce application. The /cart route retrieves the current user’s cart from the session, calculates each item’s total cost, and displays the cart summary.</p><p>The /add_to_cart/&lt;int:product_id&gt; route adds a specified product to the cart by incrementing its quantity in the session and then redirects the user back to the home page. In the /checkout route, the code aggregates cart items, calculates the total price, and formats the data for Stripe’s payment processing; if a POST request is made, it initiates a Stripe Checkout Session and redirects the user to complete the payment, while a GET request simply displays the checkout summary.</p><p>Following a successful payment, the /payment_success route creates an order record with its associated items, commits the transaction to the database, clears the cart, and redirects the user to their orders page, whereas the /payment_cancel route handles cancellations by notifying the user and redirecting them back to the checkout page.</p><p>Finally, we will handle the order route and its permissions:</p><pre><br># Order Management – users see only their orders; admins see all orders<br>@app.route(&quot;/orders&quot;)<br>@login_required<br>def orders():<br>    user = current_user()<br>    if user.role == &quot;admin&quot;:<br>        # Check if admin has permission to manage orders.<br>        permitted = permit.check(user.username, &quot;manage&quot;, &quot;order&quot;)<br>        if not permitted:<br>            flash(&quot;You are not permitted to manage orders&quot;, &quot;danger&quot;)<br>            return redirect(url_for(&#39;index&#39;))<br>        orders = Order.query.order_by(Order.timestamp.desc()).all()<br>    else:<br>        # Check if the user has permission to view orders.<br>        permitted = permit.check(user.username, &quot;view&quot;, &quot;order&quot;)<br>        if not permitted:<br>            flash(&quot;You are not permitted to view orders&quot;, &quot;danger&quot;)<br>            return redirect(url_for(&#39;index&#39;))<br>        orders = Order.query.filter_by(user_id=user.id).order_by(Order.timestamp.desc()).all()<br>    <br>    return render_template(&quot;orders.html&quot;, orders=orders, user=user)<br><br>if __name__ == &#39;__main__&#39;:<br>    app.run(debug=True)</pre><p>This code enforces strict authorization controls when accessing order information. It first ensures the user is logged in, then checks the user’s role: if the user is an admin, it verifies whether they have permission to manage orders; if the user is not an admin, it checks if they have permission to view orders. In both cases, if the required permission is not granted, the user is redirected with an appropriate error message, ensuring that only authorized actions are performed based on each user’s role and specific permissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7lL9D6knnMGACJSY.png" /></figure><p>Now it’s time to attach the frontend to this. You can get the complete project on <a href="https://github.com/khabdrick/flask-permitio">GitHub</a> where you will have the templates.</p><p>Now run the following command to initialize the database and create some sample products:</p><pre>python3 -c &quot;from main import app, db, Product; app.app_context().push(); db.create_all(); db.session.add_all([Product(name=&#39;Product 1&#39;, description=&#39;Description for product 1&#39;, price=10.99, image=&#39;https://m.media-amazon.com/images/I/61uPEs5504L._AC_UL1500_.jpg&#39;), Product(name=&#39;Product 2&#39;, description=&#39;Description for product 2&#39;, price=20.99, image=&#39;https://waziri.ng/wp-content/uploads/2024/01/Versace-Leather-White-Shoe-for-Men.jpg&#39;), Product(name=&#39;Product 3&#39;, description=&#39;Description for product 3&#39;, price=15.50, image=&#39;https://cdn.thewirecutter.com/wp-content/media/2024/11/runningshoes-2048px-09528.jpg?auto=webp&amp;quality=75&amp;width=1024&#39;)]); db.session.commit()&quot;</pre><p>Now you can run the project with:</p><pre>python main.py</pre><p>Open your browser and navigate to <a href="http://localhost:5000/">http://localhost:5000</a> to see the product listing. You can register (using the optional admin code “admin123” to create an admin), log in, add products to your cart, checkout via Stripe (in test mode), and view orders. Admin users can access the /admin page to see all users and orders. In a real-world application, you’d want a more secure method for creating or assigning admin privileges rather than relying on a hard-coded string.</p><p>When you register a new user you should see it on the User management page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CoVh7U3N6oJqmUI5.png" /></figure><p>If you login with your admin user, permit will check if that particular user is an admin, if yes you will be able to access the admin page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*LBgDnzL-L2Y1C4fo.gif" /></figure><h3>Conclusion</h3><p>Implementing role-based authorization in a Flask e-commerce platform with Permit.io ensures secure and structured access control across different user roles. By integrating Permit.io with OPA and OPAL, we leveraged a policy-driven approach to manage permissions dynamically, reducing the complexity of enforcing access restrictions in the application.</p><p>Of course, this is just a starting point but it covers most of the things you need to start using Permit for authorization. To build on what you have learned in this tutorial you can try adding features like customer-specific pricing based on roles, multi-tenant support for vendors managing their stores, attribute-based discounts and promotions, and audit logs to track admin actions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e9f2a473ff2" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/building-an-e-commerce-platform-authorization-with-flask-0e9f2a473ff2">Building an E-commerce Platform Authorization with Flask</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is Kubernetes an OS?]]></title>
            <link>https://khabdrick-dev.medium.com/is-kubernetes-an-os-ba285fa13c37?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba285fa13c37</guid>
            <category><![CDATA[os]]></category>
            <category><![CDATA[operating-systems]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Wed, 12 Feb 2025 13:31:41 GMT</pubDate>
            <atom:updated>2025-02-12T13:31:41.350Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JKsP3YuADV-L3HV-6Gtntg.png" /></figure><p>Kubernetes is not an operating system. It does not manage hardware resources like a traditional OS, nor does it provide direct access to system calls or hardware abstraction layers. Instead, Kubernetes operates as a container orchestration platform that schedules, deploys, and manages workloads across multiple machines. It builds on an existing OS rather than replacing it.</p><h3>How Kubernetes Interacts with the OS</h3><p>Kubernetes requires an underlying operating system — usually Linux — to function. It schedules containers on worker nodes that run a host OS, typically a Linux distribution like Ubuntu, CentOS, or a minimal OS such as CoreOS. Each node in a Kubernetes cluster runs the kubelet process, which interacts with the container runtime (e.g., containerd or CRI-O) to manage container execution. The container runtime itself relies on the host OS kernel for process isolation, networking, and storage management.</p><p>Unlike traditional OS schedulers, which handle single-machine process management, Kubernetes schedules workloads across a distributed cluster. The OS on each node handles low-level process execution, while Kubernetes provides an abstraction layer that orchestrates containerized applications at scale.</p><h3>Why Kubernetes Is Compared to an OS</h3><p>Kubernetes offers some OS-like functionality at the cluster level, which is why it is sometimes mischaracterized as an operating system. It provides resource allocation, security policies, process management (pods and containers), and networking abstractions. However, Kubernetes does not control hardware directly and does not replace the kernel or system utilities provided by a traditional OS.</p><p>From an architectural standpoint, Kubernetes functions as a distributed control plane rather than an OS kernel. The API server, scheduler, and controller manager coordinate the execution of workloads across nodes, ensuring optimal resource utilization and fault tolerance. These capabilities mirror aspects of an OS scheduler but operate at the level of distributed computing rather than a single machine.</p><h3>Kubernetes and Minimal OS Distributions</h3><p>Many Kubernetes deployments use minimal OS distributions optimized for running containers. Examples include Bottlerocket (from AWS), Talos Linux, and Flatcar Linux. These OSs strip away unnecessary user-space components, focusing solely on providing a stable environment for Kubernetes workloads. However, Kubernetes itself still relies on these OSs to provide essential kernel functions such as cgroups, namespaces, and network interfaces.</p><p>A common misconception is that Kubernetes abstracts away the OS entirely. While Kubernetes abstracts application deployment and service management, the underlying OS is still required for running the container runtime and handling system-level operations. Kubernetes does not provide low-level drivers, file system management, or direct device access, which remain the responsibility of the host OS.</p><h3>The Role of Kubernetes in Modern Infrastructure</h3><p>Kubernetes acts as an infrastructure abstraction layer but does not eliminate the need for an operating system. In cloud-native environments, Kubernetes enables workload portability by decoupling applications from the specifics of the host OS. This allows applications to run consistently across different environments, whether on-premises, in the cloud, or across hybrid setups.</p><p>However, Kubernetes does not remove OS dependencies; it shifts their management to infrastructure teams responsible for maintaining Kubernetes nodes. Security updates, kernel patches, and OS configurations remain essential tasks, even in a Kubernetes-managed environment. Tools like Kubernetes Operators, Helm charts, and GitOps workflows help automate infrastructure management, but they do not replace the role of the OS.</p><h3>Conclusion</h3><p>Kubernetes is not an OS, but it introduces cluster-level abstractions that resemble OS functionalities. It manages containerized workloads, schedules processes across nodes, and provides networking and security policies at scale. However, it still relies on an underlying OS to handle system-level operations. Treating Kubernetes as an operating system misunderstanding its role — it is a distributed control plane that orchestrates applications, not a replacement for the OS running on each node.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba285fa13c37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to use Kestra Python Plugin for Automating ETL Process]]></title>
            <link>https://khabdrick-dev.medium.com/how-to-use-kestra-python-plugin-for-automating-etl-process-2bf7dedf8318?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/2bf7dedf8318</guid>
            <category><![CDATA[etl-tool]]></category>
            <category><![CDATA[kestra]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[etl-pipeline]]></category>
            <category><![CDATA[etl]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Wed, 29 Jan 2025 18:06:01 GMT</pubDate>
            <atom:updated>2025-01-29T18:06:01.443Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XJ9TU8JFo9kj_5Fe.png" /></figure><p>As developers, we usually need automation of some repetitive work that gets boring over time. This is where a tool like the <a href="https://kestra.io/plugins/plugin-script-python">Kestra Python plugin</a> comes in. It can be used to do a variety of things like automating complex ETL processes, web scraping, integrating with external APIs and databases even CI/CD automation.</p><p>In this tutorial, you’ll learn how to set up the Python plugin and use it to automate data integration. Here, we will use it for automating data extraction, transformation, and loading (ETL) from two different sources into a central data warehouse.</p><h3>How to Set up Kestra and Python Plugin Locally</h3><p>In this section, we will set up Kestra to run locally, and then use the Python plugin to our flow.</p><p>First, get the <a href="https://github.com/kestra-io/kestra/blob/develop/docker-compose.yml">Kestra Docker compose file</a> and run it with the following command:</p><pre>Docker compose up</pre><p>If Kestra is now running, we can go ahead and start using the Python plugin in Kestra. We will use the <a href="https://kestra.io/plugins/plugin-script-python/tasks/io.kestra.plugin.scripts.python.script">Script task</a> which will help us run the Python code inside the Kestra YAML file, but you can check out the documentation for <a href="https://kestra.io/plugins/plugin-script-python">other options</a>.</p><p>Kestra will be running at <a href="http://localhost:8080/">http://localhost:8080/</a> so open it on your browser and create a new flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wJSfBac9GZI1h5zD" /></figure><p>You will be provided with some boilerplate code, delete that and paste in the following. This is like the basic starter for using the Python Plugin.</p><pre>id: TheKestraTask<br>namespace: kestra-task<br><br>tasks:<br>  - id: run_python<br>    type: io.kestra.plugin.scripts.python.Script<br>    warningOnStdErr: false<br>    script: | <br>      import requests<br>      import json<br><br>      response = requests.get(&quot;https://api.github.com&quot;)<br>      data = response.json()<br>      print(data)</pre><p>This task uses io.kestra.plugin.scripts.python.Script (the Python plugin) and the script attached to it send a GET request to the GitHub API, parses the JSON response, and prints the form data.</p><p>You don’t need to install the requests library using pip because the Python plugin already comes with it.</p><p>The warningOnStdErr: false setting ensures no warnings are raised for standard error output during execution.</p><p>Run the task by clicking the “<strong>Execute</strong>” button, and you should see some output data from the GitHub API in the “<strong>Logs</strong>” tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uMMz2g3n-56vyZIQ" /></figure><h3>Automating ETL Process</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/0*tNYvqhSdj9kFlzd9" /></figure><p>Here we will learn how Kestra can be used with the Python plugin to automate data extraction, transformation, and loading (ETL) from two different sources into a central data warehouse. We will use the weather API and COVID-19 data available on GitHub.</p><p>After extracting all those data, it is compiled as one and saved in a PostgreSQL database.</p><h3>Extract Data from OpenWeatherMap API</h3><p>The first step in our ETL process is getting the weather data. Here, we will write the task to extract weather data for a specific city using the OpenWeatherMap API.</p><pre>id: TheKestraTask<br>namespace: kestra-task<br><br>tasks:<br>  - id: extract_weather_data<br>    # ...<br><br>  - id: fetch_covid_data<br>    type: io.kestra.plugin.scripts.python.Script<br>    taskRunner:<br>      type: io.kestra.plugin.core.runner.Process<br>    outputFiles:<br>      - covid_data.csv<br>    warningOnStdErr: false<br>    script: |<br>      import pandas as pd<br><br>      url = &#39;https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/11-13-2022.csv&#39;<br>      covid_data = pd.read_csv(url)<br>      <br>      covid_data = covid_data[&#39;FIPS&#39;, &#39;Admin2&#39;, &#39;Province_State&#39;, &#39;Country_Region&#39;, &#39;Last_Update&#39;, &#39;Lat&#39;, &#39;Long_&#39;, &#39;Confirmed&#39;, &#39;Deaths&#39;, &#39;Recovered&#39;, &#39;Active&#39;]<br>      <br>      covid_data.to_csv(&#39;covid_data.csv&#39;, index=False)</pre><p>Here we are requesting the weather data for London from the OpenWeatherMap API using GET. It then reads a JSON response and writes it into a file named weather_data.json.</p><p>outputFiles is used to expose the created file to all the tasks.</p><p><em>You should make sure to replace the value of API_KEY in the code above with your OpenWeatherMap API key.</em></p><h3>Extract COVID-19 Data from GitHub</h3><p>As mentioned earlier, we will need another data source. Here we will extract COVID-19 data from a <a href="https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/11-13-2022.csv">GitHub repository</a>. This data includes confirmed cases, deaths, and recovered cases. This code should be added to the current flow as a new task.</p><pre>id: TheKestraTask<br>namespace: kestra-task<br><br>tasks:<br>  - id: extract_weather_data<br>    # ...<br><br>  - id: fetch_covid_data<br>    type: io.kestra.plugin.scripts.python.Script<br>    taskRunner:<br>      type: io.kestra.plugin.core.runner.Process<br>    outputFiles:<br>      - covid_data.csv<br>    warningOnStdErr: false<br>    script: |<br>      import pandas as pd<br><br>      url = &#39;https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/11-13-2022.csv&#39;<br>      covid_data = pd.read_csv(url)<br>      <br>      covid_data = covid_data[&#39;FIPS&#39;, &#39;Admin2&#39;, &#39;Province_State&#39;, &#39;Country_Region&#39;, &#39;Last_Update&#39;, &#39;Lat&#39;, &#39;Long_&#39;, &#39;Confirmed&#39;, &#39;Deaths&#39;, &#39;Recovered&#39;, &#39;Active&#39;]<br>      <br>      covid_data.to_csv(&#39;covid_data.csv&#39;, index=False)</pre><p>Just like we did with the weather data, we will be doing the same with the COVID-19 data. This is a little different since we are working with a CSV file. Here we are using pandas to write the data into a DataFrame and then extract all the columns we want to work on. Next, a new file is created in the workflow with those data. This file is also exposed to all tasks.</p><h3>Transform Data with Python</h3><p>In order to load the data into PostgreSQL, we must reformat both the weather and COVID-19 datasets to fit its organizational structure. We clean the data, perform necessary calculations, and merge them.</p><p>We will do this transformation process as the next step for importing them into our new database.</p><pre>id: TheKestraTask<br>namespace: kestra-task<br><br>tasks:<br>  - id: extract_weather_data<br>    # ...<br><br>  - id: fetch_covid_data<br>    # ...<br>  <br>  - id: transform_data<br>    type: io.kestra.plugin.scripts.python.Script<br>    outputFiles:<br>      - transformed_data.csv<br>    taskRunner:<br>      type: io.kestra.plugin.core.runner.Process<br>    warningOnStdErr: false<br>    script: |<br>      import pandas as pd<br>      import json<br><br>      # Load weather data<br>      with open(&#39;{{ outputs.extract_weather_data.outputFiles[&#39;weather_data.json&#39;] }}&#39;, &#39;r&#39;) as file:<br>          weather_data = json.load(file)<br>      <br>      # Load COVID-19 data<br>      covid_data = pd.read_csv(&#39;{{ outputs.fetch_covid_data.outputFiles[&#39;covid_data.csv&#39;] }}&#39;)<br><br>      # Transform weather data<br>      weather_df = pd.json_normalize(weather_data)<br>      weather_df[&#39;description&#39;] = weather_df[&#39;weather&#39;][0][0][&#39;description&#39;]<br>      weather_df = weather_df[[&#39;main.temp&#39;, &#39;main.humidity&#39;, &#39;description&#39;]]<br>      weather_df.columns = [&#39;temperature&#39;, &#39;humidity&#39;, &#39;description&#39;]<br>      # Aggregate COVID-19 data<br>      covid_summary = covid_data.groupby(&#39;Country_Region&#39;).agg({<br>          &#39;Confirmed&#39;: &#39;sum&#39;,<br>          &#39;Deaths&#39;: &#39;sum&#39;,<br>          &#39;Recovered&#39;: &#39;sum&#39;,<br>          &#39;Active&#39;: &#39;sum&#39;<br>      }).reset_index()<br><br>      # Merge weather data with COVID-19 summary data<br>      merged_data = pd.DataFrame({<br>          &#39;city&#39;: [&#39;London&#39;],<br>          &#39;temperature&#39;: [weather_df[&#39;temperature&#39;][0]],<br>          &#39;humidity&#39;: [weather_df[&#39;humidity&#39;][0]],<br>          &#39;weather_description&#39;: [weather_df[&#39;description&#39;][0]],<br>          &#39;confirmed_cases&#39;: [covid_summary[covid_summary[&#39;Country_Region&#39;] == &#39;United Kingdom&#39;][&#39;Confirmed&#39;].values[0]],<br>          &#39;deaths&#39;: [covid_summary[covid_summary[&#39;Country_Region&#39;] == &#39;United Kingdom&#39;][&#39;Deaths&#39;].values[0]],<br>          &#39;recovered&#39;: [covid_summary[covid_summary[&#39;Country_Region&#39;] == &#39;United Kingdom&#39;][&#39;Recovered&#39;].values[0]],<br>          &#39;active_cases&#39;: [covid_summary[covid_summary[&#39;Country_Region&#39;] == &#39;United Kingdom&#39;][&#39;Active&#39;].values[0]]<br>      })<br><br>      # Save the transformed data<br>      merged_data.to_csv(&#39;transformed_data.csv&#39;, index=False)</pre><p>The script first loads weather data from the JSON file and COVID-19 data from a CSV file. The weather data is transformed into a dataframe, selecting specific columns and renaming them for clarity. The COVID-19 data is then aggregated by country. Everything is now merged and saved to a new CSV file.</p><p><em>Note: The Python plugin also comes with pandas which is why it wasn’t installed.</em></p><h3>Load Data into PostgreSQL Data Warehouse</h3><p>Loading is the final step in our ETL workflow. Here, we load the transformed data into a PostgreSQL data warehouse. The data will include weather information and COVID-19 statistics. This task ensures that transformed data is accurately loaded into the PostgreSQL database, making it available for further analysis or reporting.</p><p>Now you can include the code to handle this.</p><pre>id: TheKestraTask<br>namespace: kestra-task<br><br>tasks:<br>  - id: extract_weather_data<br>    # ...<br><br>  - id: fetch_covid_data<br>    # ...<br>  <br>  - id: transform_data<br>    # ...<br><br>  - id: load_data<br>    type: io.kestra.plugin.scripts.python.Script<br>    taskRunner:<br>      type: io.kestra.plugin.core.runner.Process<br>    warningOnStdErr: false<br>    script: |<br>      import pandas as pd<br>      import psycopg2<br><br>      # Load transformed data<br>      transformed_data = pd.read_csv(&#39;{{ outputs.transform_data.outputFiles[&#39;transformed_data.csv&#39;] }}&#39;)<br><br>      conn = psycopg2.connect(<br>          host=&quot;postgres&quot;,<br>          database=&quot;kestra&quot;,<br>          user=&quot;kestra&quot;,<br>          password=&quot;k3str4&quot;<br>      )<br>      cursor = conn.cursor()<br><br>      # Create table if not exists<br>      create_table_query = &quot;&quot;&quot;<br>      CREATE TABLE IF NOT EXISTS transformed_data (<br>          city VARCHAR,<br>          temperature FLOAT,<br>          humidity FLOAT,<br>          weather_description VARCHAR,<br>          confirmed_cases INT,<br>          deaths INT,<br>          recovered INT,<br>          active_cases INT<br>      )<br>      &quot;&quot;&quot;<br>      cursor.execute(create_table_query)<br>      conn.commit()<br><br>      # Insert data into the table<br>      for index, row in transformed_data.iterrows():<br>          cursor.execute(<br>              &quot;INSERT INTO transformed_data (city, temperature, humidity, weather_description, confirmed_cases, deaths, recovered, active_cases) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)&quot;,<br>              (row[&#39;city&#39;], row[&#39;temperature&#39;], row[&#39;humidity&#39;], row[&#39;weather_description&#39;], row[&#39;confirmed_cases&#39;], row[&#39;deaths&#39;], row[&#39;recovered&#39;], row[&#39;active_cases&#39;])<br>          )<br>      conn.commit()<br>      cursor.close()<br>      conn.close()<br><br>      for index, row in transformed_data.iterrows():<br>          cursor.execute(<br>              &quot;INSERT INTO transformed_data (city, temperature, humidity, weather_description, confirmed_cases, deaths, recovered, active_cases) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)&quot;,<br>              (row[&#39;city&#39;], row[&#39;temperature&#39;], row[&#39;humidity&#39;], row[&#39;weather_description&#39;], row[&#39;confirmed_cases&#39;], row[&#39;deaths&#39;], row[&#39;recovered&#39;], row[&#39;active_cases&#39;])<br>          )<br>      conn.commit()<br>      cursor.close()<br>      conn.close()</pre><p>In the code above pandas is used to read the transformed data from a into a pandas DataFrame CSV file {{ outputs.transform_data.outputFiles[‘transformed_data.csv’] }}. Next, we use an SQL query to create a table named transformed_data if it does not already exist, with columns for city, temperature, humidity, weather description, confirmed cases, deaths, recovered cases, and active cases. This ensures the table structure is in place before any data insertion.</p><p>After the table is created, we iterate over each row in the DataFrame and insert the data into the transformed_data table. For each row an SQL insert statement is used to pass the correct values from the DataFrame columns to their respective table columns. After inserting all the data, we commit the transaction to ensure all changes are saved in the database. Finally, the cursor and database connection are closed to free up resources. After execution, you should see that all the tasks are executed as expected.</p><p>You can access the data by navigating to the Postgres database server running inside Docker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xtdc-KqY6-yOOpMq" /></figure><h3>Conclusion</h3><p>What we covered in this tutorial was a way of setting up the Python plugin to perform automated data integration. We automated the extraction and transformation of two different sources of data to one central Postgres data repository.</p><p>You can build on the knowledge that you have gained by implementing secrets in Kestra to secure your API key. You could also try working on better examples :D.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2bf7dedf8318" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can Kubernetes run in Debian?]]></title>
            <link>https://khabdrick-dev.medium.com/can-kubernetes-run-in-debian-3da3aabdc792?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/3da3aabdc792</guid>
            <category><![CDATA[debian-12]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[debian]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Thu, 23 Jan 2025 18:07:01 GMT</pubDate>
            <atom:updated>2025-01-23T18:07:01.584Z</atom:updated>
            <content:encoded><![CDATA[<p>Kubernetes can run in Debian. Debian is a widely used Linux distribution known for its stability and extensive package ecosystem. This tutorial demonstrates the installation and configuration of Kubernetes on a Debian-based system, covering all the required steps to set up a functioning cluster.</p><h4>Preparing the system</h4><p>Ensure the system is updated before installing Kubernetes components. Update the package list and upgrade existing packages:</p><pre>sudo apt update &amp;&amp; sudo apt upgrade -y</pre><p>Install essential tools and dependencies such as apt-transport-https, curl, and software-properties-common:</p><pre>sudo apt install -y apt-transport-https ca-certificates curl software-properties-common</pre><p>These tools are prerequisites for fetching Kubernetes packages and managing repositories.</p><h4>Adding the Kubernetes repository</h4><p>To install Kubernetes components, add the Kubernetes apt repository. Begin by importing the GPG key:</p><pre>curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -</pre><p>Next, configure the Kubernetes apt repository by creating a new source list file:</p><pre>echo &quot;deb https://apt.kubernetes.io/ kubernetes-xenial main&quot; | sudo tee /etc/apt/sources.list.d/kubernetes.list</pre><p>Update the package list to reflect the added repository:</p><pre>sudo apt update</pre><h4>Installing Kubernetes components</h4><p>Install the required Kubernetes tools: kubeadm, kubelet, and kubectl. These components enable cluster initialization, management, and control:</p><pre>sudo apt install -y kubeadm kubelet kubectl</pre><p>Mark these packages to prevent unintended updates:</p><pre>sudo apt-mark hold kubeadm kubelet kubectl</pre><p>This ensures system updates do not unintentionally disrupt your Kubernetes setup.</p><h4>Configuring the host machine</h4><p>To allow Kubernetes networking, disable swap as it conflicts with Kubernetes runtime requirements:</p><pre>sudo swapoff -a</pre><p>Prevent swap from re-enabling on reboot by commenting out the relevant line in /etc/fstab:</p><pre>sudo sed -i &#39;/ swap / s/^/#/&#39; /etc/fstab</pre><p>Ensure the required kernel modules are loaded and networking settings are configured:</p><pre>sudo modprobe overlay<br>sudo modprobe br_netfilter</pre><pre>cat &lt;&lt;EOF | sudo tee /etc/sysctl.d/kubernetes.conf<br>net.bridge.bridge-nf-call-ip6tables = 1<br>net.bridge.bridge-nf-call-iptables = 1<br>net.ipv4.ip_forward = 1<br>EOF</pre><pre>sudo sysctl --system</pre><p>These configurations enable packet forwarding and ensure the networking stack works as expected.</p><h4>Initializing the Kubernetes cluster</h4><p>Initialize the Kubernetes cluster using kubeadm. Replace POD_CIDR with the desired CIDR block for your pod network:</p><pre>sudo kubeadm init --pod-network-cidr=192.168.0.0/16</pre><p>After initialization, configure kubectl for the current user:</p><pre>mkdir -p $HOME/.kube<br>sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config<br>sudo chown $(id -u):$(id -g) $HOME/.kube/config</pre><h4>Setting up a pod network</h4><p>A pod network plugin is required for inter-pod communication. Install a compatible plugin like Calico:</p><pre>kubectl apply -f https://docs.projectcalico.org/v3.25/manifests/calico.yaml</pre><p>Verify that the pods in the kube-system namespace are running:</p><pre>kubectl get pods -n kube-system</pre><h4>Adding worker nodes</h4><p>To add worker nodes to the cluster, run the command displayed during kubeadm init on each worker node. For example:</p><pre>kubeadm join &lt;control-plane-endpoint&gt;:&lt;port&gt; --token &lt;token&gt; \<br>    --discovery-token-ca-cert-hash sha256:&lt;hash&gt;</pre><p>Replace &lt;control-plane-endpoint&gt;, &lt;port&gt;, &lt;token&gt;, and &lt;hash&gt; with the actual values from your control plane setup.</p><h4>Testing the cluster</h4><p>Verify that the cluster is functioning correctly by checking node and pod statuses:</p><pre>kubectl get nodes<br>kubectl get pods --all-namespaces</pre><p>Deploy a sample application to test pod functionality:</p><pre>kubectl create deployment nginx --image=nginx<br>kubectl expose deployment nginx --port=80 --type=NodePort</pre><p>Access the application using the NodePort assigned to the service.</p><h3>Summary</h3><p>Debian provides a solid foundation for running Kubernetes due to its compatibility and reliability. Following these steps results in a fully operational Kubernetes cluster, ready for container orchestration. Additional configurations, such as high availability or monitoring tools, can further enhance the setup depending on specific requirements.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3da3aabdc792" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Use the ping Command and How to Block it]]></title>
            <link>https://khabdrick-dev.medium.com/how-to-use-the-ping-command-and-how-to-block-it-a5ea8deb26fa?source=rss-6a733a606b36------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5ea8deb26fa</guid>
            <category><![CDATA[ubuntu-server]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[ping]]></category>
            <category><![CDATA[ubuntu-24-04]]></category>
            <dc:creator><![CDATA[Muhammed Ali]]></dc:creator>
            <pubDate>Wed, 22 Jan 2025 18:02:12 GMT</pubDate>
            <atom:updated>2025-01-22T18:02:12.238Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EwBGljBkgAuxJd_p.png" /></figure><p>Would you like to learn more about the ping command and how to block it when needed?</p><p>If the question above has come to your mind, then you are in the right place. Some people like to allow ping requests on their servers because it helps with verifying IP address connectivity, but it comes at a cost. Hence, you may need to block it.</p><p>In this article, you will learn what ping is, its usefulness, how to use the command, and how to block it when it’s necessary.</p><h3>Prerequisites</h3><ul><li>Linux Virtual Machine</li></ul><h3>What is ping?</h3><p>ping is a server utility that enables you to check the availability of an IP address. For example, if you have a random IP address, 202.78.174.220, there’s no way to confirm if it’s live or not, hence, ping.</p><p>The process of pinging involves sending an <a href="https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol">**ICMP</a> echo request** to a particular host or IP address and receiving an <strong>ICMP echo response</strong>. Given that ping is not blocked on the target server. If the IP address or host is live, you will receive some constant response showing that the IP address is live.</p><p>The ping command in Windows utilizes a 32-byte request and by default only transmits four pings. Ping packets are added with an 8-byte timestamp by the machine that receives them before sending them back to the sender. The network&#39;s round-trip delays are determined using the timestamp.</p><p>The ping command on Linux utilizes a 64-byte request and, by default, keeps sending requests until it receives a stop command. Each request is given a sequence number, and the system notifies the user when the request has been answered. It’s not necessary for the response to be received in the same sequence as the request.</p><h3>Reasons for Blocking ping Responses</h3><ol><li>Guards the machine against Distributed Denial-of-Service (DDoS) attacks.</li><li>Prevent expensive ICMP requests from your server.</li><li>To prevent <a href="https://en.wikipedia.org/wiki/Ping_flood">ping flood</a> on your virtual machine.</li></ol><h3>How to ping a Server</h3><p>You can ping a server by running the command below. Replace 202.78.174.19 with the IP address of your virtual machine.</p><pre>ping 202.78.174.19</pre><p>After running the command above, you should see something like the image below, showing that the server is active.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*sgbe_uLsaZjmXuRcg6xzjg.png" /></figure><h3>How to Block ping Response</h3><p>There are different ways to block ping responses for your server, but in this article, you will learn two, which can be applied in all use cases. The methods you will be trying out here are using the sysctl command and <strong>uncomplicated firewall</strong> (<strong>UFW</strong>). This will be done on the virtual machine, so you need to <a href="https://gcorelabs.com/support/articles/360012635517/">SSH into it</a> before going further into the article.</p><h3>Using the sysctl command</h3><p>The sysctl command can be used to read/write system parameters in the /proc/sys directory. You can use this command to write (-w) to net.ipv4.icmp_echo_ignore_all and set it to 1, which means the server should ignore all ICMP requests.</p><p>You can accomplish this by running the following command on your virtual machine:</p><pre>sudo sysctl -w net.ipv4.icmp_echo_ignore_all=1</pre><p>Now, if you try to ping the virtual machine, you will not be able to ping it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*yFjxJD0CYjD5YTAX0VzXxQ.png" /></figure><p>You can reactivate pinging by setting net.ipv4.icmp_echo_ignore_all=0.</p><h3>Using UFW</h3><p>As the name suggests, Uncomplicated Firewall (UFW) provides a simple way to set up a firewall for your server. You can use UFW to block ping requests by turning off ICMP input. You can turn off ICMP input by following the steps below:</p><ol><li>UFW comes with Ubuntu by default, but its activity is disabled. You can enable UFW on your server by running the following command:</li></ol><pre>sudo ufw enable</pre><p>2. Now allow SSH, so you can continue to access the server with SSH. You can do this by running the following command:</p><pre>sudo ufw allow ssh</pre><p>3. UFW has settings for ICMP stored in /etc/ufw/before.rules, you can edit that file by running the following command.</p><pre>sudo nano /etc/ufw/before.rules</pre><p>4. Scroll down to where you have # ok icmp codes for INPUT and replace the word “<strong>ACCEPT</strong>” with “<strong>DROP</strong>” as you see in the image below. Then type Ctrl+x and follow the prompts to save the changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*4Q7tD0Ev6V3ropMjOPR-XQ.png" /></figure><p>5. Now you have to reload UFW for the changes to take effect. You can do this by running the following command:</p><pre>sudo ufw reload</pre><p>When you try pinging the IP address, you will notice that it won’t go through. You can reverse this effect by changing the “DROP” parameters to “ACCEPT” in # ok icmp codes for INPUT.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5ea8deb26fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>