<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Goodness E. Eboh on Medium]]></title>
        <description><![CDATA[Stories by Goodness E. Eboh on Medium]]></description>
        <link>https://medium.com/@ebohgoodness.e?source=rss-d0e4b9fdc4e6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZWnUmqnPVNKK8pErStHGQg.jpeg</url>
            <title>Stories by Goodness E. Eboh on Medium</title>
            <link>https://medium.com/@ebohgoodness.e?source=rss-d0e4b9fdc4e6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 13:25:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ebohgoodness.e/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Best web scraping API for AI agents in 2026]]></title>
            <link>https://medium.com/zenrows/best-web-scraping-api-for-ai-agents-in-2026-c4406a2452b2?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4406a2452b2</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[web-scraping]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 20:50:29 GMT</pubDate>
            <atom:updated>2026-10-14T20:50:29.839Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This article was originally published on the </em><a href="https://www.zenrows.com/blog/best-web-scraping-api-for-ai-agents"><em>Zenrows blog</em></a><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nrbwQIPTHdync248hc3VPw.png" /></figure><p>Picture an AI agent built to track prices or watch job listings. In testing, it reads documentation and news sites without a hitch. Then it meets its first production target, a job board behind DataDome, and every answer it gives turns wrong.</p><p>The agent’s <a href="https://www.zenrows.com/glossary/web-data-access/what-is-a-web-scraping-api">web scraping API</a> probably looked great on paper. It returned clean Markdown, shipped an MCP server, and rendered JavaScript. Those are the features these APIs usually get compared on. The one feature that decides whether the agent sees the page at all rarely makes the list.</p><p>So we tested that feature directly. We sent 800 requests through four scraping APIs against DataDome, Cloudflare, and Akamai, then compared five APIs on everything else. Two of the four never returned a single usable page from the DataDome target.</p><h3>What you’ll learn</h3><ul><li>You’ll see how four scraping APIs performed against three major anti-bot systems and an unprotected baseline.</li><li>You’ll learn why speed and cost figures mislead when you read them apart from success rate.</li><li>You’ll see where Zenrows, Firecrawl, Jina AI Reader, CrawlForge, and Bright Data each fit best.</li><li>You’ll get working code to connect a scraping API to Claude Desktop, the OpenAI Agents SDK, and LangChain.</li></ul><h3>Why web scraping APIs for AI agents differ</h3><p>Public docs, company blogs, and news sites rarely put up a fight. E-commerce pricing pages, job boards, and industry directories are a different story. Many sit behind <a href="https://www.zenrows.com/glossary/web-data-access/what-is-an-anti-bot-system">anti-bot systems</a>.</p><p>A protected site might return a challenge page, an empty response, or partial content. To the agent, each one looks like a page. It reads whatever comes back, finds nothing useful, and carries on as if the source were empty.</p><p>That’s why this comparison starts with <a href="https://www.zenrows.com/glossary/web-data-access/what-is-protected-web-access">protected web access</a>. Output format and tooling matter only after the agent has the page.</p><h3>Benchmark methodology</h3><p>We tested Zenrows (using Fetch with mode=auto), Jina AI Reader, and CrawlForge on August 20, 2026. Firecrawl followed on August 25, 2026, in proxy=auto mode, which starts in standard mode and retries in stealth mode.</p><p>Each provider got 50 requests per target at two requests per second, for 800 requests in total. The targets were JobLookup behind DataDome, Chrono24 behind <a href="https://www.zenrows.com/blog/bypass-cloudflare">Cloudflare</a>, Reuters behind Akamai, and Wikipedia with no protection at all.</p><p>A request only counted as a success if it returned an HTTP 200 status code and a valid page title or meaningful data. A 200 with a challenge page inside counted as a failure.</p><p>Bright Data sat this one out because it requires enterprise KYC. Every figure below was recomputed on October 1, 2026, from the raw per-request results in the public <a href="https://github.com/ZenRows/ai-agent-web-scraping-benchmark">benchmark repository</a>, so you can check any of them yourself.</p><h3>Benchmark results</h3><p>On Wikipedia, every API scored 100 percent. The easy page proved nothing, and the protected targets told the story.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6OgJ-cd2wq3IWnWo82Y0Iw.png" /><figcaption>Protected-target success rate for Zenrows, Jina, Firecrawl and CrawlForge</figcaption></figure><p>Zenrows returned a usable page on all 200 of its requests, for a 100 percent average across the three protected targets. It was the only API to clear every request in the benchmark.</p><p>Jina came a close second at 99.33 percent. It failed exactly once, on JobLookup, and cleared every request to Chrono24 and Reuters.</p><p>Firecrawl averaged 66.67 percent. It was perfect on Chrono24 and Reuters, but all 50 JobLookup requests returned a reCAPTCHA page, even with stealth retries.</p><p>CrawlForge averaged zero percent on the protected targets. It got 403s from JobLookup and Chrono24 and a <a href="https://www.zenrows.com/glossary/http-errors/what-is-a-401-error">401</a> from Reuters.</p><h3>Response time and consumption</h3><p>Across all four targets, CrawlForge averaged 867 ms per request, Jina 1,257 ms, Firecrawl 1,497 ms, and Zenrows 5,215 ms. Read alone, those numbers would put CrawlForge first.</p><p>Read next to the success rates, they paint a different picture. CrawlForge was fast because it was turned away fast. Jina was the quickest API that returned content from protected targets. Zenrows took the longest on the protected pages, and those are the pages where it was the only API to clear every request.</p><p>Cost works the same way. Each provider bills in its own unit, so the figures don’t compare directly. Zenrows averaged 4.59 credits per request, Jina 24,922.58 tokens, and Firecrawl and CrawlForge one credit each. A one-credit request buys you little when the page it returns is a challenge.</p><h3>How Zenrows credits scale with protection</h3><p>The Zenrows average hides a pattern worth seeing. mode=auto escalates only as far as each target forces it to, so different pages cost very different amounts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*80b-Kw8kqhGjhc580IkGRw.png" /><figcaption>Zenrows credits per successful request by target protection</figcaption></figure><p>The unprotected Wikipedia page cost one credit. Reuters, behind Akamai, cost ten. Chrono24, behind Cloudflare, averaged 2.36 because not every request needed the full escalation.</p><p>For an agent whose targets mix easy and hard pages, that matters. You pay in proportion to the protection each page puts up.</p><h3>Best web scraping APIs for AI agents compared</h3><h3>Zenrows</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ivY82O2gIweO6PrcnrU0Q.png" /><figcaption><em>Zenrows homepage</em></figcaption></figure><p>Zenrows is the strongest fit for agents that need protected and dynamic sources. Beyond the benchmark, Zenrows publishes a 99.93 percent success rate across its wider target set. <a href="https://docs.zenrows.com/fetch/api-reference">Zenrows Fetch</a> handles Cloudflare, DataDome, Akamai, and PerimeterX in one call, along with <a href="https://www.zenrows.com/glossary/web-data-access/what-is-javascript-rendering">dynamic website support</a>.</p><p>It returns <a href="https://www.zenrows.com/glossary/web-data-extraction/what-is-llm-ready-content">LLM-ready</a> Markdown, plain text, or PDF. Zenrows Extract returns <a href="https://www.zenrows.com/glossary/web-data-extraction/what-is-schema-based-extraction">structured JSON</a> without pre-written CSS selectors.</p><p>Its <a href="https://www.zenrows.com/agents">Agent Toolkit</a> ships on every plan. The <a href="https://www.zenrows.com/blog/zenrows-mcp-server">MCP server</a> exposes scrape, extract, and more than 30 browser_* tools, with setup guides for hosts like Claude Desktop, Cursor, and VS Code and frameworks like the OpenAI Agents SDK and Google ADK. A <a href="https://www.zenrows.com/cli">CLI</a> serves terminal-native agents.</p><p>The most unusual piece is <a href="https://agents.zenrows.com/">agentic payments</a>. An agent can read a machine-readable price catalogue, settle through x402, MPP, or a card, and finish a purchase when it hits a 402 challenge. For an autonomous agent, that’s the gap between a run that finishes and one that stalls overnight waiting for a human.</p><p><a href="https://docs.zenrows.com/batch/setup">Zenrows Batch</a> handles async jobs of up to 100,000 URLs. The free tier gives you 5,000 credits a month with no card, and paid plans start at $16/month. Billing is <a href="https://www.zenrows.com/glossary/web-data-access/what-does-pay-per-successful-request-mean">pay per successful request</a>, so blocked requests cost nothing.</p><h3>Firecrawl</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RlwZ15B5DfCwdKMau7oeQw.png" /><figcaption><em>Firecrawl homepage</em></figcaption></figure><p><a href="https://www.zenrows.com/blog/best-firecrawl-alternative-for-anti-bot-bypass">Firecrawl</a> suits agents whose sources are unprotected or lightly protected. Its output is built for LLMs, with Markdown, JSON Schema generation, and metadata extraction designed for <a href="https://www.zenrows.com/glossary/ai-agents-and-the-web/what-is-web-scraping-for-rag">RAG</a> and chunking.</p><p>It also offers /agent and /interact endpoints for multi-page exploration and browser control. Its AI Extract feature needs a separate subscription. The free plan includes 1,000 credits a month, and paid plans start at $19/month.</p><h3>Jina AI Reader</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R55rid7qPWVTa5QSo8B0nQ.png" /><figcaption><em>Jina AI Reader homepage</em></figcaption></figure><p>Jina converts any URL into <a href="https://www.zenrows.com/glossary/web-data-extraction/what-is-html-to-markdown-conversion">LLM-ready Markdown</a> when you prepend r.jina.ai/ to it. It needs no setup and includes image captioning for multimodal agents.</p><p>Its token usage tracks the amount of content returned, making large crawls harder to budget for. Jina offers 10 million free tokens, and paid usage starts at $50. It’s a strong match for quick lookups of documentation, news, and public content.</p><h3>CrawlForge</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NrnA3z-wxpuT0BBRkYzIMQ.png" /><figcaption><em>CrawlForge homepage</em></figcaption></figure><p>CrawlForge is MCP-native, with 31 specialized tools an agent can discover without wrapper code. It produces token-efficient Markdown, can run extraction schemas locally through Ollama, and keeps pricing predictable per tool.</p><p>Its free tier includes 1,000 credits, and paid plans start at $19/month. It works best for agents on MCP-native hosts that mostly read unprotected sources.</p><h3>Bright Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Hv8MULr2OnkhAGSz3_VJA.png" /><figcaption><em>Bright Data homepage</em></figcaption></figure><p><a href="https://www.zenrows.com/blog/best-bright-data-alternative-for-self-service-scraping">Bright Data</a> is the enterprise option, with more than 150 million IPs and SOC 2 Type II and ISO 27001 certifications. Its Web Unlocker, Browser API, and Residential Proxies products each have $499/month Scale plans.</p><p>That pricing puts it beyond a typical individual developer or small team. It fits enterprise teams with five-figure data budgets, compliance requirements, and dedicated data engineers.</p><h3>Integrating Zenrows into your agent</h3><p>If protected access matters for your targets, connecting Zenrows takes a few lines. All three patterns below need a Zenrows API key from the <a href="https://app.zenrows.com/register">signup page</a>.</p><h3>Claude Desktop</h3><p>The quickest route is an MCP host. Add this block to claude_desktop_config.json, then restart <a href="https://docs.zenrows.com/integrations/claude-desktop">Claude Desktop</a>.</p><pre>{<br>  &quot;mcpServers&quot;: {<br>    &quot;zenrows&quot;: {<br>      &quot;command&quot;: &quot;npx&quot;,<br>      &quot;args&quot;: [&quot;-y&quot;, &quot;@zenrows/mcp&quot;],<br>      &quot;env&quot;: {<br>        &quot;ZENROWS_API_KEY&quot;: &quot;YOUR_ZENROWS_API_KEY&quot;<br>      }<br>    }<br>  }<br>}</pre><p>Claude will now call the scrape and browser_* tools whenever a task needs live web data. If you’d like to compare this server with the alternatives, we tested seven <a href="https://www.zenrows.com/blog/best-scraping-mcp-servers">MCP servers for web scraping</a>.</p><h3>OpenAI Agents SDK</h3><p>The <a href="https://docs.zenrows.com/integrations/openai-agents-sdk">OpenAI Agents SDK</a> supports MCP natively through HostedMCPTool. That means you can point an agent at the hosted Zenrows server with no custom function-calling code.</p><pre>import asyncio<br>import os<br>from agents import Agent, HostedMCPTool, Runner</pre><pre>ZENROWS_API_KEY = os.environ[&quot;ZENROWS_API_KEY&quot;]</pre><pre>agent = Agent(<br>    name=&quot;Web Research Assistant&quot;,<br>    instructions=&quot;Use live web data when you need current information.&quot;,<br>    tools=[<br>        HostedMCPTool(<br>            tool_config={<br>                &quot;type&quot;: &quot;mcp&quot;,<br>                &quot;server_label&quot;: &quot;zenrows&quot;,<br>                &quot;server_description&quot;: &quot;Web data access through ZenRows.&quot;,<br>                &quot;server_url&quot;: &quot;https://mcp.zenrows.com/mcp&quot;,<br>                &quot;authorization&quot;: ZENROWS_API_KEY,<br>                &quot;require_approval&quot;: &quot;never&quot;,<br>            }<br>        )<br>    ],<br>)</pre><pre>async def main():<br>    result = await Runner.run(<br>        agent,<br>        &quot;Visit <a href="https://news.ycombinator.com/">https://news.ycombinator.com/</a> and summarize the three most recent posts.&quot;,<br>    )<br>    print(result.final_output)</pre><pre>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(main())</pre><p>If you’d prefer to register Fetch as a plain function tool, we walk through that in <a href="https://www.zenrows.com/blog/web-aware-agent-openai-agents-sdk-zenrows">building a web-aware agent with the OpenAI Agents SDK</a>.</p><h3>LangChain</h3><p>For <a href="https://www.zenrows.com/blog/langchain-web-scraping">LangChain</a>, the langchain-zenrows package turns Zenrows into a tool the agent calls whenever it needs live data.</p><pre>from langchain_zenrows import ZenRowsUniversalScraper<br>from langchain_openai import ChatOpenAI<br>from langgraph.prebuilt import create_react_agent<br><br>llm = ChatOpenAI(model=&quot;gpt-4o-mini&quot;)<br><br>zenrows_tool = ZenRowsUniversalScraper()<br><br>agent = create_react_agent(llm, [zenrows_tool])<br><br>result = agent.invoke({<br>    &quot;messages&quot;: [<br>        {<br>            &quot;role&quot;: &quot;user&quot;,<br>            &quot;content&quot;: &quot;Scrape https://www.etsy.com/ in Markdown and return the four cheapest products in JSON.&quot;<br>        }<br>    ]<br>})</pre><p>Set ZENROWS_API_KEY as an environment variable before you run it.</p><h3>What the benchmark settles</h3><p>Output format and tool count are the easy part of this category, and every API here does them well. Protected access decides whether your agent returns an answer or a challenge page, and that’s where the tools disagree most.</p><p>Across 800 requests, Zenrows was the only API that returned a usable page every time, and it charged one credit for the page that needed nothing. Jina came closest and runs faster when the target is open. Firecrawl and CrawlForge are reasonable picks for agents whose targets stay public.</p><p>So choose based on the targets your agent will hit. If any of them sit behind an anti-bot system, <a href="https://app.zenrows.com/register">start with 5,000 free credits</a> and run your own list through Fetch before you commit. The full per-target tables and the side-by-side comparison are in the <a href="https://www.zenrows.com/blog/best-web-scraping-api-for-ai-agents">original post</a>.</p><h3>Key takeaways</h3><ul><li>Every API scored 100 percent on an unprotected page, so test on the protected targets your agent will hit.</li><li>Zenrows was the only API to return a usable page on all 200 benchmark requests, and Jina came second at 99.33 percent.</li><li>Response time and cost figures only mean something next to success rate, because a fast failure still fails.</li><li>With mode=auto, Zenrows charged one credit for an unprotected page and ten for an Akamai-protected one.</li><li>You can connect a scraping API to Claude Desktop, the OpenAI Agents SDK, or LangChain in a few lines of code.</li></ul><h3>FAQ</h3><p><strong>Can AI agents scrape websites protected by Cloudflare or DataDome?</strong></p><p>They can, as long as the API they call handles the protection. In our benchmark, two of the four APIs tested returned nothing usable from the DataDome target across 50 requests each. Zenrows returned a valid page on every request to the Cloudflare, DataDome, and Akamai targets.</p><p><strong>Is there a free web scraping API for AI agents?</strong></p><p>Several have free tiers. Zenrows offers 5,000 credits a month with no card, Firecrawl offers 1,000 credits, Jina AI Reader offers 10 million tokens, and CrawlForge offers 1,000 credits. Bright Data also offers 5,000 free credits a month, shared across several products.</p><p><strong>What’s the difference between a web scraping API and a scraping MCP server?</strong></p><p>A web scraping API fetches the page. An MCP server hands that service to an agent as a callable tool. Zenrows offers both, so the same capability works over REST, through the CLI, or as a tool your agent discovers automatically.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4406a2452b2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/zenrows/best-web-scraping-api-for-ai-agents-in-2026-c4406a2452b2">Best web scraping API for AI agents in 2026</a> was originally published in <a href="https://medium.com/zenrows">Zenrows</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloudflare Workers vs EC2: how edge deployments affect latency across regions]]></title>
            <link>https://medium.com/@ebohgoodness.e/cloudflare-workers-vs-ec2-how-edge-deployments-affect-latency-across-regions-08bb64b9818a?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/08bb64b9818a</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[edge-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Thu, 30 Apr 2026 14:27:07 GMT</pubDate>
            <atom:updated>2026-04-30T15:02:09.483Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zRY7lb6b11nnkYT1zvIE9A.png" /></figure><p>Edge deployments claim to solve the puzzle of high network latency. You’d hear the common saying, “The edge is faster, and latency ranges between 10–50 ms.” But how accurate are those figures? Are they absolute?</p><p>For teams with workflows across multiple locations, serving users all over the globe, what’s important is knowing how much of this claim is a measurable reality vs. fluff. They want to understand the practical nuances that edge deployments carry: the middle-mile reality, cold-start delay, routing tax, etc.</p><p>This article explores how edge deployments affect latency in practice. I ran a simple experiment using Cloudflare Workers and EC2 to uncover where the edge succeeds and where it still has limitations. By the end, you’ll have a clearer view of what those latency claims actually look like and insights you can apply to production-grade deployments.</p><h3>Does deploying code closer to users actually reduce latency in a measurable way?</h3><p>To determine whether “closer” translates to measurable speed, you have to move past the term “latency” and examine the network path anatomy. The path can be broken down into four parts.</p><p>The first mile, the middle mile, the protocol tax, and the processing tail. When you study each delay in these segments as a cumulative stack rather than a single number, it becomes easier to understand how they arise and where the edge makes a difference. Let’s observe each.</p><h4><strong>The first mile (last mile from the provider’s perspective)</strong></h4><p>Depending on the perspective, for users, it’s the first mile; for providers, it is the last mile. Think of it as the road from within a street before joining the digital highway. The first mile is the path that connects the end-user device to the <a href="https://www.cloudflare.com/learning/cdn/glossary/internet-exchange-point-ixp/">Internet Exchange Point</a> (IXP).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cWZS7zu4uhNw1DhxoHg4lw.png" /><figcaption>The first-mile network path from the user’s device to the Internet Exchange Point (IXP), where traffic enters the broader internet.</figcaption></figure><p>Within these connection paths, delays can arise. Here are the types:</p><ul><li><strong>Jitter (the inconsistency tax): </strong>The time it takes for packets to travel from the user’s device to the cell tower can vary. This is often caused by network congestion during peak hours or signal degradation from the ISP.</li><li><strong>Congestion and oversubscription:</strong> ISPs do not provide users with dedicated communication pipelines, so they rely on contention ratios (e.g., 1:50). The assumption is that not everyone will use the internet at the same time. If that happens, it can lead to lag and <a href="https://en.wikipedia.org/wiki/Bufferbloat">bufferbloat</a> due to queuing.</li><li><strong>Equipment resolution time: </strong>In theory, a user request traveling 10 km to a local exchange should take about 0.05 ms, since signal propagation in fiber is close to the speed of light. But in reality, it can take anywhere from 10 ms to 100 ms because routers need time to process and forward packets to the IXP.</li></ul><p>The key point is that the edge has virtually no control over these delays because it only receives data at the IXP. Irrespective of the system, whether edge or centralized cloud, first-mile delays directly impact overall latency.</p><h4><strong>The middle mile</strong></h4><p>The middle mile is the physical network of subsea fiber-optic cables that connect major hub stations around the world. But if you think about data traveling through these cables in terms of geographical distance, you would most likely imagine the wrong routes.</p><p>For example, data requests traveling from Lagos, Nigeria, to us-east-1 (Virginia) almost never go straight across the Atlantic.</p><p>The <a href="https://www.cloudflare.com/learning/security/glossary/what-is-bgp/">Border Gateway Protocol</a> routes traffic from Lagos to Lisbon or London (via<a href="https://www.submarinecablemap.com/submarine-cable/west-africa-cable-system-wacs"> West Africa Cable System</a> (WACS)/<a href="https://www.submarinecablemap.com/submarine-cable/africa-coast-to-europe-ace">Africa Coast to Europe</a> (ACE)), then to New York, and finally to Virginia. This routing directly increases travel distance and is subject to the physical condition of the underlying network infrastructure. A faulty router in Portugal might cause a reroute, with your data passing through Brazil before reaching Virginia. In terms of latency, a request that would have taken ~70–100 ms on a more direct path might instead take 150–300 ms due to detours and routing decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-T1aQXtyXC5nZHeJOB45Tg.png" /><figcaption>The middle mile tax: routing paths from Lagos to Virginia showing how detours and BGP decisions add latency.</figcaption></figure><p>Edge systems can help reduce latency here. Once an edge system, such as Cloudflare, receives your request from a large IXP, it can route your data over its own private network. It also uses technologies like <a href="https://www.cloudflare.com/application-services/products/argo-smart-routing/">Argo Smart Routing</a> to detect which paths are currently the fastest and switch routes in near real time.</p><h4><strong>The protocol tax</strong></h4><p>Before a user can resolve code, especially on the first request, the network has to negotiate the connection. For secure HTTPS requests, this means the user’s browser and the server hosting the code must perform a <a href="https://developer.mozilla.org/en-US/docs/Glossary/TCP_handshake">TCP handshake </a>(SYN, SYN-ACK, ACK) and a <a href="https://www.cloudflare.com/learning/ssl/why-use-tls-1.3/">TLS 1.3</a> handshake (key exchange/certificate). These add up to about two round-trip times (RTT) between the user and the server.</p><p>In a situation where a user in Toronto is hitting a server in London with a 100 ms RTT, that’s already about 150–200 ms of TCP and TLS overhead before the request even starts. All of this adds up as latency.</p><p>The edge doesn’t directly remove the protocol tax; round trips still have to occur. But it can reduce latency if the Toronto user is hitting an edge node in Toronto rather than a server in London. Aside from the localized node advantage, even for a first-time user accessing a site, edge nodes can benefit from already established connection pools. This is because edge systems maintain global connectivity to major data centers being accessed by users.</p><p>So, as long as there is already an active connection from the edge node to the origin in London, a first-time user may not fully pay the TCP setup cost.</p><h4><strong>The processing tail</strong></h4><p>The latency tax spent at the processing tail is a matter of computing architecture rather than core networking itself. The total processing tail is an addition of initialization latency (cold start) and execution latency. These values vary depending on workload and platform, but you can use representative ranges to understand how they behave.</p><p>Since edge Workers use <a href="https://developers.cloudflare.com/workers/reference/how-workers-works/#isolates">V8 isolates</a>, they can experience cold starts, especially when the node hasn’t been called before. Whereas a bare-metal server or a VM that’s always running wouldn’t need to start up. When you look at it this way, the edge can actually add to latency at this stage.</p><p>Execution latency is also an area where the edge can introduce more delay. When serving a simple API, the edge might have an execution latency of ~2 ms. Still, if heavy image processing is required or complex algorithmic computation is involved, the edge might experience a higher latency of 100–500 ms. This is due to the CPU limits edge nodes have. A single Worker operates under constraints such as ~128 MB of memory and CPU time limits that can be very low (e.g., 10–50 ms) depending on the plan. Even though paid plans allow higher CPU time limits, they are still constrained compared to centralized servers.</p><p><strong>Think of the math like this:</strong></p><p>Total Processing Tail = Initialization + Execution</p><p><strong>Scenario A (Simple API):</strong></p><ul><li>Edge: 5 ms (Init) + 2 ms (Exec) = 7 ms</li><li>EC2: 0 ms (Init) + 1 ms (Exec) = 1 ms</li><li>The edge is barely slower on compute</li></ul><p><strong>Scenario B (Heavy Image Processing):</strong></p><ul><li>Edge: 5 ms (Init) + 500 ms (Exec due to CPU limits) = 505 ms</li><li>EC2: 0 ms (Init) + 50 ms (Exec on a stronger CPU) = 50 ms</li><li>Here, EC2’s raw compute power outweighs the edge’s location advantage</li></ul><p>But this math alone is not the full picture. Remember the other taxes along the way: the first mile, the middle mile, and the protocol tax. The edge helps reduce latency on the middle mile and protocol tax, as highlighted earlier. This means the edge reduces latency along the network path until the bottleneck becomes compute.</p><p>I like to think of an edge node as a high-speed drone. It’s sitting right outside your door, but it can only carry a small package and has a limited battery life.</p><p>So while the commonly marketed 10–50 ms latency can be achieved, system architecture, network path, the user’s local connection, and the type of code all influence the result.</p><p>To better understand these latency trade-offs, let’s look at real-world results from a centralized server and an edge deployment that handle the same request.</p><h3>Experiment setup</h3><p>I built a controlled latency probe to run on both <a href="https://developers.cloudflare.com/workers/reference/how-workers-works/">Cloudflare Workers</a> and an <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Instances.html">EC2 instance</a>. The goal of the experiment is to observe where the edge affects latency compared to a centralized server. Below are the probe codes.</p><p>Edge Worker latency probe:</p><pre><br>export default {<br>  async fetch(request, env, ctx) {<br>    const start = Date.now();<br>    const url = new URL(request.url);<br><br>    // identify test runs<br>    const testId = url.searchParams.get(&quot;test&quot;) || &quot;default&quot;;<br><br>    // simulate small work<br>    await new Promise(r =&gt; setTimeout(r, 5));<br><br>    const end = Date.now();<br><br>    const data = {<br>      message: &quot;Edge Latency Probe&quot;,<br><br>      // Edge + location info<br>      colo: request.cf.colo,<br>      country: request.cf.country,<br>      city: request.cf.city,<br><br>      // Experiment tracking<br>      test_id: testId,<br><br>      // Worker execution time<br>      worker_processing_ms: end - start,<br><br>      // Timestamp<br>      server_time: new Date(end).toISOString(),<br><br>      // Debug info<br>      user_agent: request.headers.get(&quot;user-agent&quot;)<br>    };<br><br>    return new Response(JSON.stringify(data, null, 2), {<br>      headers: {<br>        &quot;Content-Type&quot;: &quot;application/json&quot;,<br>        &quot;Cache-Control&quot;: &quot;no-store&quot;,<br>        &quot;Server-Timing&quot;: `cfEdge;desc=&quot;${request.cf.colo}&quot;`<br>      }<br>    });<br>  }<br>};</pre><p>EC2 server latency probe:</p><pre><br>const http = require(&quot;http&quot;);<br><br>const server = http.createServer((req, res) =&gt; {<br>  const start = Date.now();<br><br>  setTimeout(() =&gt; {<br>    const end = Date.now();<br><br>    const data = {<br>      message: &quot;Centralized server&quot;,<br>      region: &quot;us-east-1&quot;,<br>      processing_time_ms: end - start,<br>      timestamp: new Date(end).toISOString()<br>    };<br><br>    res.writeHead(200, {<br>      &quot;Content-Type&quot;: &quot;application/json&quot;,<br>      &quot;Cache-Control&quot;: &quot;no-store&quot;<br>    });<br><br>    res.end(JSON.stringify(data));<br>  }, 5);<br>});<br><br>server.listen(80, () =&gt; {<br>  console.log(&quot;Server running on port 80&quot;);<br>});</pre><p>To better understand how latency is distributed across the request, I extracted the following values from each request.</p><h4><strong>Key values used in the analysis</strong></h4><p><strong>1. Processing time</strong></p><p>For the Worker:</p><pre>worker_processing_ms: end - start</pre><p>For the server:</p><pre>processing_time_ms: end - start</pre><p>This captures a simulated ~5 ms delay to represent the execution portion of the processing tail.</p><p><strong>2. Location</strong></p><pre>colo: request.cf.colo,<br>country: request.cf.country,<br>city: request.cf.city,</pre><p>This shows the actual edge location (colo) where the request was processed, which may differ from the user’s physical location.</p><p>For the server, I hardcoded the region used:</p><pre>region: &quot;us-east-1&quot;,</pre><p><strong>3. Total request time</strong></p><p>The browser Network tab captures the total request duration. The “Time” column reflects the end-to-end latency, including network transit, protocol overhead, and server processing.</p><p>All latency values in the results represent total request time observed in the browser.</p><h4><strong>Test setup</strong></h4><p>I ran each test three times across three locations (Nigeria, UK, and Canada) for both the centralized server and the edge Worker.</p><p>Below is a screenshot of one of the runs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gfT-LL4xYITIcTmw7-Kyuw.png" /><figcaption>Example DevTools capture showing first-request latency spike and subsequent warm responses.</figcaption></figure><p>The full results across all locations are summarized below.</p><h4><strong>Test results</strong></h4><p><strong>Central Server (EC2):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ij8kP9EXuXg6ARjnUfB4GQ.jpeg" /><figcaption>Latency results for the centralized server (EC2) across three locations (three runs per location).</figcaption></figure><p><strong>Edge Workers (Cloudflare):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QQAOlG79LwC10c-JMyKRTA.jpeg" /><figcaption>Latency results for Cloudflare Workers across three locations, including serving colo and warm request averages.</figcaption></figure><p>With these results and trends, it is easier to map how each system is affected by network paths and how it behaves under them.</p><h3>Experiment interpretation</h3><p>The first thing to notice is how significantly higher the latency (400–770 ms) of the first edge run is compared to the other runs. This is a result of the protocol tax and an initialization phase on the first request. It’s not as if the central server doesn’t pay the protocol tax, but with the edge, the initial connection enters a global network and is routed within it to the serving colo. This adds additional delay on the first request. The central server, on the other hand, terminates at a known, fixed region once. The cold start also adds a small amount of delay during the initial run.</p><p>The subsequent runs show the edge normalizing below 100 ms, apart from the runs in Nigeria. In those Nigerian runs, the requests are not served from a local colo; they are being routed to London. This highlights how the middle mile adds to latency on the edge, even after execution has stabilized.</p><p>The centralized server, on the other hand, shows similar ranges across locations. Since the server is in Virginia (us-east-1), this explains why Canada has lower latency than London and why London has lower latency than Nigeria. The distance the requests have to travel varies.</p><p>London, in particular, shows a more noticeable variation across runs for both the edge and centralized server. This is due to first-mile conditions and network path variability. While edge systems can optimize parts of the middle mile, they cannot remove all routing effects.</p><p>The averages across three runs at all edge locations are higher than those of the centralized server. This, of course, is because of the initial cost on the edge. But subsequent requests show how edge performance improves once the runtime is warm and requests are served closer to the user.</p><p>In all, the test shows that edge systems can achieve the expected 50–100 ms latency, but only when requests are served from nearby colos and after initial connection and runtime overheads have been resolved.</p><p>Here is the takeaway from this experiment</p><ul><li>Protocol tax, along with routing and initialization overhead, drives the latency spike in edge executions.</li><li>Edge performance depends on proximity to the serving colo.</li><li>Centralized servers are consistent, but edge systems outperform them after warm-up.</li></ul><p>One thing the experiment did not account for is the complexity of the code being served. Heavier or more compute-intensive workloads may be better suited to a resource-rich centralized server.</p><p>It would also be unfair to say centralized servers can’t achieve lower latency, though there is a catch.</p><h3>Can centralized systems match edge latency?</h3><p>If a team wants to use Amazon Web Services to match results from Cloudflare, they have to combine multiple services. This introduces an architectural overhead.</p><p>First, they would need to use <a href="https://docs.aws.amazon.com/global-accelerator/latest/dg/what-is-global-accelerator.html">AWS Global Accelerator</a>. It helps pick up user traffic at the nearest AWS edge location and route it over the AWS private backbone to the application.</p><p>To run code closer to users, they need to use <a href="http://aws.amazon.com/lambda/edge/">Lambda@Edge</a> with <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Introduction.html">CloudFront</a>. Unlike a Worker, which runs as an application at the edge, Lambda@Edge is tied to CloudFront and is executed as part of request or response handling. This means the team would have to manage CloudFront distributions, behaviors, and invalidations.</p><p>Even with these components, lower latency depends on how the application is structured. If the core application still runs on EC2 in a single region, propagation delay will still occur unless the user is in the same region as the server.</p><p>To fully replicate edge-like latency, the team would need to adopt a multi-region architecture. They would need to deploy services across multiple regions, set up global databases to replicate data, and configure routing and failover.</p><p>At that point, achieving similar latency becomes a matter of infrastructure design rather than a default capability. It requires additional time, cost, and operational effort.</p><p>Now that the nuances of edge and centralized servers for achieving lower latency are clear, it becomes important to know exactly when to use the edge and when not to.</p><h3>When edge deployments make sense (and when they don’t)</h3><p>After breaking down the different latency taxes and reviewing the experiment results, here’s a practical guide to help decide when edge systems are the right choice for your workload.</p><p><strong>Edge is most useful when:</strong></p><ul><li><strong>Users are globally distributed:</strong> The biggest gains come from reducing middle-mile latency. In the experiment, once the worker was warm, requests from all three locations were significantly faster when served closer to the user.</li><li><strong>Your workloads are lightweight:</strong> Edge runtimes perform best with small, fast operations. The processing tail remains minimal, allowing network advantages to dominate total latency.</li><li><strong>Latency differences matter in practice:</strong> For use cases that depend on fast response times, the edge reduces distance and routing overhead between the user and execution.</li></ul><p><strong>Practical use cases include:</strong></p><ul><li>Global APIs serving users worldwide</li><li>Authentication and authorization endpoints</li><li>Real-time dashboards and status checks</li><li>Interactive apps and A/B testing</li><li>Feature flags, redirects, and lightweight personalization</li><li>Bot protection, rate limiting, and security middleware</li></ul><p><strong>Edge is less ideal when:</strong></p><ul><li><strong>Heavy computation is required:</strong> The processing tail becomes the dominant factor. Tasks such as video transcoding, complex algorithms, or machine learning inference perform better on centralized systems with more resources.</li><li><strong>Large memory workloads are involved:</strong> Working with large in-memory data or file processing is less suitable due to edge runtime constraints.</li><li><strong>Tasks are long-running:</strong> Workloads that require sustained execution are better handled in centralized or regional compute environments.</li></ul><p>If you found this useful, feel free to connect on <a href="https://www.linkedin.com/in/goodness-eboh">LinkedIn</a> and follow me on <a href="https://x.com/busybrainx99">Twitter</a> — I’m always open to tech discussions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=08bb64b9818a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Not Python? A DevOps Take]]></title>
            <link>https://medium.com/@ebohgoodness.e/why-not-python-a-devops-take-07cff5b59dbd?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/07cff5b59dbd</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Fri, 30 May 2025 21:20:39 GMT</pubDate>
            <atom:updated>2025-05-30T21:20:39.169Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QPRZ8uMZ-kg7K6n9_DxFuw.png" /></figure><p>I’d go as far as to say it: <a href="https://www.python.org/">Python</a> is the most versatile language in modern programming.</p><p>That’s why I’m still surprised that some DevOps and cloud engineers underestimate Python, especially in the early stages of their careers.</p><p>Sure, the cloud space screams automation tools like <a href="https://docs.ansible.com/">Ansible</a>, provisioning frameworks like <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html">CloudFormation</a> and <a href="https://developer.hashicorp.com/terraform">Terraform</a>, and pipeline systems like <a href="https://www.jenkins.io/">Jenkins</a>, <a href="https://docs.gitlab.com/ci/">GitLab CI</a>, or <a href="https://github.com/features/actions">GitHub Actions</a>. But the kind of functionality Python unlocks for us can’t be ignored. Think: writing intelligent automation workflows, building serverless applications, handling cloud data at scale, or even experimenting with AI integrations — Python keeps showing up.</p><p>The point isn’t to become a “Python developer.” The point is to build better engineering systems, and Python is one of the best tools for it.</p><p>In this article, we’ll explore how DevOps and cloud engineers can put Python to actual use by solving real problems and laying the foundation for more innovative, more scalable infrastructure.</p><h3>Where does Python fit in DevOps and Cloud?</h3><p>Yes, <a href="https://en.wikipedia.org/wiki/Bash_(Unix_shell)">Bash</a> and <a href="https://learn.microsoft.com/en-us/powershell/scripting/overview?view=powershell-7.5">PowerShell</a> can be used for scripting, but they’re OS-dependent. Python, as a cross-platform programming language, lets you write scripts that work consistently across environments. It’s widely used to build everything from cloud deployment automation and secrets manager integrations to CLI tools, data workflows, and event-driven systems.</p><p>Let’s look at real-world use cases where Python shines in DevOps workflows.</p><h3>1. Automation of repetitive tasks</h3><p>One of Python’s most useful roles in DevOps is automating repetitive tasks. Across the workflow, there are jobs that don’t need to be manual — server setup, network configuration, or cleanup routines. Python’s clean syntax and wide range of libraries make it easy to write scripts that interact with systems and APIs.</p><p>For example, a DevOps engineer can use <a href="https://pypi.org/project/docker-py/">docker-py</a> to clean up unused images, containers, and networks:</p><pre>import docker<br>client = docker.from_env()<br>client.images.prune()<br>client.containers.prune()</pre><h3>2. Infrastructure as code (IaC)</h3><p>Python manages cloud infrastructure directly via SDKs. Tools like <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">boto3</a> for AWS, <a href="https://github.com/googleapis/google-api-python-client">google-api-python-client</a> for GCP, and <a href="https://pypi.org/project/azure-mgmt/">azure-mgmt</a> for Azure allow engineers to interact with cloud services programmatically — spinning up instances, configuring networking, or creating storage.</p><pre>import boto3<br>ec2 = boto3.client(&#39;ec2&#39;)<br>response = ec2.run_instances(<br> ImageId=&#39;ami-0abcdef1234567890&#39;,<br> MinCount=1,<br> MaxCount=1,<br> InstanceType=&#39;t2.micro&#39;<br>)<br>print(response)</pre><p>Python also complements tools like Terraform and Ansible, especially when extending workflows or injecting dynamic logic.</p><h3>3. Container orchestration</h3><p>While <a href="https://kubernetes.io/">Kubernetes</a> workloads are often defined using YAML, Python becomes useful when working with <a href="https://kubernetes.io/docs/concepts/overview/kubernetes-api/">Kubernetes APIs</a> or scripting deployment logic. Using libraries or subprocess calls, engineers can deploy Helm charts, manage namespaces, or scale deployments dynamically.</p><pre>import subprocess<br>def deploy_helm_chart(chart_name, release_name, namespace=&quot;default&quot;):<br> subprocess.run(<br> [&quot;helm&quot;, &quot;upgrade&quot;, &quot; - install&quot;, release_name, chart_name, &quot; - namespace&quot;, namespace],<br> check=True<br> )<br> print(f&quot;Helm chart {chart_name} deployed as {release_name} in {namespace}.&quot;)<br>deploy_helm_chart(&quot;nginx&quot;, &quot;nginx-release&quot;, &quot;default&quot;)</pre><p>This level of control is useful when building internal platform tools or integrating with deployment pipelines.</p><h3>4. CI/CD scripting</h3><p>Python is often used to customize or trigger CI/CD workflows. In tools like Jenkins or GitLab CI, Python scripts can integrate with APIs, trigger builds, upload artifacts, or notify teams. It adds logic where configuration files alone fall short.</p><pre>import requests<br>url = &#39;http://your-jenkins-server/job/your-job-name/build&#39;<br>response = requests.post(url, auth=(&#39;user&#39;, &#39;token&#39;))<br>print(response.status_code)</pre><p>You can build logic that responds to code changes, runs tests, and deploys applications based on specific criteria.</p><h3>5. Monitoring and log analysis</h3><p>Python is useful for collecting and analyzing logs from services, web servers, or cloud platforms. For example, logs from <a href="https://aws.amazon.com/cloudwatch/">AWS CloudWatch</a> can be scanned for error patterns or filtered for troubleshooting.</p><pre>import boto3<br>cloudwatch = boto3.client(&#39;logs&#39;)<br>response = cloudwatch.filter_log_events(<br> logGroupName=&#39;/aws/lambda/my-lambda-function&#39;,<br> filterPattern=&#39;ERROR&#39;,<br> limit=5<br>)<br>for event in response[&#39;events&#39;]:<br> print(f&quot;Error found: {event[&#39;message&#39;]} at {event[&#39;timestamp&#39;]}&quot;)</pre><p>You can take this further by automating alerts or integrating with dashboards.</p><h3>6. ML and AI in cloud workflows</h3><p>Python is a key player in machine learning. That becomes valuable in cloud environments where data is used to predict failures or optimize resources. For example, fetching performance metrics using boto3, processing them with <a href="https://pandas.pydata.org/"><strong>pandas</strong></a>, and feeding the data into models.</p><pre>import boto3<br>import pandas as pd<br>cloudwatch = boto3.client(&#39;cloudwatch&#39;, region_name=&#39;us-west-2&#39;)<br>response = cloudwatch.get_metric_data(<br> MetricDataQueries=[{<br> &#39;Id&#39;: &#39;m1&#39;,<br> &#39;MetricStat&#39;: {<br> &#39;Metric&#39;: {<br> &#39;Namespace&#39;: &#39;AWS/EC2&#39;,<br> &#39;MetricName&#39;: &#39;CPUUtilization&#39;,<br> &#39;Dimensions&#39;: [{&#39;Name&#39;: &#39;InstanceId&#39;, &#39;Value&#39;: &#39;i-1234567890abcdef0&#39;}],<br> },<br> &#39;Period&#39;: 300,<br> &#39;Stat&#39;: &#39;Average&#39;<br> },<br> &#39;ReturnData&#39;: True<br> }],<br> StartTime=&#39;2025–01–01T00:00:00Z&#39;,<br> EndTime=&#39;2025–02–01T00:00:00Z&#39;<br>)<br>cpu_data = pd.DataFrame(response[&#39;MetricDataResults&#39;][0][&#39;Values&#39;], columns=[&#39;CPUUtilization&#39;])<br>cpu_data[&#39;Timestamp&#39;] = pd.to_datetime(response[&#39;MetricDataResults&#39;][0][&#39;Timestamps&#39;])<br>print(cpu_data)</pre><p>Once the data is collected, you can train models to detect patterns and predict failures.</p><pre>from sklearn.ensemble import RandomForestClassifier<br>from sklearn.model_selection import train_test_split<br>from sklearn.metrics import classification_report<br>import pandas as pd<br>data = pd.read_csv(&#39;resource_metrics.csv&#39;)<br>X = data[[&#39;cpu_usage&#39;, &#39;memory_usage&#39;, &#39;disk_io&#39;]]<br>y = data[&#39;failure_occurred&#39;]<br>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)<br>model = RandomForestClassifier()<br>model.fit(X_train, y_train)<br>y_pred = model.predict(X_test)<br>print(classification_report(y_test, y_pred))</pre><p>This kind of integration blends AI into your infrastructure, not as a buzzword, but as part of smarter operations.</p><p>We’ve looked at how Python supports the DevOps and cloud stack at multiple layers. But tools don’t matter in isolation. The real value comes when you tie them to your own path.</p><h3>What do you do with this?</h3><p>We’ve just walked through how Python shows up across DevOps and Cloud. From CI/CD scripting to Kubernetes orchestration, even into AI and ML in cloud environments. These use cases aren’t just theoretical; they’re already in practice.</p><p>What you don’t want to do is jump into learning Python on impulse. It might seem exciting at first, but direction matters more than hype. Take a step back. Look at where you are in your career. Identify the areas where Python would actually make a difference, and start learning with those use cases in mind.</p><p>If this resonated with you, feel free to connect on <a href="https://www.linkedin.com/in/goodness-eboh">LinkedIn</a> and follow me on <a href="https://x.com/busybrainx99">Twitter</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=07cff5b59dbd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What It Takes to Securely Deploy a Microservice Application on Kubernetes Using GCP, Terraform…]]></title>
            <link>https://medium.com/@ebohgoodness.e/what-it-takes-to-securely-deploy-a-microservice-application-on-kubernetes-using-gcp-terraform-afe794348bea?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/afe794348bea</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[argo-cd]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Mon, 31 Mar 2025 20:09:14 GMT</pubDate>
            <atom:updated>2025-03-31T20:09:14.803Z</atom:updated>
            <content:encoded><![CDATA[<h3>What It Takes to Securely Deploy a Microservice Application on Kubernetes Using GCP, Terraform, ArgoCD, and GitHub Actions</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zFL2jWAO3wUhtAbcSnZDug.png" /></figure><p>To improve my understanding of <a href="https://cloud.google.com/docs">Google Cloud Platform (GCP)</a>, I deployed a microservices application using a combination of some powerful tools. I used <a href="https://www.terraform.io/">Terraform</a><strong> </strong>to provision and manage the Kubernetes infrastructure and <a href="https://argo-cd.readthedocs.io/en/stable/">ArgoCD</a> to guarantee continuous monitoring and synchronization of application resources.</p><p>The goal was to build a project focusing on automation and security to prevent the exposure of sensitive credentials. To that effect, I integrated <a href="https://cloud.google.com/security/products/secret-manager?hl=en">Google Secret Manager (GSM)</a> and the <a href="https://external-secrets.io/latest/">External Secrets Operator</a> for secret management. I also added <a href="https://prometheus.io/">Prometheus</a> and <a href="https://grafana.com/">Grafana</a> to the frame to maintain observability and track the system’s performance.</p><p>Check out the full project code on GitHub:<a href="https://github.com/busybrainx99/GCP_Microsvc_Project.git"> GCP Microservices Project</a></p><p>This article will dive deep into what this project entails, highlighting how I successfully created an automated, secure, and resilient deployment. I’ll also discuss the challenges I faced along the way and share troubleshooting tips to help others overcome similar obstacles.</p><h3>Project Architecture</h3><p>The architecture of this project is structured around integrating multiple technologies in a way that each tool is configured to ease the Job of the next. For instance, Terraform provisions an IAM service account on GCP to allow the External Secrets Operator to pull secrets securely from Google Secret Manager (GSM) into the Kubernetes cluster.</p><p>Meanwhile, <a href="https://helm.sh/">Helm</a> packages and manages all the charts, making it easier for ArgoCD to deploy and synchronize them effectively. All this is to ensure automation, continuous synchronization, secure secret management, and real-time monitoring. Let’s break down the key technologies involved:</p><h4>Terraform</h4><p>Despite various IaC tools like <a href="https://www.pulumi.com/">Pulumi</a><strong>, </strong><a href="https://www.crossplane.io/">Crossplane</a>,<strong> </strong>and<strong> </strong><a href="https://www.redhat.com/en/ansible-collaborative">Ansible</a> being available, I chose Terraform because of its rich libraries, flexibility, and the prospect of expanding the project setup. In this project, Terraform handled:</p><ul><li>Provisioning the GKE cluster and node pools.</li><li>Configuring VPCs, subnets, and firewall rules.</li><li>Creating an IAM service account to grant secure access to secrets.</li></ul><p>Terraform’s state and lock files were securely stored in <a href="https://cloud.google.com/storage?utm_source=google&amp;utm_medium=cpc&amp;utm_campaign=emea-pt-all-en-dr-bkws-all-all-trial-e-gcp-1707574&amp;utm_content=text-ad-none-any-dev_c-cre_679779053710-adgp_Hybrid+%7C+BKWS+-+MIX+%7C+Txt+-+Storage+-+Cloud+Storage-kwid_43700078507317071-kwd-11642151515-userloc_9221229&amp;utm_term=kw_google+cloud+storage-net_g-plac_&amp;&amp;gad_source=1&amp;gclid=Cj0KCQjw16O_BhDNARIsAC3i2GDucBBmrYj9HAEdshUyYLCl1qDhAHX4xdbQgi70zSxMSP9dv7P_3QIaAoWnEALw_wcB&amp;gclsrc=aw.ds&amp;hl=en">Google Cloud Storage (GCS)</a> to maintain consistency and collaboration across different environments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vBnXCET8uiiU6jU7" /></figure><h4>Google Kubernetes Engine(GKE)</h4><p><a href="https://cloud.google.com/kubernetes-engine?hl=en">GKE</a> automates tasks like deployment, scaling, and self-healing of workloads while still managing its underlying components like nodes and the control plane. In this project, it housed and managed the microservices application and other resources overseeing:</p><ul><li>Orchestration of containerized workloads.</li><li>Scaling, networking, and load balancing.</li><li>High availability to support consistent application performance.</li></ul><p>None of the services or application components would be deployed or maintained effectively without GKE.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*t9RPZlDctUZU3t3k" /></figure><h4>Google Secret Manager GSM</h4><p>The use of GSM significantly enhanced the security of the project by changing how I initially planned to handle secrets. Adding GSM allowed me to inject secrets securely into the Kubernetes cluster using the External Secrets Operator and the <a href="https://external-secrets.io/v0.4.4/api-clustersecretstore/">ClusterSecretStore</a>. This approach ensured that sensitive data like API keys and passwords remained protected and dynamically accessible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qLc26tYe7GfND1Dk" /></figure><h4>Monitoring Stack</h4><p>To maintain visibility and track the health status of the cluster and deployed resources, I used a combination of Prometheus, Grafana, and <a href="https://prometheus.io/docs/alerting/latest/alertmanager/">Alertmanager</a> to create a robust monitoring and alerting system.</p><h4>Prometheus</h4><p>Prometheus was responsible for scraping metrics from various services and monitoring the health of the cluster and the deployed applications. It collected and stored time-series data, making identifying trends, system behavior, and potential bottlenecks easier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FJUylHqdksP-jMHl" /></figure><h4>Grafana</h4><p>Grafana was configured to visualize the metrics collected by Prometheus. It provided an intuitive interface to analyze performance trends using visually appealing and interactive dashboards. This helped track key indicators and quickly identify any anomalies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BZfADfcVCzPbzE1b" /></figure><h4>Alert-Manager</h4><p>A critical part of this project was setting up effective alerting rules to ensure real-time responses to system changes. I configured Alertmanager with custom alerting rules in Prometheus to monitor Pod and Node health. Alerts were routed to Slack and Email during both uptime and downtime, providing immediate notifications and enabling quick responses to any anomalies.</p><p>The alerting rules ensured that important events, such as service failures or resource exhaustion, triggered timely notifications to keep the system resilient and secure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IubAD8bf4Oa858L7" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FIOXcMiTge2S1Ghd" /></figure><h4>ArgoCD (GitOps for Kubernetes)</h4><p>ArgoCD was pivotal in this project as it managed the continuous delivery of Kubernetes resources. It automatically tracked changes made to the application manifests stored in the GitHub repository and applied those updates to the cluster.</p><p>One of the standout features of ArgoCD is its ability to detect configuration drift. It continuously monitored the desired state defined in version control and compared it with the actual state of the cluster. ArgoCD corrected the state whenever a drift occurred by automatically restoring the cluster to the intended configuration.</p><p>By maintaining a consistent and secure application state, ArgoCD facilitated smooth updates and guaranteed that configuration changes were accurately reflected in the deployed environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6wvTK6p-78s4OfOo" /></figure><h4>GitHub Actions</h4><p>Since the repository was hosted on GitHub, it made sense to build a deployment and cleanup pipeline using <a href="https://github.com/actions">GitHub Actions</a>. One interesting aspect of this tool is that it offers pre-built actions that simplify the complexity of creating CI/CD pipelines. For example, <a href="https://github.com/google-github-actions/setup-gcloud">google-github-actions/setup-gcloud@v2</a> enabled efficient authentication to GCP, allowing resources to be deployed directly to my account without requiring complex setup steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l8rMn6tSmP7LucOT" /></figure><h4>Ingress &amp; NGINX Controller</h4><p>To expose the application to external users and route incoming traffic to the correct services, I deployed the <a href="https://github.com/kubernetes/ingress-nginx">NGINX Ingress Controller</a>. It acted as the gateway, managing HTTP and HTTPS traffic while also supporting load balancing. Without this, the microservices would not be accessible from the internet.</p><h4>Cert-Manager &amp; ClusterIssuer</h4><p>For secure communication and HTTPS traffic, I used <a href="https://cert-manager.io/">Cert-Manager</a> with a ClusterIssuer configured to obtain TLS certificates from <a href="https://letsencrypt.org/">Let’s Encrypt</a>. This automated the certificate lifecycle and maintained secure connections. These certificates protect external communication, preventing unauthorized access and data interception.</p><p>With these technologies, I achieved the goal I aimed for in this project. However, I faced some challenges along the way, and I would like to discuss some of them.</p><h3>Key Challenges faced</h3><h4>1. Configuring Prometheus Scraping and Alerts</h4><p>Writing the alert rules was relatively straightforward, but I encountered two significant challenges:</p><p><strong>a. Getting Prometheus to Scrape the Right Metrics</strong></p><p>This part took considerable time because I had to configure liveness probes for all 14 containers deployed through the manifest. These containers included databases and application instances, each requiring different ports. Since I used a public microservices application, I had no control over the underlying application code and quickly discovered that I was limited to specific exposed metrics.</p><p>I initially considered adding a <a href="https://github.com/prometheus/blackbox_exporter">Blackbox Exporter</a> to monitor HTTP-based endpoints, but the application was not correctly configured to return HTTP status codes. As a result, I focused on tracking Node and Pod uptime using the Node Exporter.</p><p><strong>b. Incorporating Custom Alerting Rules with Prometheus Helm Chart</strong></p><p>Another challenge was integrating custom alerting rules when deploying Prometheus through the official Helm chart. I initially tried to apply a manually created <a href="https://kubernetes.io/docs/concepts/configuration/configmap/">ConfigMap</a>, but Prometheus kept throwing errors. I later discovered that the best way to include custom rules was by specifying them in a prometheus values file.</p><p>I had to carefully map the correct ports in the values file — for instance, port 80 was used for scraping the frontend application, while the database required port 27017. Configuring the service manifests to target the correct containers and ports was also a critical part of this process</p><h4>2. Configuring ArgoCD</h4><p>While ArgoCD is incredibly powerful when fully configured, setting it up, primarily through a CI/CD pipeline, was a different experience. Here are some critical lessons I learned:</p><ul><li><strong>Pre-create Custom Project Names:</strong> If you use a project name other than the default, ensure it’s created before applying ArgoCD manifests.</li><li><strong>Strict Naming Conventions:</strong> ArgoCD does not allow project names to begin with capital letters. This caused initial failures that took some time to debug.</li><li><strong>Correct Field Assignment:</strong> The helm field should not contain the repository link. Instead, the repoURL field should be used.</li><li><strong>Always Add targetRevision:</strong> Ensure the targetRevision field is included to specify which version or branch should be synced.</li></ul><h4>3. Handling Secrets with GSM and External Secrets Operator</h4><p>Importing secrets from Google Secret Manager (GSM) into the Kubernetes cluster posed a unique challenge. I ran into errors until I realized I needed to create a separate <strong>IAM service account</strong> with secret access permissions — distinct from the account used to provision the Terraform resources. The ClusterSecretStore was returning false values, making it difficult to diagnose the issue.</p><p><strong>Lessons Learned:</strong></p><ul><li>Always create a <strong>dedicated service account</strong> with permissions from the Secret Manager Secret Accessor. It’s best to provision this via Terraform to maintain consistency.</li><li>Ensure that the naming convention in GSM matches the key field in the ExternalSecret manifest.</li><li>Generate a key.json formatted key for the secret service account and properly configure it in the ClusterSecretStore to avoid authentication errors.</li></ul><p>Let’s explore some troubleshoot tips that can help out when working on similar projects.</p><h3>Troubleshooting Tips</h3><ol><li><strong>Prometheus Not Scraping Metrics or Delayed Data</strong></li></ol><p>When Prometheus fails to scrape metrics or shows delayed data, misconfigured services or incorrect scrape configs are usually the cause.</p><p><strong>Fix:</strong></p><ul><li>Confirm that ports correctly map to the <strong>/health</strong> endpoints in the service manifest.</li></ul><pre>kubectl describe svc &lt;service-name&gt; -n &lt;namespace&gt;</pre><ul><li>Double-check that liveness and readiness probes are correctly configured for all deployments.</li><li>Validate <strong>scrape_configs</strong> in <strong>your-prometheus-values-file</strong> for target accuracy.</li><li>Port-forward Prometheus and check targets in the UI.</li></ul><pre>kubectl port-forward svc/prometheus-server -n monitoring 9090:80</pre><p>Access<a href="http://localhost:9090/targets"> http://localhost:9090/targets</a> to identify errors.</p><p><strong>2. Prometheus Randomly Restarting or Failing to Start</strong></p><p>If Prometheus restarts unexpectedly or fails to start, a faulty <strong>ConfigMap</strong> with invalid alerting rules or misconfigured scrape configs may be the culprit.</p><p><strong>Fix:</strong></p><ul><li>Inspect logs to diagnose errors.</li></ul><pre>kubectl logs -l app=prometheus-server -n monitoring</pre><ul><li>Retrieve and Edit the ConfigMap: Save the existing ConfigMap to a local file for inspection and correction.</li></ul><pre>kubectl get configmap prometheus-server -n monitoring -o yaml &gt; prometheus-configmap.yml</pre><ul><li>Reapply Corrected Config.</li></ul><pre>kubectl apply -f prometheus-configmap.yml -n monitoring</pre><ul><li>Restart Prometheus Pod.</li></ul><pre>kubectl delete pod -l app=prometheus-server -n monitoring</pre><p><strong>3. ArgoCD Syncing Errors</strong></p><p>When ArgoCD shows an<strong> Unknown</strong> or <strong>OutOfSync</strong> status, it’s usually due to incorrect repo references, namespace mismatches, or misconfigured Application manifests.</p><p><strong>Fix:</strong></p><ul><li>Check that the <strong>repoURL</strong> and <strong>targetRevision</strong> fields in the ArgoCD Application manifest point to the correct GitHub repo and branch.</li><li>Confirm that the namespace defined in the Application manifest matches the one where the resources are deployed.</li><li>Validate that the <strong>argocd</strong> namespace is correctly referenced in all manifests.</li></ul><p><strong>Run:</strong></p><pre>argocd app get &lt;app-name&gt;</pre><p>It provides a summary of sync status repo details.</p><pre>argocd app describe &lt;app-name&gt;</pre><p>It shows a more detailed breakdown, including events, sync history, and errors.</p><p><strong>4. Secret injection Failures with External Secrets</strong></p><p>If secrets are not injected into Kubernetes from Google Secret Manager (GSM), the issue may lie in the configuration of the ClusterSecretStore or secret references.</p><p>Fix:</p><ul><li>Verify that the <strong>ClusterSecretStore</strong> references the correct <strong>key.json</strong> file generated for the <strong>gke-secret-access</strong> service account.</li></ul><pre>kubectl describe clustersecretstore gcp-secret-store</pre><ul><li>Ensure that the IAM service account has the <strong>Secret Manager Secret Accessor</strong> permission:</li></ul><pre>gcloud projects get-iam-policy &lt;GCP_PROJECT_ID&gt; \<br> - flatten=&quot;bindings[].members&quot; \<br> - format=&quot;table(bindings.role, bindings.members)&quot; \<br> | grep gke-secret-access</pre><ul><li>Confirm that the secret names in GSM match the <strong>remoteRef.key</strong> in the ExternalSecret manifest.</li></ul><pre>kubectl describe externalsecret &lt;secret-name&gt;</pre><ul><li>Check if Secrets Exist in GSM:</li></ul><pre>gcloud secrets list - format=&quot;table(name)&quot;</pre><p><strong>5. Errors During Helm Chart Application</strong></p><p>If Helm charts for Prometheus, Cert-Manager, or External Secrets fail to apply:</p><ul><li>Make sure that <strong>helm repo add</strong> and <strong>helm repo update</strong> are run before applying the charts.</li><li>Double-check that <strong>— namespace</strong> and <strong>— create-namespace</strong> flags are included to prevent namespace errors.</li></ul><p><strong>Quick Tip:<br></strong> If using ArgoCD, confirm that the <strong>chart</strong> and <strong>repoURL</strong> fields are correctly defined in the Application manifest.</p><p><strong>6. Alertmanager Failing to Send Alerts</strong></p><p>If Alertmanager is not sending alerts to Slack or Email:</p><ul><li>Check that the <strong>api_url_file</strong> in the <strong>slack_configs</strong> points to the correct Slack webhook.</li><li>Ensure that the SMTP credentials in <strong>email_configs</strong> are valid and the correct ports (e.g., 587) are being used.</li><li>Verify that alerting rules are correctly configured in <strong>your-prometheus-values-file.</strong></li></ul><p>Quick Debug:</p><pre>kubectl logs -l app=alertmanager -n monitoring</pre><p><strong>7. Terraform Fails with State Lock Error</strong></p><p>If Terraform shows a state lock error due to concurrent operations:</p><ul><li>Check the lock file in Google Cloud Storage (GCS) and ensure that no other processes are running.</li><li>Manually remove the lock if needed using:</li></ul><pre>gsutil rm gs://&lt;your-bucket&gt;/terraform/state/default.tflock</pre><p><strong>8. Kubernetes Service Not Reaching the Correct Pods</strong></p><p>If the Ingress or Service is not routing traffic properly:</p><ul><li>Verify that the <strong>Ingress</strong> manifest targets the correct service name and port.</li><li>Check the <strong>Service</strong> type (ClusterIP, NodePort, or LoadBalancer) and ensure that the appropriate type is used.</li><li>Confirm that DNS entries or <strong>hosts</strong> in the Ingress manifest match the target application’s domain.</li></ul><p><strong>Check:</strong></p><pre>kubectl describe svc &lt;service-name&gt;<br>kubectl describe ingress &lt;ingress-name&gt;</pre><h3>Final Thoughts</h3><p>This project was a comprehensive deep dive into automating microservices deployment on Google Cloud Platform (GCP). I built a secure, scalable, and resilient application pipeline by combining Terraform for infrastructure management, ArgoCD for continuous delivery, and Prometheus/Grafana for monitoring.</p><p>A significant part of the learning process was troubleshooting complex configurations in Prometheus and fine-tuning ArgoCD manifests to maintain a consistent application state. Integrating Google Secret Manager (GSM) and the External Secrets Operator reinforced the importance of managing secrets securely in production environments.</p><p>In the future, I plan to explore multi-cloud integration and extend this setup to manage hybrid workloads more effectively.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afe794348bea" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Kubernetes? An Introduction to Container Orchestration.]]></title>
            <link>https://medium.com/@ebohgoodness.e/what-is-kubernetes-an-introduction-to-container-orchestration-ba72aea47eb8?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba72aea47eb8</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[cloud-native]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Thu, 06 Feb 2025 18:46:57 GMT</pubDate>
            <atom:updated>2025-02-06T18:46:57.712Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QRMJL5miQddRFMz4K96fcw.png" /></figure><p>The cloud-native era didn’t appear overnight; it has been a steady evolution of the technology we see today. From the rise of cloud computing technology in the early 2000s to the early experimentation with containerization in the mid-2000s, this transformation has been anything but sudden.</p><p>Over time, this golden era introduced <a href="https://aws.amazon.com/microservices/"><strong>microservice architectures</strong></a> driven by the power of cloud computing and containerization, whose adoption was cemented with the advent of <a href="https://www.docker.com/"><strong>Docker</strong></a>. These innovations laid the foundation for the modern cloud ecosystem. But with these groundbreaking advancements came a new challenge: How do we efficiently manage and scale this growing complexity?</p><p>The advent of <a href="https://kubernetes.io/"><strong>Kubernetes</strong></a> was inevitable as there was a need to create an intelligent system that could properly manage the powerhouse created by microservices and containers. It didn’t just optimize cloud-native technologies; it unified them. It brought order to chaos, cementing orchestration as a fundamental pillar of cloud-native computing.</p><p>Despite the rapid advancements in cloud technology, Kubernetes remains the backbone of modern cloud infrastructure, steering the future of application deployment and scalability. This article explores Kubernetes’ technical prowess, efficiency, and role in shaping modern technology.</p><h3>Why Use Kubernetes?</h3><p>Several container orchestration technologies competed for the spotlight around the same time Kubernetes emerged. But Kubernetes had a distinct advantage — it wasn’t built from scratch like many of its competitors. Instead, it was inspired by <a href="https://kubernetes.io/blog/2015/04/borg-predecessor-to-kubernetes/"><strong>Borg</strong></a>, an internal system used by Google to manage large-scale distributed computing jobs. With years of real-world experience behind it, Google engineers designed Kubernetes with a deep understanding of what was truly needed to orchestrate containers at scale.</p><p>Before Kubernetes, managing containers was a nightmare — a manual, time-consuming, and error-prone process:</p><ul><li>Scaling and deploying containers had to be done manually.</li><li>Load balancing required engineers to configure and manage load balancers for each service.</li><li>Service discovery and networking were complex — IP addresses and service endpoints had to be manually assigned for containers to communicate.</li><li>Resource allocation and monitoring were slow and inefficient, making it difficult to optimize performance.</li><li>Self-healing didn’t exist — engineers had to restart or replace containers manually when containers failed.</li><li>As workloads grew, operational overhead spiraled out of control, making containerized applications harder to manage at scale.</li></ul><p>These are some of the luxuries we take for granted today — things that <a href="https://kubernetes.io/docs/concepts/overview/"><strong>Kubernetes automates and standardizes</strong></a>, making container orchestration feel effortless.</p><p>Without automation, deploying, scaling, and recovering containers would still be a manual and time-consuming process. Cloud-native applications wouldn’t have reached the scale they operate at today.</p><p>To truly understand the power Kubernetes wields, we need to dissect its architecture. Only then can we answer the question:</p><p>What makes Kubernetes so efficient at orchestrating thousands of containers seamlessly?</p><h3>Understanding Kubernetes Cluster Architecture</h3><p>The structure of Kubernetes may seem complex, but we can simplify it by understanding its two core components:</p><ol><li>The Control Plane (Master Node) — Manages the cluster, makes scheduling decisions, and maintains overall system health.</li><li>Worker Node(s) — Where application workloads (containers) run.</li></ol><p>Together, these components and their subcomponents form what is known as a <a href="https://kubernetes.io/docs/concepts/architecture/"><strong>Kubernetes cluster</strong></a>. While the control plane runs the core components that orchestrate the cluster, the worker nodes execute the workloads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*scoG93EDEu2USa9Q" /></figure><h4>Kubernetes Worker Node Breakdown</h4><p>You can think of the worker node as the hands and legs of a human body — receiving instructions from the control plane (the brain) and executing actions accordingly.</p><p>The components of a worker node include:</p><ul><li>Pod — The smallest unit in a Kubernetes cluster. Each pod houses one or more containers, but Kubernetes only recognizes pods — not individual containers.</li><li>Kubelet — It is the agent running on each worker node enabling communication between the control plane and the container runtime. It ensures that the correct containers are running inside pods.</li><li>Container Runtime — The underlying software responsible for pulling container images from registries, unpacking them, and running them. While <a href="https://kubernetes.io/docs/setup/production-environment/container-runtimes/"><strong>Kubernetes supports different runtimes</strong></a>, Docker remains the most widely used.</li><li>Kube-proxy — A network proxy that runs on each node, managing network rules to enable pod-to-pod communication inside the cluster and external access when needed.</li></ul><h4>Kubernetes Control Plane Breakdown</h4><p>If the worker nodes are like the hands and legs, the control plane is the brain — coordinating, monitoring, and making scheduling decisions. Just like different parts of the human brain handle specific functions, the control plane consists of multiple components, each playing a critical role.</p><ul><li>Kube-apiserver — Acts as the entry point for all Kubernetes operations. It exposes the Kubernetes API, handling communication between internal Kubernetes components and external tools like kubectl and cloud controllers.</li><li>Kube-scheduler — Assigns pods to worker nodes. It assesses resource requirements (CPU, memory, etc.) and places pods on the most suitable nodes.</li><li>Kube-controller-manager — Runs various controllers responsible for self-healing, scaling, and managing cluster state. It ensures that pods scale properly, services update correctly, and storage is attached when needed.</li><li>etcd — A distributed key-value store that serves as the single source of truth for the cluster. It stores all Kubernetes configurations, resource states, and metadata.</li><li><a href="https://kubernetes.io/docs/concepts/architecture/cloud-controller/"><strong>Cloud-controller-manager</strong></a> — Enables Kubernetes to integrate with cloud providers like <a href="https://aws.amazon.com/kubernetes/"><strong>AWS</strong></a>, <a href="https://cloud.google.com/learn/what-is-kubernetes?hl=en"><strong>GCP</strong></a>, and <a href="https://azure.microsoft.com/en-us/solutions/kubernetes-on-azure/get-started"><strong>Azure</strong></a>. It separates cloud-specific operations from Kubernetes’ core logic, determining which cluster components interact with cloud resources.</li></ul><p>With these various components working together, Kubernetes delivers scalability, self-healing, automated deployments, and resource monitoring — features that have made it the cornerstone of modern cloud-native applications.</p><p>While these components make Kubernetes powerful, they also introduce the challenge of managing and maintaining them. Balancing infrastructure management with actual application deployment can quickly become overwhelming.</p><h3>Self-Managed vs. Managed Kubernetes Services</h3><p>Running a self-managed Kubernetes cluster means taking on the control plane’s full responsibility — provisioning, maintaining, and securing its components, ensuring high availability, and managing upgrades. While this approach gives you control over networking, scaling, and security configurations, it comes at the expense of your time and expertise. As these clusters grow over time, so does the effort, expertise, and time required to keep everything running smoothly, often leading to significant operational overhead.</p><p>To solve this problem, early Kubernetes contributors and open-source communities built tools like <a href="https://kops.sigs.k8s.io/"><strong>KOPS</strong></a><strong>, </strong><a href="https://github.com/kubernetes-sigs/kubespray"><strong>Kubespray</strong></a><strong>, </strong><a href="https://velero.io/"><strong>Heptio Ark( now Velero)</strong></a><strong>, and </strong><a href="https://www.rancher.com/"><strong>Rancher Kubernetes Engine (RKE)</strong> </a>to simplify Kubernetes management. These solutions automated cluster provisioning, networking, and control plane operations, reducing the complexity of running Kubernetes at scale.</p><p>Cloud providers later adopted and scaled these ideas, introducing fully managed services solutions that eliminated the need for users to manage the control plane themselves. Among these solutions, one stands out as an efficient, scalable, and fully managed cloud version of Kubernetes — <a href="https://docs.aws.amazon.com/whitepapers/latest/overview-deployment-options/amazon-elastic-kubernetes-service.html"><strong>Amazon Elastic Kubernetes Service (EKS</strong>)</a>.</p><h3>Why Choose Amazon EKS Over Self-Managed Kubernetes?</h3><p>One of the biggest advantages that EKS brings to the table is control plane management. While setting up self-managed Kubernetes clusters gives teams full control over security and customization, it also introduces significant operational complexity and costs — both in terms of financial investment and manpower required to oversee its components.</p><p>With EKS, AWS takes care of the control plane for you. Giving you advantages such as:</p><ul><li>High availability and fault tolerance — Ensuring the control plane is always up and running across multiple availability zones.</li><li>Security and networking — Allowing Seamless integration with <a href="https://docs.aws.amazon.com/eks/latest/userguide/security-iam-service-with-iam.html"><strong>AWS IAM</strong></a>, <a href="https://docs.aws.amazon.com/eks/latest/userguide/creating-a-vpc.html"><strong>VPC</strong></a>, and <a href="https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html"><strong>security groups</strong></a><strong> </strong>for enhanced security.</li><li>Monitoring and logging — Built-in CloudWatch and AWS-native monitoring for better visibility.</li><li>Automatic upgrades and patching — Keeping Kubernetes versions updated without downtime.</li></ul><p>But what makes EKS flexible? While AWS handles the control plane, you still have complete control over your worker nodes, allowing you to:</p><ul><li>Deploy and manage workloads exactly as you would in a self-managed cluster.</li><li>Optimize your infrastructure with EC2 instances or use serverless Kubernetes with <a href="https://docs.aws.amazon.com/eks/latest/userguide/fargate-getting-started.html"><strong>AWS Fargate</strong></a>.</li><li>Configure autoscaling for better resource efficiency.</li></ul><p>With EKS, the control plane operates just like a self-managed Kubernetes cluster, except you don’t have to manage it. You get all the benefits without the hassle of maintaining uptime, securing APIs, and ensuring smooth communication between components.</p><p>Of course, EKS comes with a cost, but compared to the time and engineering effort required to manage Kubernetes manually, it’s a worthy trade-off. It’s not just about saving money — it’s about saving time and operational burden.</p><p>Instead of spending weeks hiring engineers, setting up infrastructure, and fine-tuning a Kubernetes cluster, you can deploy an EKS cluster within minutes — AWS even provides comprehensive guides to help teams <a href="https://aws.amazon.com/eks/"><strong>get started</strong></a>.</p><h3>Conclusion</h3><p>Kubernetes began as a bold idea from Joe Beda, Brendan Burns, and Craig McLuckie — a vision that reshaped modern cloud computing. They pushed forward until their voices were heard and their innovation welcomed. Kubernetes evolved even further under the <a href="https://www.cncf.io/"><strong>Cloud Native Computing Foundation (CNCF)</strong></a>, which now oversees its growth along with a rich ecosystem of projects designed to extend its capabilities.</p><p>Today, Kubernetes is more than just an orchestration tool — it’s an open-source empire filled with innovation, collaboration, and groundbreaking ideas. It continues to be the brightest star in the cloud-native empire, shaping the future of application deployment and scalability.</p><p>As Kubernetes adoption grew, managing clusters became a challenge. This is where managed services like AWS EKS came in, allowing teams to harness the full power of Kubernetes while benefiting from managed infrastructure, security, and flexibility. It provides a seamless way to run Kubernetes without the operational burden, ensuring that developers can focus on what truly matters — their applications.</p><p>Kubernetes is already here, but its potential continues to define the future of cloud-native technology.</p><h3>Further Reading</h3><p>Here are some additional resources to expand your knowledge on Kubernetes and Amazon EKS:</p><ul><li><a href="https://www.cncf.io/blog/2017/11/07/joe-beda-explains-inner-workings-kubernetes/"><strong>The Inner Workings of Kubernetes by Joe Beda</strong></a></li><li><a href="https://kubernetes.io/docs/concepts/configuration/overview/"><strong>Kubernetes Config Best Practices</strong></a>,</li><li><a href="https://docs.aws.amazon.com/eks/latest/best-practices/introduction.html"><strong>AWS EKS Best Practices</strong></a>,</li><li><a href="https://www.cncf.io/reports/"><strong>CNCF Reports</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba72aea47eb8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DevOps Engineering Specializations: The Financial Challenges You’ll Face]]></title>
            <link>https://medium.com/@ebohgoodness.e/devops-engineering-specializations-the-financial-challenges-youll-face-151bdad1b3b5?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/151bdad1b3b5</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[career-development]]></category>
            <category><![CDATA[cost-optimization]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Thu, 30 Jan 2025 15:11:03 GMT</pubDate>
            <atom:updated>2025-01-30T15:51:21.033Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vHHlxTq3x946n6ZIJOdXNQ.png" /></figure><p><strong>Why is learning to be a DevOps engineer so expensive?</strong> I recently came across this question on <a href="https://www.reddit.com/r/devops/comments/1hqyfsa/why_is_learning_to_be_a_devops_engineer_so/"><strong>Reddit</strong></a>, and it stuck with me. The thread had over 70 comments, with people sharing their thoughts and experiences. One common debate was whether DevOps is a single job title or a broad discipline encompassing multiple specializations. Another major discussion point was the high cost of practicing and testing DevOps.</p><p>Beginners often struggle because they incur unexpected expenses when experimenting with infrastructure. Meanwhile, experienced engineers and teams face significant costs when running tests.</p><p>Reading through these comments, I had a few realizations. DevOps is indeed a broad field with multiple areas of specialization. It’s not a single role but rather a collection of roles and responsibilities. However, regardless of the specialization, the issue of cost — whether for learning, practicing, or testing — remains a major challenge.</p><p>I’ve been there, too, unexpectedly incurring costs while trying to practice or carry out deployments.</p><p>In a bid to contribute to the topic, I decided my thoughts on this require more than a Reddit comment. So, let’s explore the various areas of specialization within DevOps and examine the cost challenges associated with each, highlighting the financial hurdles at different stages of the journey.</p><h3>Areas of Specialization in DevOps and Associated Cost</h3><p>I like to think of DevOps as the dog family. Just as various breeds have differences in behavior, size, and appearance, they all share certain characteristics that define them as dogs. Similarly, DevOps engineering encompasses multiple areas of specialization, each involving slightly distinct activities, tools, and technologies. Yet, they all share common principles and practices that unify them under the broad name of DevOps.</p><p>Each specialization contributes uniquely to the larger DevOps ecosystem, addressing different aspects of the software development lifecycle. In the following sections, we’ll delve into these specializations, starting with a closer look at their roles and responsibilities, as well as the financial challenges they often entail.</p><h3>Release Management</h3><p>Release Management involves the deployment of software updates and the management of the entire release lifecycle. Its primary goal is to ensure that every version of an application can be smoothly deployed and managed across different environments. This process spans various stages of development, bridging the gap between development and operations teams. These stages include <strong>planning</strong>, <strong>building</strong>, <strong>testing</strong>, <strong>preparing</strong>, and <strong>deployment</strong>.</p><p><strong>Below is a visual representation of the release lifecycle stages, illustrating how each phase connects seamlessly</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kUJ5cUsaLJ8Viuaq" /></figure><p>Release Management operates as a continuous cycle, with each new version passing through these stages. The stages work together as part of a continuous process, where each step sets the foundation for the next. When executed properly, this approach helps teams deploy software efficiently while reducing the chances of errors or delays.</p><p>Key activities in release management include:</p><ul><li><strong>Planning release schedules:</strong> Defining timelines and goals for each release.</li><li><strong>Coordinating between development and operations teams:</strong> Ensuring seamless collaboration and communication.</li><li><strong>Creating and managing pipelines:</strong> Automating workflows for builds, tests, and deployments.</li></ul><h3>Cost Incurred Within Release Management</h3><p>Release management is critical to deploying updates smoothly across environments, but it often comes with financial challenges:</p><p><strong>For Beginners</strong></p><p>Costs can quickly add up from exceeding the free-tier limits of tools like <a href="https://circleci.com/"><strong>CircleCI</strong></a> or spinning up <a href="https://kubernetes.io/"><strong>Kubernetes clusters</strong></a> on cloud platforms for testing purposes.</p><p><strong>For experienced engineers and enterprises</strong></p><p>The need for advanced features and flexibility often drives the use of more robust tools to support large teams and complex workflows. Tools like <a href="https://learn.microsoft.com/en-us/azure/devops/user-guide/what-is-azure-devops?view=azure-devops&amp;toc=%2Fazure%2Fdevops%2Fget-started%2Ftoc.json"><strong>Azure DevOps</strong></a>, <a href="https://docs.github.com/en/enterprise-cloud@latest"><strong>GitHub Enterprise</strong></a>, and <a href="https://www.atlassian.com/software/bamboo"><strong>Atlassian Bamboo</strong></a> provide powerful capabilities but come with significant costs, especially as usage scales.</p><h3>Quality Assurance (QA) and Testing</h3><p>While release management and QA may share CI/CD processes as an integral part of their activities, they are distinct in focus. QA and testing prioritize the quality of software, integrating various testing methodologies such as <a href="https://www.browserstack.com/guide/what-is-shift-left-testing"><strong>shift-left testing</strong></a>,<strong> </strong><a href="https://www.accelq.com/blog/qa-automation/"><strong>test automation</strong></a>, <a href="https://www.qatouch.com/blog/continuous-testing/"><strong>continuous testing</strong></a><strong>,</strong> and <a href="https://www.devlane.com/blog/performance-testing-in-quality-assurance-learn-the-basics"><strong>performance testing</strong></a> throughout the development lifecycle.</p><p>The primary aim of QA is to ensure faster feedback loops during development, identify errors early, and, where possible, correct issues within the codebase before deployment.</p><p>Depending on the type of codebase, QA engineers utilize tools such as <a href="https://www.selenium.dev/"><strong>Selenium</strong></a>, <a href="https://www.cypress.io/"><strong>Cypress</strong></a>,<a href="https://docs.pytest.org/en/stable/"><strong>Pytest</strong></a>, <a href="https://www.opentext.com/products/enterprise-performance-engineering"><strong>LoadRunner</strong></a>, <a href="https://www.jenkins.io/"><strong>Jenkins</strong></a>, and <a href="https://www.sonarsource.com/products/sonarqube/"><strong>SonarQube</strong></a> for various activities like;</p><ul><li><strong>Test Planning and Design:</strong> Defining the testing strategy, identifying test cases, and setting quality benchmarks.</li><li><strong>Automated Testing:</strong> Creating and executing automated test scripts to validate functionality and performance.</li><li><strong>Manual Testing:</strong> Performing exploratory or edge-case testing where automation isn’t feasible.</li><li><strong>Continuous Testing:</strong> Running tests in CI/CD pipelines to catch issues early in the development lifecycle.</li><li><strong>Performance Testing:</strong> Evaluating application performance under varying loads using tools like LoadRunner and JMeter.</li><li><strong>Bug Reporting and Tracking:</strong> Documenting issues and collaborating with development teams to resolve them.</li></ul><h3>Cost Incurred Within QA and Testing</h3><p><strong>For Beginners</strong></p><p>Beginners typically rely on foundational tools like <strong>Selenium</strong>, <a href="https://junit.org/junit5/"><strong>JUnit</strong></a>, <strong>Cypress</strong>, and <a href="https://www.postman.com/"><strong>Postman</strong></a> to learn automation and testing concepts without incurring significant costs. However, these initial savings often diminish due to:</p><ul><li><strong>Limited Free-Tier Features:</strong> Tools like <a href="https://www.selenium.dev/documentation/grid/"><strong>Selenium Grid</strong></a> may require cloud infrastructure for parallel testing, leading to additional expenses.</li><li><strong>Cloud Infrastructure Costs:</strong> Testing automation for heavy applications may exceed the capabilities of local hardware, necessitating cloud instances or services.</li></ul><p><strong>For Experienced Engineers and Enterprises</strong></p><p>Experienced engineers and enterprises adopt advanced and scalable tools to manage complex workflows and robust testing frameworks. These tools include<a href="https://testcafe.io/"> <strong>TestCafe</strong></a>, <a href="https://www.testrail.com/"><strong>TestRail</strong></a>,<a href="https://jmeter.apache.org/"> <strong>JMeter</strong></a>, <strong>LoadRunner</strong>, and<a href="https://www.browserstack.com/"> <strong>BrowserStack</strong></a>. Costs can escalate due to:</p><ul><li><strong>Licensing Fees:</strong> Tools like <strong>LoadRunner</strong> require annual fees that increase with team size and usage.</li><li><strong>Parallel Testing Costs:</strong> Running tests across multiple browsers or devices often results in high expenses for tools like <strong>BrowserStack</strong>.</li><li><strong>Infrastructure Needs:</strong> Enterprises may incur significant costs to maintain production-like staging environments and scale testing infrastructure.</li></ul><h3>Site Reliability Engineering (SRE)</h3><p>Site Reliability Engineering (SRE) is a specialization within DevOps that integrates software engineering principles with IT infrastructure and operations tasks to ensure system reliability, scalability, and efficiency.</p><p>In practice, SRE encompasses a range of critical activities, including:</p><ul><li><strong>Monitoring and Optimization:</strong> Continuously tracking system performance and improving resource utilization.</li><li><strong>Capacity Planning:</strong> Predicting and preparing for scaling demands to meet growing application needs.</li><li><strong>Disaster Recovery:</strong> Designing failover systems and strategies to ensure rapid recovery during unexpected incidents.</li><li><strong>Automation:</strong> Streamlining infrastructure provisioning, deployment, and operational workflows using code.</li></ul><p>Engineers in this field leverage tools and platforms such as <a href="https://kubernetes.io/docs/concepts/overview/#why-you-need-kubernetes-and-what-can-it-do"><strong>Kubernetes</strong></a>, <a href="https://www.terraform.io/"><strong>Terraform</strong></a>, <a href="https://www.redhat.com/en/ansible-collaborative?intcmp=7015Y000003t7aWQAQ"><strong>Ansible</strong></a>, <strong>cloud environments (e.g., </strong><a href="https://aws.amazon.com/"><strong>AWS</strong></a><strong>, </strong><a href="https://cloud.google.com/"><strong>GCP</strong></a><strong>, </strong><a href="https://azure.microsoft.com/en-us/"><strong>Azure</strong></a><strong>)</strong>, and <a href="https://www.datadoghq.com/"><strong>Datadog</strong></a> to carry out these activities effectively.</p><p><strong>For Beginners</strong></p><p>When using cloud-based infrastructure like <strong>Kubernetes</strong> or virtual machines, beginners often face costs, as local tools like <a href="https://kubernetes.io/docs/tutorials/hello-minikube/"><strong>Minikube</strong></a> can strain hardware. Advanced tools like <strong>Datadog</strong>, <a href="https://newrelic.com/"><strong>New Relic</strong></a>, and <a href="https://www.pagerduty.com/"><strong>PagerDuty</strong></a> offer limited free-tier features, with costs scaling for additional functionality like advanced monitoring or incident management.</p><p><strong>For Advanced Engineers and Enterprises</strong></p><p>Replicating production environments incurs higher costs for enterprises and experienced engineers, using tools like <strong>Datadog</strong>, <a href="https://aws.amazon.com/cloudwatch/"><strong>AWS CloudWatch</strong></a>, and <a href="https://www.gremlin.com/"><strong>Gremlin</strong></a> for monitoring and resilience testing. Platforms like <a href="https://www.servicenow.com/products/it-operations-management/what-is-site-reliability-engineering.html"><strong>ServiceNow</strong></a> add further costs for large teams and advanced <a href="https://www.servicenow.com/products/itsm/what-is-itsm.html"><strong>ITSM</strong></a> features.</p><h3>DevSecOps Engineering</h3><p>DevSecOps integrates security practices into the software development lifecycle, from the initial design stages to deployment and beyond. The term <strong>DevSecOps</strong> stems from combining <strong>Development (Dev)</strong>, <strong>Security (Sec)</strong>, and <strong>Operations (Ops)</strong>, emphasizing the incorporation of security into every aspect of development and operations.</p><h4>Key Activities of DevSecOps Engineers</h4><p>DevSecOps professionals focus on ensuring that systems are secure throughout their lifecycle. Their responsibilities typically include:</p><ul><li><strong>Integrating Security into Software Design:</strong> Embedding security principles into applications’ architecture and coding standards.</li><li><strong>Identifying and Evaluating Threats:</strong> Detecting and analyzing potential vulnerabilities and risks in software and infrastructure.</li><li><strong>Implementing Automated Security Checks:</strong> Incorporating security tests into CI/CD pipelines to identify issues early in development.</li><li><strong>Providing Training and Guidance:</strong> Educating teams on security best practices to prevent vulnerabilities during development.</li><li><strong>Preparing for and Responding to Incidents:</strong> Design processes to effectively address security breaches, malware infections, insider threats, phishing attacks, and other potential incidents.</li></ul><h3>Cost Incurred Within DevSecOps</h3><p><strong>For Beginners</strong></p><p>Beginners often rely on accessible, free, or open-source tools to learn and implement foundational <a href="https://www.checkpoint.com/cyber-hub/cloud-security/devsecops/7-devsecops-best-practices/"><strong>security practices</strong></a>. Common tools include <a href="https://www.zaproxy.org/"><strong>OWASP ZAP</strong></a>, <a href="https://www.sonarsource.com/products/sonarqube/"><strong>SonarQube</strong></a>, <a href="https://github.com/quay/clair"><strong>Clair</strong></a>,<a href="https://github.com/docker/docker-bench-security"> <strong>Docker Bench for Security</strong></a>, and <a href="https://snyk.io/"><strong>Snyk</strong></a>. While many of these tools are free, limitations in free-tier features and the need to spin up cloud infrastructures for testing can result in unexpected costs.</p><p><strong>For Experienced Engineers and Enterprises</strong></p><p>Advanced tools like <a href="https://checkmarx.com/"><strong>Checkmarx</strong></a>, <a href="https://www.aquasec.com/"><strong>Aqua Security</strong></a>, <a href="https://www.veracode.com/"><strong>Veracode</strong></a>, and <a href="https://www.splunk.com/en_us/products/splunk-security-orchestration-and-automation.html"><strong>Splunk Soar</strong></a> are essential for securing complex systems. However, these tools come with significant licensing and operational costs, especially when scaling across large infrastructures or integrating into enterprise workflows.</p><h3>CloudOps Engineering</h3><p>CloudOps is an area of DevOps that oversees the provisioning, automation, monitoring, scaling, management, and optimization of cloud-based infrastructures, applications, and services. It requires expertise in various cloud technologies and adherence to industry best practices to build, deploy, and automate software efficiently.</p><h4>Key Responsibilities of CloudOps Engineers</h4><p>Engineers in this specialization are tasked with:</p><ul><li><strong>Cloud Infrastructure Deployment and Management:</strong> Setting up and maintaining resources such as virtual machines, storage, and networks.</li><li><strong>Cloud Migration:</strong> Moving on-premises applications and data to cloud platforms like <strong>AWS</strong>, <strong>GCP</strong>, <strong>Azure</strong>, and <a href="https://www.digitalocean.com/"><strong>DigitalOcean</strong></a>.</li><li><strong>Automation:</strong> Using tools such as <a href="https://www.terraform.io/"><strong>Terraform</strong></a> and <a href="https://www.redhat.com/en/ansible-collaborative"><strong>Ansible</strong></a> to simplify the provisioning, configuration, and management of cloud resources.</li><li><strong>Cost Optimization:</strong> Monitoring and controlling cloud usage to minimize expenses.</li><li><strong>Configuration Management:</strong> Ensuring compliance and consistency across cloud environments.</li><li><strong>Containerization and Orchestration:</strong> Leveraging tools like <a href="https://hub.docker.com/"><strong>Docker</strong></a> for packaging applications into portable containers and <strong>Kubernetes</strong> for managing the deployment, scaling, and orchestration of these containerized applications.</li></ul><h3>Cost Incurred Within CloudOps</h3><p><strong>For Beginners</strong></p><p>Beginners often rely on free-tier and open-source tools like <strong>Terraform</strong>, <strong>Docker</strong>, and free cloud credits from platforms like <strong>GCP</strong>, <strong>Azure</strong> and <strong>AWS</strong> to learn and practice. While these options help minimize costs, challenges arise from:</p><ul><li><strong>Spinning Up Larger Instances:</strong> Testing more resource-intensive setups can exceed free-tier limits.</li><li><strong>Data Transfers:</strong> Transferring data within or across cloud regions incurs additional charges.</li><li><strong>Forgotten Resources:</strong> Leaving instances or storage running unintentionally can quickly accumulate costs.</li></ul><p><strong>For Experienced Engineers and Enterprises</strong></p><p>Evading costs becomes much harder at advanced levels, as production-grade environments demand robust tools and infrastructure. Cost challenges include:</p><ul><li><strong>Advanced Cloud Resources:</strong> Running high-performance instances or multi-region deployments on <strong>AWS</strong>, <strong>Azure</strong>, or similar platforms.</li><li><strong>High Availability and Disaster Recovery:</strong> Ensuring uptime and resilience requires redundant infrastructure, which doubles costs.</li><li><strong>CI/CD Pipelines:</strong> Running pipelines at scale involves compute, storage, and data transfer costs, especially in enterprise workflows.</li></ul><h3>NETOPS</h3><p>NetOps, short for <strong>Network Operations</strong>, is a specialization in DevOps that focuses on automating, provisioning, and orchestrating network infrastructure to ensure seamless communication between services within an application. It brings DevOps principles and tools to traditional networking, enabling more efficient setup, management, and maintenance of network configurations.</p><h4>What NetOps Involves</h4><p>NetOps enhances networking by applying practices like <strong>automation</strong>, <strong>monitoring</strong>, and <strong>orchestration</strong> to reduce manual effort and improve reliability. These practices ensure that networks are resilient, scalable, and capable of supporting modern application architectures.</p><h4>Common Tools Used in NetOps</h4><p>While NetOps utilizes some networking-specific tools, it also leverages widely used DevOps tools for broader tasks:</p><ul><li><strong>Configuration Management:</strong> <strong>Ansible</strong>, <a href="https://www.puppet.com/"><strong>Puppet</strong></a> for automating network configurations.</li><li><strong>Monitoring and Management:</strong> <a href="https://www.solarwinds.com/"><strong>SolarWinds</strong></a>, <strong>Datadog</strong>, and <a href="https://prometheus.io/"><strong>Prometheus</strong></a> for monitoring network performance and identifying issues.</li><li><strong>Incident Response and Reporting:</strong> <strong>ServiceNow</strong> for managing incidents and generating reports.</li><li><strong>Network Automation:</strong> <a href="https://www.cisco.com/site/us/en/products/networking/catalyst-center/index.html"><strong>Cisco DNA Center</strong></a> or <a href="https://www.juniper.net/us/en/products/sdn-and-orchestration/contrail.html"><strong>Juniper Contrail</strong></a> for automating complex networking workflows.</li></ul><h3>Cost Incurred Within NetOps</h3><p><strong>For Beginners</strong></p><p>Similar to other DevOps specializations, beginners lean towards free-tier tools like <strong>AWS</strong>, <strong>GCP</strong>, and open-source options like <strong>Ansible</strong> and <strong>Prometheus</strong>. However, challenges arise when:</p><ul><li><strong>Scaling Beyond Free Tiers:</strong> Accessing advanced features or larger capacities often incurs additional costs.</li><li><strong>Local Hardware Limitations:</strong> Resource-heavy tasks may require cloud-based infrastructure, leading to cloud expenses.</li></ul><p><strong>For Experienced Engineers and Enterprise</strong></p><p>Experienced engineers and enterprises require tools capable of managing complex, dynamic network environments. These environments often involve scaling to support multi-region deployments and handling applications with multiple interconnected services. Advanced tools like <strong>SolarWinds</strong>, <a href="https://netboxlabs.com/oss/netbox/"><strong>NetBox</strong></a>, and <a href="https://www.paloaltonetworks.com/"><strong>Palo Alto Networks</strong></a> are commonly used to meet these demands.</p><p>However, the cost of using these tools can be significant due to:</p><ul><li><strong>Licensing Fees:</strong> Premium tools often come with high subscription or licensing costs.</li><li><strong>Operational Costs:</strong> Scaling networks and ensuring high availability and security across regions requires substantial investment in infrastructure and monitoring.</li></ul><h3>DataOps</h3><p>Data Operations entails designing solutions to automate data management processes, reducing the time required to derive valuable insights from application or business data while ensuring data quality. DataOps engineers focus on building pipelines to extract, transform, and load (ETL) data, making it accessible for analysis, metric building, monitoring, compliance checks, and versioning.</p><p>These engineers utilize tools such as <a href="https://airflow.apache.org/"><strong>Apache Airflow</strong></a> for scheduling and monitoring, <a href="https://luigi.readthedocs.io/en/stable/index.html"><strong>Luigi</strong></a> for building pipelines, <a href="https://www.matillion.com/"><strong>Matillion</strong></a> for implementing cloud-based ETL,<a href="https://kafka.apache.org/"> <strong>Apache Kafka</strong></a>, and <strong>Datadog</strong> to ensure the delivery of quality and reliable data.</p><h3>Cost Incurred Within DataOps</h3><p><strong>For Beginners</strong></p><p>Beginners often rely on open-source and free tools to learn data pipelines and management fundamentals. However, costs can arise when experimenting with tools like <strong>Apache Airflow</strong>, as they may require cloud infrastructure such as virtual machines, storage, or data transfers, which can exceed free-tier limits.</p><p><strong>For Experienced Engineers and Enterprises</strong></p><p>Experienced engineers and enterprises use advanced tools to handle larger datasets and manage robust systems spanning multiple teams and geographical locations. Costs are driven by:</p><ul><li><strong>Licensing Fees:</strong> Premium tools often come with significant annual costs.</li><li><strong>Cloud Resource Usage:</strong> Scaling data pipelines and processing large datasets incurs high cloud storage and compute costs.</li><li><strong>Compliance:</strong> Adhering to industry regulations like <a href="https://gdpr-info.eu/"><strong>GDPR</strong></a> or <a href="https://www.hhs.gov/hipaa/index.html"><strong>HIPAA</strong></a> adds additional expenses for monitoring, logging, and security.</li></ul><h3>Conclusion</h3><p>DevOps is a unique field where even learning comes with visible costs. Ironically, cost optimization is a critical part of what DevOps practitioners aim to achieve in their workflows.</p><p>Can we, as beginners and experts, minimize these costs? Absolutely. It starts with understanding the tools available in each specialization and combining them with industry best practices. This shared journey of learning and growth is what drives us forward.</p><p>To support this, I’ve created a video that dives deeper into cost challenges and potential solutions, starting with CloudOps. I believe it will provide valuable insights and practical strategies — be sure to check it out!</p><p>📺 <strong>Watch the video here:</strong> <a href="https://youtu.be/XCKqaqnR-ZA"><strong>CloudOps Deep Dive: Cost Challenges and Solutions</strong></a><br>🔔 <strong>Subscribe to my YouTube channel for more DevOps insights: </strong><a href="https://www.youtube.com/@CloudFolklore"><strong>CloudFolklore</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=151bdad1b3b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Revolutionizing Platform Engineering and Workload Deployments: Northflank Review]]></title>
            <link>https://medium.com/@ebohgoodness.e/revolutionizing-platform-engineering-and-workload-deployments-northflank-review-676367368dd4?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/676367368dd4</guid>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[markdown]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Wed, 08 Jan 2025 08:42:50 GMT</pubDate>
            <atom:updated>2025-01-08T08:53:51.048Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8IQvcWcao8R2CCriNms4Rw.png" /></figure><p>In recent years, platform engineering has emerged as a promising approach to simplifying deployments, aiming to let developers focus on writing great code without being bogged down by infrastructure. But the reality often feels different.</p><p>As a developer, have you ever spent weeks coding, only to find yourself stuck configuring CI/CD pipelines or building the environments to test them? Or been handed a piece of code by the development team, only to waste days setting up Kubernetes clusters and Infrastructure as Code (IaC) scripts for deployment? Maybe you’ve struggled with crafting endless YAML files for ingress, issuers, and certificates, only for HTTPS setups to fail. Perhaps you’ve spun up a cluster only to hit resource limits or written liveness probes for monitoring, only to see them crash and burn.</p><p>I’ve been there, and I bet you have too. But what if there was a better, easier way? A way to deploy Kubernetes workloads, automate CI/CD pipelines, monitor cloud-native applications, scale, troubleshoot, and more — all without writing endless lines of IaC code or YAML files. Imagine cutting deployment time from days to just hours. Let me tell you, it’s so straightforward even my monkey, Ambrosia, has left the circus to become a deployment guru.</p><p>Got your attention? Great. Here’s the deal: I’m about to change how you think about developer productivity and cloud-native deployment workflows. You’re welcome!</p><h3><a href="https://northflank.com/features">NORTHFLANK</a>: the solution to Cloud Deployment woes!</h3><p>At first glance, you might assume Northflank is just another Platform as a Service (PaaS) solution. However, a closer look reveals its unique ability to let developers deploy workloads in a DIY manner — without the need to provision or manage underlying resources.</p><p>Where platforms like Heroku and Render stop, Northflank goes further. It provides an already provisioned cluster environment to deploy your resources while giving you control over your workloads’ deployment. By simplifying infrastructure management, Northflank enhances the developer experience and fosters better collaboration among teams.</p><p>This is made possible through Northflank’s direct integration with <a href="https://kubernetes.io/docs/concepts/overview/kubernetes-api/">Kubernetes APIs</a>, eliminating the operational overhead of setting up clusters manually or through <a href="https://developer.hashicorp.com/terraform/tutorials/aws-get-started/infrastructure-as-code">Infrastructure as Code (IaC)</a>.</p><p>With built-in access to popular databases like <a href="https://www.postgresql.org/">PostgreSQL</a>, <a href="https://www.mysql.com/">MySQL</a>, <a href="https://www.mongodb.com/">MongoDB</a>, <a href="https://redis.io/">Redis</a>, and more, Northflank simplifies <a href="https://northflank.com/docs/v1/application/databases-and-persistence/deploy-a-database">database management</a>, allowing teams to focus on deploying their applications rather than stressing over environment configurations or infrastructure setup.</p><p>Northflank also offers flexibility by providing access to its resources through an <a href="https://northflank.com/docs/v1/api/use-the-api">API</a>, <a href="https://northflank.com/docs/v1/api/use-the-cli">CLI</a>, <a href="https://northflank.com/docs/v1/api/use-the-javascript-client">Javascript Client</a>, and a user-friendly UI. This combination empowers developers and DevOps engineers to break down the complexity of managing workloads of all scales — from small to enterprise-level.</p><p>This streamlined approach accelerates feedback loops between teams, making development and deployment faster and more efficient. And that’s just the beginning — Northflank offers a suite of exciting features designed to make your life easier. Here are just a few highlights:</p><h3>Key Features of Northflank</h3><ul><li><strong>Effortless CI/CD Integration</strong>: Northflank facilitates easy connection with <a href="https://github.com/">GitHub</a>, <a href="https://bitbucket.org/">Bitbucket</a>, and <a href="https://about.gitlab.com/">GitLab</a>, automatically updating your workloads with every push or commit. Say goodbye to endless YAML files — Northflank takes care of it for you.</li><li><strong>Kubernetes Abstraction and BYOC</strong>: The Platform provides built-in Kubernetes clusters across the US, Europe, and Asia. For added flexibility, it supports external cloud providers like <a href="https://aws.amazon.com/">AWS</a>, <a href="https://azure.microsoft.com/en-gb/?msockid=031996ed27976b89079583ed26b06a96">Azure</a>, <a href="https://cloud.google.com/gcp?utm_source=bing&amp;utm_medium=cpc&amp;utm_campaign=emea-ng-all-en-bkws-all-all-trial-e-gcp-1011340&amp;utm_content=text-ad-none-any-DEV_c-CRE_-ADGP_Hybrid+%7C+BKWS+-+EXA+%7C+Txt+-+GCP+-+General+-+v3-KWID_43700061984001165-kwd-77103452682241:loc-137-userloc_152628&amp;utm_term=KW_gcp-NET_o-PLAC_&amp;&amp;cm_mmc=BE-PPC-DS3A-_-google-_-1707574+%7C+GCP+%7C+DR+%7C+ESS01+%7C+EMEA+%7C+NG+%7C+en+%7C+Hybrid+%7C+SEM+%7C+BKWS+-+EXA+%7C+Txt%7C+Bing_Pure_Brand_Non+Affected-_-Hybrid+%7C+BKWS+-+EXA+%7C+Txt+-+GCP+-+General+-+v3-_-gcp&amp;matchtype=e&amp;kwd-77103452682241:loc-137&amp;gclid=1859388c864d16b1af0a5fd111e178ff&amp;gclsrc=3p.ds&amp;msclkid=1859388c864d16b1af0a5fd111e178ff">GCP</a>, <a href="https://www.civo.com/">Civo</a>, and <a href="https://www.oracle.com/cloud/">Oracle</a> through its <a href="https://northflank.com/docs/v1/application/cloud-providers/use-other-cloud-providers-with-northflank">“Bring Your Own Cloud” (BYOC) feature</a>.</li><li><strong>Team Roles and Permissions</strong>: <a href="https://northflank.com/docs/v1/application/secure/use-role-based-access-control">Built-in role management</a> makes it easy to assign and manage permissions, ensuring secure and efficient team workflows.</li><li><strong>Simplified Cluster Management</strong>: With built-in tools for <a href="https://northflank.com/docs/v1/application/scale/scale-on-northflank">scaling</a>, <a href="https://northflank.com/docs/v1/application/databases-and-persistence/database-observability-and-monitoring">monitoring</a>, and <a href="https://northflank.com/docs/v1/application/observe/configure-health-checks">health checks</a>, Northflank helps you focus on building great applications instead of troubleshooting infrastructure</li><li><strong>Domain and TLS Management</strong>: Easily link your existing domains to the platform and request automatic TLS certificate provisioning to secure your deployments.</li><li><strong>Flexible Application Configuration</strong>: The platform makes managing <a href="https://northflank.com/docs/v1/application/run/inject-runtime-variables">environment variables</a> straightforward, allowing application components to be seamlessly linked for both <a href="https://northflank.com/docs/v1/application/network/networking-on-northflank">private and public-facing</a> deployments.</li></ul><p>While Northflank simplifies deployment workflows significantly, features like BYOC, <a href="https://northflank.com/docs/v1/application/getting-started/build-and-deploy-your-code#create-a-combined-service">combined service integrations</a>, and <a href="https://northflank.com/docs/v1/application/cloud-providers/create-custom-resource-plans#kubernetes-resource-configuration">Kubernetes abstractions</a> may require familiarity with cloud or Kubernetes concepts. For teams equipped with this knowledge, the platform transforms deployment processes into an efficient and seamless experience.</p><p>Now you have an idea of what Northflank is capable of, let’s dive into a real-world example.</p><h3>Deploying a Full-Stack Application with <a href="https://northflank.com/use-cases/app-platform-for-kubernetes">Northflank</a></h3><p>To put Northflank to the test, I deployed a full-stack application using the platform, exploring its features and capabilities. While there’s much to uncover, I’ve documented the key steps and insights in a dedicated <a href="https://github.com/busybrainx99/Northflank-Experience">README file</a>.</p><p>In the README, you’ll find:</p><ul><li>How I linked my <a href="https://github.com/busybrainx99">GitHub repository</a> and <a href="https://northflank.com/docs/v1/application/getting-started/set-up-a-pipeline#create-a-build-service">set up services</a>.</li><li>How I added their <a href="https://northflank.com/docs/v1/application/databases-and-persistence/access-a-database#deploy-a-database-with-tls">built-in database</a> add-on and configured <a href="https://northflank.com/docs/v1/application/getting-started/add-a-and-verify-domain">Domain</a> and <a href="https://northflank.com/docs/v1/application/domains/domains-on-northflank#certificate-generation">TLS</a>.</li><li>Insights into Northflank’s <a href="https://northflank.com/docs/v1/application/network/enable-multi-project-networking">networking</a>, <a href="https://northflank.com/docs/v1/application/scale/scale-cpu-and-memory">scaling</a>, <a href="https://northflank.com/docs/v1/application/databases-and-persistence/database-observability-and-monitoring#backups-imports-and-restores">monitoring</a>, and <a href="https://northflank.com/docs/v1/application/release/manage-ci-cd">CI/CD pipeline</a>.</li></ul><p>To explore the rest, <a href="https://app.northflank.com/signup">sign up</a> for Northflank and dive into its services and features. With a <a href="https://northflank.com/docs/v1/application/billing/pricing-on-northflank#deploy-for-free">free tier</a> available, you can explore NorthFlank’s capabilities without spending a dime.</p><h3>CONCLUSION</h3><p>Northflank has demonstrated that there’s a better way to handle deployments — a way to fast-track feedback loops and empower developers to deploy workloads in a self-service manner without the stress of managing the underlying infrastructure.</p><p>One standout feature is the workflow abstraction — it eliminates the need for YAML files in pipeline setups entirely. Imagine the possibilities when you no longer have to worry about infrastructure. From a developer’s perspective, it feels like taking all the best parts of DevOps and making them seamless.</p><p>What feature would you find most transformative? <a href="https://app.northflank.com/signup">Sign up</a>, try it for yourself, and see how Northflank can transform your workflow.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=676367368dd4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform ABC: Understanding Infrastructure as Code (IaC) with Ease (Part 1)]]></title>
            <link>https://medium.com/@ebohgoodness.e/terraform-abc-understanding-infrastructure-as-code-iac-with-ease-part-1-4202695f2968?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/4202695f2968</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[hashicorp]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Sun, 30 Jun 2024 22:55:45 GMT</pubDate>
            <atom:updated>2024-07-01T11:38:58.645Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0YW92KFiykNZD55N_HI08A.png" /></figure><p>Before the advent of Infrastructure as Code (IaC), managing cloud resources was a cumbersome and error-prone process. System administrators and DevOps engineers had to manually provision, configure, and manage servers and other infrastructure components. This manual approach led to inconsistencies, human errors, and significant challenges in scaling and maintaining infrastructure, often resulting in deployment failures and security vulnerabilities.</p><p>Infrastructure as Code revolutionized cloud resource management by defining and managing infrastructure through code. IaC brings automation, consistency, and <a href="https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control">version control</a> to infrastructure provisioning, enabling reproducible and scalable environments. It also simplifies maintenance and documentation, ensuring that infrastructure setups are easier to understand, manage, and modify as needed.</p><p>Terraform, developed by HashiCorp, is one of the best-suited tools for implementing IaC. It is <a href="https://www.synopsys.com/blogs/chip-design/cloud-native-vs-cloud-agnostic.html">provider-agnostic</a>, supporting a wide range of cloud providers, and uses declarative language to describe the desired state of infrastructure. Terraform’s state management, modularity, and extensibility, along with a strong community and ecosystem, make it an ideal choice for efficiently managing modern cloud infrastructure.</p><p>In this article, part one of our Terraform ABC series, I’ll take you through various foundational aspects of Terraform. We will start with an overview of Infrastructure as Code (IaC), delve into Terraform basics, explore HashiCorp Configuration Language (HCL), understand blocks and arguments, examine Terraform state, and go through essential Terraform commands. Additionally, we’ll have a practical demo where we’ll launch a resource using the Terraform local provider. This structured approach ensures you gain a solid understanding of Terraform’s core capabilities. Stay tuned for subsequent parts where we will cover more advanced topics such as variables, outputs, data sources, lifecycles, counts, and for_each.</p><h3>Infrastructure as code (IAC)</h3><p>With Infrastructure as Code (IaC), you can envision all of your resources as code. This includes virtual machines on your cloud provider, S3 buckets for storing objects on AWS, databases, and even on-premise hardware in your data center. IaC allows these software and hardware components to be represented as code, enabling provisioning, management, destruction, and upgrades through simple commands. By defining infrastructure as code, organizations can version control their setups and changes, storing them centrally for easy collaboration across engineering teams.</p><h3>Benefits of IAC</h3><p>Implementing IAC offers a lot of advantages that streamline and enhance infrastructure management. Here are some key benefits:</p><ol><li>IaC ensures that infrastructure is provisioned identically every time by defining the desired configuration in the code and eliminating human error or inconsistencies from the manual configuration.</li><li>Configuration files in IaC, such as software code, can be version-controlled, allowing for tracking changes, reverting to previous configurations, and team collaboration.</li><li>Teams can share and collaboratively edit IaC definitions, promoting better communication and ensuring a unified infrastructure blueprint.</li><li>IaC enables automated infrastructure provisioning and management, freeing IT staff from repetitive tasks and allowing for faster deployments and infrastructure scaling.</li></ol><h3>Use Cases For IAC</h3><p>IaC is valuable in various scenarios, including:</p><ol><li>IaC excels at provisioning and configuring cloud resources like virtual machines, storage, and networking components on platforms like AWS, Azure, or GCP.</li><li>IaC can be used to manage on-premises servers, automating configuration and deployment processes.</li><li>With IaC, network configurations, such as firewalls, routers, and VLANs, can be defined and managed as code.</li></ol><h3>What is Terraform?</h3><p>Terraform is an open-source IAC tool that empowers users to define and manage infrastructure using a high-level configuration language known as HashiCorp Configuration Language (HCL). While Terraform also supports <a href="https://www.json.org/json-en.html">JSON</a>(JavaScript Object Notation) for configuration, HCL is predominantly used. This tool enables automation of infrastructure provisioning and management across various service providers such as <a href="https://aws.amazon.com/">AWS</a>, <a href="https://azure.microsoft.com/en-us/">Azure</a>, <a href="https://cloud.google.com/">Google Cloud</a>, and <a href="https://www.digitalocean.com/">DigitalOcean</a>, as well as <a href="https://www.hpe.com/emea_europe/en/what-is/on-premises-vs-cloud.html">on-premises</a> setups and <a href="https://www.techtarget.com/searchcloudcomputing/definition/hybrid-cloud#:~:text=With%20hybrid%20cloud%2C%20enterprises%20deploy,and%20more%20data%20deployment%20options.">hybrid</a> configurations.</p><p>Terraform’s primary objective is to manage the lifecycle of your infrastructure according to the specific definitions you set in the configuration files. This includes tasks like creating, updating, and safely versioning infrastructure. By defining infrastructure as code, Terraform ensures that your infrastructure is provisioned consistently, repeatably, and predictably. This approach minimizes the risk of human error and enhances the overall efficiency of infrastructure management.</p><h3>Why Use Terraform?</h3><ul><li><strong>Declarative Language</strong>: Terraform defines your infrastructure using clear, concise language. Instead of specifying how to achieve the desired state, you describe what you want, and Terraform figures out the steps to get there</li><li><strong>Readability</strong>: The configuration files are designed to be easy to read and write, making it more straightforward to understand, share, and maintain your infrastructure definitions.</li><li><strong>Flexibility</strong>: Terraform supports many cloud providers, including AWS, Azure, Google Cloud, and many others. It can also manage on-premises and hybrid environments, giving you the flexibility to work with various platforms.</li><li><strong>Unified Workflow</strong>: Terraform allows you to manage multiple providers and services within a single workflow. This unified approach simplifies infrastructure management and ensures consistency across different environments.</li><li><strong>Immutable Infrastructure</strong>: Terraform encourages using immutable infrastructure, where changes are made by replacing resources rather than modifying them. This approach leads to more predictable and stable deployments, reducing the risk of configuration drift.</li><li><strong>State File</strong>: Terraform maintains a state file that tracks the current state of your infrastructure. This state file allows Terraform to understand which resources it manages and what changes must be applied, ensuring accurate updates.</li><li><strong>Incremental Updates</strong>: Terraform can apply incremental changes to your infrastructure using the state file, making only the necessary updates and minimizing disruptions.</li><li><strong>Dependency Management</strong>: Terraform builds a dependency graph of all resources defined in the configuration. This graph determines how resources should be created or updated, allowing for efficient and parallel execution when possible.</li><li><strong>Efficient Planning</strong>: Understanding the dependencies between resources enables Terraform to plan and apply changes effectively, reducing the risk of errors and downtime.</li><li><strong>Reusability</strong>: Terraform supports modular configurations, allowing you to create reusable modules that can be shared across projects. This modularity promotes best practices and consistency in your infrastructure.</li><li><strong>Automation</strong>: Terraform automates the entire infrastructure provisioning process, from initial setup to updates and destruction. This automation saves time, reduces the risk of manual errors, and allows your team to focus on more strategic tasks.</li><li><strong>Extensive Ecosystem</strong>: Terraform has a vibrant community and a rich ecosystem of plugins (providers and modules) that extend its functionality. This ecosystem integrates various services and tools, enhancing Terraform’s capabilities.</li><li><strong>Support and Resources</strong>: Comprehensive documentation, tutorials, and community support make it easy for users to find help and resources to get started and resolve issues.</li><li><strong>Best Practices</strong>: Terraform can be used to enforce security best practices within your infrastructure. You can define access controls, resource tagging, and other security measures directly in your configuration code, ensuring compliance and protection.</li><li><strong>Consistent Provisioning</strong>: Terraform ensures that your infrastructure is provisioned identically every time. This consistency eliminates the inconsistencies and errors with manual configuration, leading to more reliable deployments.</li><li><strong>Track Changes</strong>: Terraform configuration files can be stored in version control systems like Git. This integration allows you to track changes, revert to previous versions if needed, and collaborate effectively with your team, ensuring a transparent and controlled infrastructure management process.</li></ul><h3>Understanding Terraform’s Configuration Language</h3><p>HCL, terraform’s recognized configuration language, is a human-readable and machine-friendly language developed by the reputable Hashicorp. It’s designed to write configuration files that are easily understandable, making it ideal for straightforward provisioning and management of infrastructures.</p><p>HCL files typically have the .tf extension and are made up of blocks and arguments.</p><h3>Blocks</h3><p>These are the primary units of the HCL structure. They can be viewed as containers that group related configuration settings. Blocks can be classified into various types, and each of these types serves a specific purpose in organizing and defining your infrastructure as code. Some common block types include:</p><h4>Provider Block</h4><p>HCL offers various providers (e.g., AWS, Azure, Google Cloud, etc.) that enable the creation of infrastructure. These providers are defined within the provider block. The keyword ‘provider’ initiates the code, followed by the specific provider type. The provider block specifies the configuration settings for the chosen provider.</p><p><strong>Example of Provider Block</strong></p><pre>provider &quot;aws&quot; {<br> region = &quot;us-east-1&quot;<br>}</pre><h4>Resource Block</h4><p>This block defines the different infrastructure resources that can be created. The keyword “resource” is used at the start of the code, followed by the resource type and its user-defined name. The resource type is a predefined name specified in the Terraform documentation.</p><p><strong>Example of Resource Block</strong></p><pre>resource &quot;aws_instance&quot; &quot;web&quot; {<br> ami = &quot;ami-00beae93a2d981137&quot; # Change to your desired AMI<br> instance_type = &quot;t2.micro&quot;<br>}</pre><h4>Variable Block</h4><p>The variable block in Terraform declares variables that assign parameters within your configuration files. Instead of hardcoding values directly in resource blocks, you define these values as variables. This approach enhances flexibility and reusability, allowing dynamic adjustment of configurations without modifying the code itself.</p><p>The keyword “variable” is used at the beginning of the code, followed by the variable name. Various optional arguments, including description, type, default, validation, and sensitive settings, can be specified within the variable block.</p><p><strong>Example of Variable Block</strong></p><pre>variable &quot;instance_count&quot; {<br> description = &quot;Number of instances to create&quot;<br> type = number<br> default = 1<br>}<br>variable &quot;instance_type&quot; {<br> description = &quot;Type of instance to use&quot;<br> type = string<br>}</pre><p><strong>Usage in Resource Block</strong></p><pre>resource &quot;aws_instance&quot; &quot;web&quot; {<br> count = var.instance_count<br> instance_type = var.instance_type<br> ami = &quot;ami-00beae93a2d981137&quot;<br>}</pre><h4>Output Block</h4><p>The output block defines values that are exposed after the configuration is applied. It allows you to retrieve and display specific information about your infrastructure, such as IP addresses, instance IDs, or any computed values that you want to share or use in other parts of your configuration file. The keyword “output” is used at the beginning of the code, followed by the output name, which is used to reference the output value within the Terraform configuration.</p><p><strong>Example of Output Block</strong></p><pre>output &quot;instance_id&quot; {<br> value = aws_instance.web.id<br> description = &quot;The ID of the created instance&quot;<br>}</pre><h3><strong>Arguments</strong></h3><p>Arguments in Terraform are key-value pairs within blocks that define configuration settings for elements such as resources, outputs, and variables in Terraform configurations. These arguments can accept values of various types, including strings, numbers, lists, maps, and more.</p><p><strong>Examples of arguments</strong></p><pre>ami = &quot;ami-123456&quot; <br>instance_type = &quot;t2.micro&quot;</pre><h3>Terraform State</h3><p>Terraform state can be seen as a log file containing vital information about the infrastructure being managed. This essential file gives Terraform comprehensive knowledge about the resources under its control, including when to update and destroy them. The Terraform state is stored in a JSON-formatted file called<strong> terraform.tfstate</strong>. It gets created and updated when the<strong> terraform apply</strong> command is run.</p><p>When you execute the <strong>terraform apply</strong> command, Terraform performs the following actions:</p><ul><li><strong>Creates the State File</strong>: If it doesn’t already exist, Terraform creates the terraform.tfstate file.</li><li><strong>Updates the State File</strong>: If it does exist, Terraform updates the state file to reflect the current state of the managed infrastructure, including any new resources created, existing resources updated, or resources destroyed during the application process.</li></ul><p>When the <strong>terraform destroy </strong>command is run, Terraform performs these actions:</p><ul><li><strong>Empties the State File</strong>: The state file gets emptied to indicate that the managed resources have been destroyed.</li><li><strong>Creates a Backup State File</strong>: A backup state file is created, which contains a log of the previously created resources. This backup helps in auditing and troubleshooting by preserving the last known state before destruction.</li></ul><p>The state file maintains a mapping between:</p><ul><li><strong>Declared Resources</strong>: These are resources defined in your Terraform configuration files (e.g., virtual machines, storage buckets).</li><li><strong>Actual Infrastructure</strong>: Reflects the real-world resources created and managed by Terraform on your cloud provider or on-premises environment.</li></ul><p><strong>Example Configuration</strong></p><pre>{<br>  &quot;version&quot;: 4,<br>  &quot;terraform_version&quot;: &quot;1.8.5&quot;,<br>  &quot;serial&quot;: 69,<br>  &quot;lineage&quot;: &quot;d3e17b29-c068-e258-0eb0-d063d43c6213&quot;,<br>  &quot;outputs&quot;: {},<br>  &quot;resources&quot;: [<br>    {<br>      &quot;mode&quot;: &quot;data&quot;,<br>      &quot;type&quot;: &quot;aws_caller_identity&quot;,<br>      &quot;name&quot;: &quot;current&quot;,<br>      &quot;provider&quot;: &quot;provider[\&quot;registry.terraform.io/hashicorp/aws\&quot;]&quot;,<br>      &quot;instances&quot;: [<br>        {<br>          &quot;schema_version&quot;: 0,<br>          &quot;attributes&quot;: {<br>            &quot;account_id&quot;: &quot;295682243665&quot;,<br>            &quot;arn&quot;: &quot;arn:aws:iam::295682243665:user/terraform_user&quot;,<br>            &quot;id&quot;: &quot;295682243665&quot;,<br>            &quot;user_id&quot;: &quot;AIDAUJWAI7RI7QLKAIJHL&quot;<br>          },<br>          &quot;sensitive_attributes&quot;: []<br>        }<br><br>      ]<br>   }</pre><p>The Terraform state file above gives insight into two main aspects:</p><ol><li><strong>Metadata of the State File:</strong> This includes information about the state file itself and the Terraform run.</li><li><strong>State of a Specific Resource:</strong> This part contains details about the managed resources in your AWS infrastructure.</li></ol><p>Here’s a breakdown of the key components of the state file:</p><p>The state file and terraform run metadata contain info like <strong>version</strong>, <strong>terraform_version</strong>, <strong>serial</strong>, <strong>lineage</strong>, and <strong>outputs.</strong></p><p>The second part gives insights into the state of a specific resource called <strong>aws_caller_identity</strong> in your AWS infrastructure, including details like account ID and user ARN. Understanding the state file structure helps manage and troubleshoot your Terraform-managed infrastructure. Here is a break down;</p><ul><li><strong>version</strong>: Indicates the version of the state file format. Here, it is <strong>version 4</strong>.</li><li><strong>terraform_version</strong>: The version of Terraform used to create or update this state file. In this example, it is <strong>1.8.5</strong>.</li><li><strong>serial</strong>: A counter that increments each time the state file is modified. It helps Terraform identify changes in the state.</li><li><strong>lineage</strong>: A unique identifier for the state file that remains consistent across modifications, ensuring continuity.</li><li><strong>outputs</strong>: This section would list any outputs defined in your Terraform configuration. In this snippet, it is empty.</li><li><strong>resources</strong>: This section lists all resources managed by Terraform. Each resource includes:</li><li><strong>mode</strong>: Specifies whether the resource is a managed or data resource. In this case, it is a <strong>data resource</strong>.</li><li><strong>type</strong>: The type of resource, which is <strong>aws_caller_identity,</strong> in this example.</li><li><strong>name</strong>: The name assigned to the resource in the Terraform configuration here is <strong>current</strong>.</li><li><strong>provider</strong>: The provider used to manage this resource is specified as <strong>registry.terraform.io/hashicorp/aws</strong>.</li><li><strong>instances</strong>: A list of instances for this resource type. Each instance includes:</li><li><strong>schema_version</strong>: The schema version of the resource.</li><li><strong>attributes</strong>: The actual attributes of the resource instance, such as:</li><li><strong>account_id</strong>: The AWS account ID.</li><li><strong>arn</strong>: The Amazon Resource Name (ARN) of the IAM user.</li><li><strong>id</strong>: The AWS account ID.</li><li><strong>user_id</strong>: The unique ID of the IAM user.</li><li><strong>sensitive_attributes</strong>: Lists attributes considered sensitive. In this snippet, it is empty.</li></ul><h4>Importance of State Files</h4><p>The state file is crucial for Terraform for the following reasons:</p><h4>1. Tracking Infrastructure State:</h4><p>Terraform depends on the state file to keep track of the current state of deployed resources. This allows Terraform to understand the existing resources and their current attributes.</p><h4>2. Guiding Infrastructure Changes:</h4><p>When you make changes to your Terraform configuration, Terraform compares the desired state (your configuration) with the recorded state in the state file. This process determines which resources must be created, updated, or deleted to align with your desired infrastructure.</p><h4>3. Performance Enhancement:</h4><p>The state file is particularly valuable for large-scale infrastructures. It enhances Terraform’s performance by maintaining a record of existing resources. This optimization enables Terraform to concentrate on necessary changes instead of re-evaluating the entire infrastructure during each operation.</p><h4>Storage and Best Practices</h4><p>Regarding storage and best practices, Terraform typically stores the state file locally in the directory where commands are executed. However, it is highly recommended that remote state storage solutions be utilized for collaborative or production environments.</p><h4>Best Practices for Managing State Files:</h4><ul><li><strong>Version Control: </strong>It is crucial to include the state file in version control (e.g., Git) to track changes and facilitate rollback if necessary effectively.</li><li><strong>Remote Backend Storage:</strong> The state file must be stored in a remote backend such as Amazon S3, Azure Blob Storage, or HashiCorp Consul. This centralized approach enhances security, access control, and disaster recovery capabilities.</li><li><strong>Encryption:</strong> To protect sensitive infrastructure data, it is imperative to ensure that the state file is encrypted both at rest and in transit.</li><li><strong>Access Control:</strong> Strict access controls should be implemented to limit who can modify the state file, thus guarding against unauthorized changes.</li></ul><h3>Terraform Commands</h3><p>Terraform provides a command-line interface for managing configuration files. There are four main commands used to manage Terraform configurations:</p><ul><li><strong>terraform init</strong>: Initializes the working directory and downloads any required providers and modules specified in the configuration files. This command prepares the Terraform environment for use.</li><li><strong>terraform plan</strong>: Generates an execution plan that previews the changes Terraform will apply to your infrastructure based on the current configuration files. It outlines additions, modifications, or deletions of resources.</li><li><strong>terraform apply</strong>: Executes the changes outlined in the execution plan generated by the terraform plan. This command creates or modifies infrastructure resources according to the configuration files, bringing the actual state of the infrastructure closer to the desired state.</li><li><strong>terraform destroy</strong>: Safely and efficiently destroys the infrastructure managed by Terraform. It removes all resources created by Terraform based on the configuration files, ensuring the clean-up of cloud resources to avoid unnecessary costs.</li></ul><p>For more Terraform commands and detailed documentation, refer to this article: <a href="https://spacelift.io/blog/terraform-commands-cheat-sheet">Terraform Cheat Sheet — 23 Terraform CLI Commands &amp; Examples</a></p><h3>Terraform Demo</h3><p>Before proceeding to the demo phase, ensure that Terraform is installed on your computer. I recommend using a Linux operating system. If you are using macOS or Windows, you can provision a Linux virtual machine and install Terraform on it. Here is <a href="https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli">HashiCorp’s documentation</a> on how to install Terraform.</p><p>You can work with Terraform directly on your Linux VM, but I recommend linking your Linux machine to VS Code via <a href="https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh">Remote SSH.</a> Here is a <a href="https://youtu.be/9pX4vhxTsBw?si=Z8JdYDxyghrpb-lG">video</a> on how to do this on my <a href="https://www.youtube.com/channel/UCRxWqZE9K3by1ldarkEoS5g">YouTube channel: CLoudFolklore</a>.</p><h3>Prerequisites</h3><p>Welcome to the demo section of the article! To follow along, please ensure you have the following:</p><ul><li>Terraform installed on your system</li><li>A stable internet connection</li><li>A readiness to absorb knowledge</li></ul><h3>Demo one: Launching Resources with Terraform Local Resource Deployment.</h3><p>Step one: Create a Directory for Terraform Configurations:</p><ul><li>Keeping your Terraform configurations organized in separate directories for different projects is considered good practice. Therefore, for our new project, please create a new directory:</li></ul><pre>mkdir terraform-local-resource<br>cd terraform-local-resource</pre><p>Step two: Create a Basic Configuration File:</p><ul><li>Create a new file named <strong>main.tf</strong> and paste the following content to provision a local file:</li></ul><pre>resource &quot;local_file&quot; &quot;welcome&quot; {<br> content = &quot;Welcome to our first terraform demo&quot;<br> filename = &quot;${path.module}/welcome.txt&quot;<br>}</pre><p>I want to clarify that<strong> ${path.module}</strong> is a built-in interpolation variable that places your created resource, in our case “welcome.txt”, in the absolute path where your configuration file <strong>main.tf</strong> is located.</p><p>If you want <strong>welcome.txt</strong> to be placed in a different path, such as your device’s home directory, you would need to specify that path instead of <strong>${path.module}</strong>, for example: <strong>/home/goodness/welcome.txt</strong>, assuming your device’s home path is<strong> /home/goodness.</strong></p><p>Step three: Running terraform Commands</p><ul><li>Initialize terraform</li></ul><pre>terraform init</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hOk59o1mbLWhIAX_" /></figure><p>When you initialize Terraform, you should receive a result similar to the figure above. During initialization, Terraform accesses its registry to fetch the latest version of the resources specified, installs them, and creates a lock file to record your selections.</p><pre>terraform plan</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iRjsOEs4Oe1R8jQZ" /></figure><p>Running <strong>terraform plan</strong> provides a detailed preview of the changes that Terraform will make when you execute your code. It shows exactly what resources will be added, changed, or destroyed based on your Terraform files&#39; current state and configuration.</p><pre>terraform apply</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eUsauCJGpMVjLGtE" /></figure><p>Running the command <strong>terraform apply</strong> and confirming with<strong> yes</strong> will create the specified resource(s) successfully. This command executes the planned actions based on your Terraform configuration and applies them to your infrastructure as defined.</p><p>Step four: Reviewing the state and outputs:</p><p>After running <strong>terraform apply</strong>, you can check the state file, <strong>terraform.tfstate, </strong>to see how Terraform tracks resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hjkzmsKMwlx767X5" /></figure><p>To view the output of our resource, you can check the specified file or directory, which in our case is <strong>welcome.txt</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*epMAxOM57OXzh4UB" /></figure><h3>Conclusion</h3><p>By grasping the power of Infrastructure as Code (IaC) and diving into Terraform’s core functionalities, you’ve established a strong foundation for infrastructure automation. Remember, this is just the first part! A vast world of advanced Terraform features awaits, ready to empower you to tackle intricate infrastructure challenges, streamline deployments, and unlock Terraform’s true potential.</p><p>Stay tuned for the next part of this series, where we’ll delve deeper into advanced Terraform topics. We’ll explore variables, outputs, data sources, lifecycles, counts, and for_each. These powerful tools will equip you to become a Terraform master!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4202695f2968" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Networking 101: Core Concepts Every IT Professional Should Know]]></title>
            <link>https://medium.com/@ebohgoodness.e/networking-101-core-concepts-every-it-professional-should-know-a24f60633c8f?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/a24f60633c8f</guid>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Fri, 31 May 2024 17:35:59 GMT</pubDate>
            <atom:updated>2024-05-31T18:01:37.079Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ykvz1PYyefGx8AkNsN1DXQ.png" /></figure><p>In this ever-involving world where technology trends tend to change as fast as the speed of light, it is essential to maintain a grasp of the fundamentals. Networking is a foundational concept that shouldn’t just be waved aside. However, as developers and engineers get immersed in the exciting world of cloud computing, conquering complexities like <a href="https://aws.amazon.com/what-is/containerization/">containerization</a>, <a href="https://www.vmware.com/topics/glossary/content/cloud-orchestration.html">orchestration</a>, <a href="https://www.vmware.com/topics/glossary/content/cloud-automation.html">automation</a>, and <a href="https://www.linkedin.com/advice/0/how-can-you-use-system-administration-scripts">scripting</a> take center stage. Whereas, essential networking concepts like DNS, routing, switching, and gateways are often left unexplored territory.</p><p>However, a strong foundation in these fundamentals can elevate your professional skills and set you apart in this rapidly evolving landscape.</p><p>Consider, for instance, a cloud engineer deploying <a href="https://www.sentinelone.com/blog/scalable-architecture/">scalable architectures</a> on a virtual machine. Without a solid grasp of networking concepts, how can one ensure seamless communication between containers or the efficient routing of traffic across nodes? Or envision the software developer creating the next groundbreaking web application — how can they assure a reliable domain name resolution without understanding the convolutions of <strong>DNS</strong>? The repercussions of neglecting networking basics span various disciplines in the tech space, not just the cloud discipline.</p><p>This article illuminates essential networking concepts, providing valuable insights for tech professionals seeking a deeper understanding of network infrastructure.</p><h3>What is Networking</h3><p>Networking, the act of connecting computers and other devices to share resources and information, is a crucial aspect of IT. It involves the design, implementation, and management of the infrastructure that enables data exchange between various devices, such as computers, servers, and mobile devices. This connectivity is essential for accessing the internet, sharing files, and communicating through email or instant messaging.</p><p><strong><em>Critical Components of Networking:</em></strong></p><ol><li><em>Devices: These include computers, servers, switches, routers, and other hardware that facilitate connectivity and data exchange.</em></li><li><em>Transmission Media: The physical or wireless channels through which data travels. This includes cables (like Ethernet) and wireless signals (like Wi-Fi).</em></li><li><em>Protocols: Rules and conventions for communication between network devices. Protocols like </em><a href="https://www.geeksforgeeks.org/tcp-ip-model/"><em>TCP/IP</em></a><em>, </em><a href="https://www.cloudflare.com/learning/ddos/glossary/hypertext-transfer-protocol-http/"><em>HTTP</em></a><em>, and</em><a href="https://www.geeksforgeeks.org/file-transfer-protocol-ftp-in-application-layer/"><em> FTP</em></a><em> ensure data is transmitted accurately and efficiently.</em></li><li><em>Network Services: These services facilitate network functionality, including DNS for domain name resolution, </em><a href="https://www.comptia.org/blog/what-is-dhcp"><em>DHCP</em></a><em> for dynamic IP address assignment, and more.</em></li></ol><p>Networking has the power to scale, enabling the creation of everything from <a href="https://www.cisco.com/c/en/us/products/switches/what-is-a-lan-local-area-network.html">local area networks (LANs)</a> within a single building to <a href="https://www.cisco.com/c/en/us/products/switches/what-is-a-wan-wide-area-network.html">wide area networks (WANs)</a> that span entire regions or countries. This scalability showcases the vast potential and growth opportunities in networking.</p><p>To effectively manage and troubleshoot these networks, the OSI (Open Systems Interconnection) model provides a crucial conceptual framework.</p><h3>The OSI Model:</h3><p>We can’t talk about networking without chipping in the OSI Model, as it is a core part of networking. OSI serves as a universal language for understanding how different devices and protocols on a network interact without regard to these devices’ internal structure and technology. It is divided into seven layers, each serving a specific function and interacting with the layers directly below and above it.</p><h3>Layers of the OSI Model</h3><ul><li><em>Physical Layer (Layer 1): This layer deals with the physical transmission of data through cables (twisted-pair, coaxial, fibre optic) or wireless signals (WiFi, Bluetooth). It defines the “how” of getting data bits onto the network. Essentially, the physical layer ensures that the raw data bits are converted into a physical signal (electrical or electromagnetic) that can be transmitted over the network medium.</em></li><li><em>Data Link Layer (Layer 2): This layer plays a crucial role in ensuring error-free data transfer between devices directly connected over a physical link (cable or wireless). It does this by breaking data into manageable units (frames) and adding error detection mechanisms to identify corrupted data during transmission. Additionally, the Data Link Layer controls the flow of data to prevent overwhelming the receiving device with too much data at once. Techniques like stop-and-wait are employed to regulate this flow.</em></li><li><em>Network Layer (Layer 3): This layer is the control centre for routing data packets across different networks. It uses IP addresses, unique identifiers for devices on the internet, to direct packets towards their final destinations. Unlike Layer 2, which uses </em><a href="https://www.techtarget.com/searchnetworking/definition/MAC-address"><em>MAC addresses</em></a><em> for local communication, the Network Layer enables communication across vast distances. It breaks down large packets of data into smaller fragments for easy transportation and reassembles them at their destination.</em></li><li><em>Transport Layer (Layer 4): This layer plays a vital role in ensuring reliable and efficient data transfer between applications on different devices. It ensures end-to-end communication between the layers of the OSI model, prevents congestion and establishes and maintains connections between devices for the duration of a data transfer session. The key protocols at this layer include:</em></li><li><em>TCP (Transmission Control Protocol):</em></li><li><em>Ensures reliable data transfer.</em></li><li><em>Guarantees in-order delivery.</em></li><li><em>Handles error correction.</em></li></ul><p><em>UDP (User Datagram Protocol):</em></p><ul><li><em>Ensures faster transmission of data packets without error checking.</em></li><li><em>Suitable for time-sensitive applications like video streaming.</em></li></ul><p><em>The Transport Layer is crucial for maintaining the integrity and efficiency of data transfer, making it a fundamental component of the OSI model.</em></p><ul><li><em>Session Layer (Layer 5): This layer establishes connections, manages data flow, and ensures messages arrive in order. Like a virtual call coordinator, it handles errors and terminates sessions when communication ends. This layer is essential for reliable application communication across networks.</em></li><li><em>Presentation Layer (Layer 6): This layer is the translator between applications and the network. Imagine two people speaking different languages trying to communicate. This layer takes the data from the application layer, which might be in a specific format for that program. It translates it into a format the network can understand and transmit efficiently. Situated as the sixth layer within the OSI model, it is a crucial intermediary between the application layer and the network format. Its primary function is translating data to ensure seamless communication across diverse systems.</em></li><li><em>Application Layer (Layer 7): This is the highest layer of the OSI Model, serving as the interface between the network and end-user application. They offer services like remote access, messaging, etc., that cater to the needs of the user application. It abstracts the complexities of lower-level network protocols, presenting a simplified interface to applications. Examples of its protocols are internet protocols like HTTP, file transfer protocol (FTP), and email protocols like POP and SMTP.</em></li></ul><h3>Importance of Networking in IT Infrastructure</h3><p>Networking is the backbone of IT. This skeletal support enables interconnection between devices, applications, technologies, and users. Without a robust network, applications cannot communicate, devices cannot share data, and the digital ecosystem would come to a standstill. Whether it’s a small business or a global enterprise, its operations’ efficiency, security, and reliability depend heavily on the strength and understanding of its networking capabilities.</p><p>IT professionals worldwide must be able to understand and fully manage networking concepts efficiently. Let’s explore some key concepts that lay the groundwork for effective networking.</p><h3>Key Concepts</h3><h3>Switching</h3><p><em>This involves intelligently directing data streams within a local network. Think of the data streams as cars and the switch as a traffic warden, ensuring that data from one device reaches the correct device. Efficient switching ascertains fast and accurate data travel, reducing latency and improving performance.</em></p><p>A network switch is a crucial networking device that facilitates communication within a LAN. Unlike <a href="https://www.geeksforgeeks.org/what-is-network-hub-and-how-it-works/">hubs</a> and <a href="https://www.tutorialspoint.com/what-are-repeaters-in-computer-network">repeaters</a> that forward data to all connected devices, switches intelligently forward data frames to only the intended recipient.</p><p>Switches operate in the data link layer of the OSI model. At this layer, data units are encapsulated into frames, and switches use the MAC (Media Access Control) addresses to send these data frames between devices within the same local network. By analyzing the MAC addresses of incoming data and maintaining a table of MAC addresses to port mappings, switches decide the appropriate devices to send the frames to. Switches can be classified into two forms: physical and virtual.</p><p>Physical switches are hardware devices that physically connect devices like computers, servers, etc., within a LAN. They come in various configurations with different numbers of ports depending on network needs. Common use cases include office LANS and data centers.</p><p>Virtual switches are software-based equivalents of physical switches that operate within virtual environments like VMs and hypervisors. Common use cases include cloud environments and virtualized data centers.</p><h4>How Switching Works</h4><p>Let’s take two Linux computers, A and B, with IP addresses 198.162.205.1 and 198.162.205.2, respectively. Without the presence of a switch, they wouldn’t be able to connect and communicate with each other. The switch acts as a bridge, allowing data frames to flow from A to B. An important component called the network interface allows the exchange of data frames between devices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xvJmkrJIIbC4GmLz" /></figure><h3>Routing</h3><p><em>This involves finding the best path for data to travel through. It’s similar to how Google Maps finds the best routes to avoid traffic and other forms of delay. Efficient routing ensures that data packets navigate complex network connections quickly, thereby maintaining fast and secure communication across devices in a network.</em></p><p>While switching involves getting devices within the same network to communicate, routing involves using a specialized device called a router to connect devices across different networks. Unlike switches that rely on MAC addresses, routers rely on only IP addresses to get the best path for transporting data packets to their final destination.</p><p>Routers operate on layer 3 of the OSI model, managing data packets, which include network layer information such as source and destination IP addresses. They determine network paths based on routing algorithms and protocols. At this layer, data units are encapsulated in packets. While switching focuses on LAN management, routing focuses on WAN and inter-network traffic management.</p><h4>Routing Table</h4><p>A routing table is a database stored within a router that lists the routes to various network destinations. It provides the vital information needed to forward packets along the best path. Some key components of a routing table include;</p><ul><li><em>Destination Network: This field specifies the IP address of the destination network</em></li><li><em>Subnet Mask: This field indicates the network portion of the destination IP, allowing the router to match addresses within that specified network range.</em></li><li><em>Next Hop: This field specifies the IP address of the next router or gateway to which data packets should be forwarded</em></li><li><em>Outgoing interface: This field specifies the network interface on a router that should be used to forward data packets to the next hop.</em></li></ul><h4>How Routing Works</h4><p>Consider four devices, A, B, C, and D, with IP addresses 198.162.205.10, 198.162.205.11, 198.162.206.10, and 198.162.206.11, respectively. Devices A and B are connected to the same network via a switch with the network address 198.162.205.0, while devices C and D are connected to another network via a different switch with the network address 198.162.206.0.</p><p>Since A and B are on the same network, they can communicate directly, as can C and D. However, communication is impossible between A and C, A and D, B and C, or B and D because they are on different networks.</p><p>By connecting a router to the switches of these devices, the router can access an IP address from the first network and another IP address from the second network. This configuration allows the router to route data packets between the networks, enabling communication between devices on different networks. For instance, data can flow from device A to device C through the router, facilitating inter-network communication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NkWI_NQroD9KBPPb" /></figure><h3>Gateway</h3><p><em>Devices on different networks must have a gateway to communicate and exchange data. Imagine two separate towns with no way to access each other. Then, an engineer comes in and builds a bridge. The bridge makes it possible for people from town A to access town B and vice versa. Gateways are network points that act as entrances to another network, like linking your office LAN in one city to your office LAN in another.</em></p><p>Gateways act as general-purpose connection points facilitating data transfer between network architectures or protocols. Think of the gateway as a translator and devices on different networks as foreign nationals. With the aid of a gateway, the nationals can communicate and transact. There are various kinds of gateways depending on the purpose they serve; some include:</p><ul><li><em>Protocol Translators: These gateways enable communication between networks that use different protocols by converting one protocol to another. An example is allowing communication between devices using</em><a href="https://www.geeksforgeeks.org/what-is-ipv4/"><em> IPV4 </em></a><em>and </em><a href="https://www.cisco.com/c/en/us/solutions/ipv6/overview.html#~q-a"><em>IPV6</em></a><em> or email gateway translating between </em><a href="https://www.cloudflare.com/learning/email-security/what-is-smtp/"><em>SMTP</em></a><em> and </em><a href="https://support.microsoft.com/en-us/office/what-are-imap-and-pop-ca2c5799-49f9-4079-aefe-ddca85d5b1c9"><em>POP.</em></a></li><li><em>Firewall Gateways: These gateways act as security barriers, filtering incoming and outgoing traffic based on defined rules and protecting the network from attacks.</em></li><li><em>VoIP( Voice over IP)Gateways: These gateways allow voice calls over the Internet and traditional telephony networks, allowing seamless communication across networks.</em></li></ul><h3>Default Gateway</h3><p><em>Gateway and Default Gateway are similar. Still, the main difference is that while devices use gateways when the route to a destination device is known, default gateways are the specific gateways that devices use when they don’t know the particular route to a destination. For example, a default gateway is typically used to link your office LAN to the Internet.</em></p><p>When people talk about gateways in network communication, they often refer to default gateways. While a gateway acts as a general connection point between networks, a default gateway is a crucial device on a LAN that serves as the exit point for traffic destined for external networks, including the Internet. The router often acts as the default gateway in a typical office network.</p><h4>How Default Gateway Works</h4><p>When a device on your local network needs to communicate with an external network, such as an internet website, it sends the data packets to the default gateway. The default gateway is typically a router that forwards the data to the appropriate external network. It handles the routing and translation required for the data to reach its destination.</p><p>Let’s take the illustration we made for the concept of routing. If any of those devices (A, B, C, or D) want to access IP addresses outside their local networks, they can do so via the default gateway. The default gateway routes the data packets to the correct external network, ensuring they reach their intended destination.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h6rr_cBafspJsG1S" /></figure><h3>DNS (The Domain Name System)</h3><p><strong><em>Domain names</em></strong><em> are easily readable and memorable names set in the DNS to access the IP addresses of specific websites and web pages. Instead of memorizing IPs like </em><strong><em>142.250.190.110</em></strong><em> to access Google, you can simply use</em><a href="http://www.google.com/"><em> </em></a><a href="http://www.google.com/"><em>www.google.com</em></a><em>.</em><strong><em>DNS</em></strong><em> matches hard-to-remember IP addresses, which computers use to identify each other on a network, to easily memorable, human-readable names like</em><a href="http://www.google.com/"><em> </em></a><a href="http://www.google.com/"><em>www.google.com</em></a><em>. DNS is just like a phonebook that matches names with phone numbers</em></p><p>Without DNS, remembering complex number sequences for websites would be a nightmare! DNS makes the internet user-friendly, allowing us to access websites and online services using memorable names.</p><h4>How DNS Resolves Domain Names</h4><p>When you enter a domain name in your browser, your computer contacts a local DNS server (recursive resolver). It then works behind the scenes, querying other DNS servers (root, TLD, and authoritative) until it finds the corresponding IP address.</p><p><strong><em>DNS Servers Explained</em></strong></p><ul><li><strong><em>Recursive Resolvers:</em></strong><em> These are the first responders, receiving your queries and delegating the search for the IP address.</em></li><li><strong><em>Root Servers are the root of the DNS hierarchy.</em></strong><em> They point resolvers to the TLD servers.</em></li><li><strong><em>TLD Servers:</em></strong><em> These manage specific domain extensions (like .com or .org) and direct resolvers to the authoritative nameservers for domains within their TLD.</em></li><li><strong><em>Authoritative Nameservers:</em></strong><em> The final stop, these store the actual IP address for a specific domain and provides the answer to the recursive resolver’s query.</em></li></ul><p><em>By working together, these components ensure smooth navigation on the internet, letting you access websites using user-friendly domain names.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*N8uY9JVrBvDC3fp9" /></figure><h3>Conclusion</h3><h4>Recap</h4><p>We have explored various essential theoretical concepts that form the bedrock of networking. We delved into the roles of switches, gateways, and routers, understanding how they work together to transmit data within and across networks. We also explored the role default gateways, DNS (the Internet’s address book), and the OSI model play in network communication.</p><h4>Final Thoughts</h4><p>Networking plays a key role in the infrastructure of modern technology, facilitating seamless communication, data transfer, and connectivity across various platforms and devices. It enables the integration of different technologies, ensuring they work together efficiently to deliver robust and reliable services.</p><p>Having a solid grasp of these theoretical concepts in networking is crucial for understanding how the network operates and forms the bedrock for hands-on network management and troubleshooting.</p><h4>Call to Action</h4><p>This is just the beginning of your networking journey. A vast world is out there, spiced with knowledge waiting to be consumed. Here are some resources to help deepen your understanding and gain practical experience.</p><h4>Online Courses and Tutorials:</h4><ul><li><em>Alison’s Free Course: </em><a href="https://alison.com/courses/it"><em>Alison</em></a><em> offers a lot of free courses, including a </em><a href="https://alison.com/course/basics-of-computer-networking"><em>fundamental networking course</em></a><em>. However, you must pay to get the certificates. You can also take full advantage of their IT courses and update your knowledge.</em></li><li><em>Facyber’s Networking Github Repository: Facyber’s </em><a href="https://github.com/facyber/awesome-networking"><em>awesome-networking </em></a><em>repository points you to various networking courses, tutorials, certifications, books etc. It is a very rich GitHub repository; you should visit and take advantage of available information.</em></li><li><em>Udemy: There are a few networking courses I would recommend on Udemy; they include </em><a href="https://www.udemy.com/course/complete-networking-fundamentals-course-ccna-start/"><em>The Complete Networking Fundamental Course</em></a><em>, C</em><a href="https://www.udemy.com/course/computer-networking-masterclass-interview-success/?couponCode=OF52424"><em>omputer Networking Masterclass</em></a><em>, and</em><a href="https://www.udemy.com/course/fundamentals-of-networking-for-effective-backend-design/"><em> Fundamentals of Network Engineering</em></a><em>.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a24f60633c8f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DEPLOYING LARAVEL USING THE LAMP STACK ON UBUNTU]]></title>
            <link>https://medium.com/@ebohgoodness.e/deploying-laravel-using-the-lamp-stack-on-ubuntu-18bd6c3c6c92?source=rss-d0e4b9fdc4e6------2</link>
            <guid isPermaLink="false">https://medium.com/p/18bd6c3c6c92</guid>
            <category><![CDATA[php]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[laravel]]></category>
            <category><![CDATA[mysql]]></category>
            <category><![CDATA[apache]]></category>
            <dc:creator><![CDATA[Goodness E. Eboh]]></dc:creator>
            <pubDate>Sat, 27 Apr 2024 14:46:29 GMT</pubDate>
            <atom:updated>2024-05-16T13:17:02.695Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_nouvLlPovuxD6wBjna14Q.png" /></figure><p>I wanted to install Laravel using the lamp stack on my Ubuntu virtual machine. I also intended to automate the entire process with bashscript and then use Ansible to run it remotely on a slave system. At first, it was a bit of a challenge to deploy it manually, but once I could, it was quite simple to automate the entire process.</p><p>In this article, I will walk you through the procedures on how to manually set up Laravel with the lamp stack. By the end of this article, you will not only have a working deployed Laravel application but also understand why we had to go through all of the procedures that we did.</p><h3>LARAVEL</h3><p>Laravel is a famous open-source PHP framework that offers resources and features for more effective web application development. Laravel’s MVC (Model-View-Controller) design creates a codebase that is easier to manage, test, and comprehend since it separates data, presentation, and logic.</p><h3>LAMP</h3><p>LAMP is a very popular software stack consisting of:</p><ul><li>Linux: Linux is the operating system that serves as the foundation of this stack. Linux is known for its security, reliability, and flexibility, making it easier to run various software and applications. Laravel can run on any operating system (macOS, Windows, etc.); however, Linux is commonly used.</li><li>Apache: Apache is a web server component of the lamp stack that serves web pages to users’ browsers. It accepts incoming requests from users’ browsers and delivers the necessary content. Laravel apps require PHP, and Apache is simple to set up and integrates well with PHP.</li><li>Mysql: MySQL is a popular relational database management system for storing, retrieving, and organizing data. Laravel’s ORM (Object-Relational Mapping) layer allows Mysql to interface with it easily.</li><li>PHP: PHP is a server-side scripting language that is used for developing web pages. Since Laravel is built with PHP, the LAMP stack is efficient for executing Laravel’s code and generating dynamic web content.</li></ul><h3>Prerequisites</h3><p>Here are the tools you need to be able to follow this tutorial.</p><ul><li>An Ubuntu operating system</li><li>The Root user or a user with sudo privileges</li><li>Command line or Terminal window</li><li>A mind ready to consume knowledge.</li><li>And of course a good internet connection.</li></ul><h3>SETTING UP THE LAMP STACK</h3><p>Of course, the first step is having an Ubuntu version of the Linux operating system to enable you to install the other components of the stack.</p><h4>STEP 1: Update Package Index</h4><p>Before downloading packages, make sure your Ubuntu machine’s package index is updated. I like to think of the package index as a local library that stores book editions, and updating the library means receiving the most recent editions of these books. In our scenario, we want to get the most recent version of these packages into the machine’s local library.</p><pre>sudo apt update</pre><p>The number of upgradeable packages is displayed as a result of this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0dw5UfDhZbFRCwsM" /></figure><h4>STEP 2: Install Apache2</h4><p>I am running a Vagrant Ubuntu virtual machine (VM) on top of my host PC. When utilizing a virtual machine, I normally recommend port forwarding. So I’m going to expose port 80 of my VM to port 8085 of my host system, allowing me to access my Apache webserver from my host machine’s port 8085.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MLApKu9aaD4zhzmI" /></figure><p>let us proceed with the Apache installation. I usually use the -y flag to automatically answer yes to any questions or confirmations that the package manager might offer throughout the installation process.</p><pre>sudo apt -y install apache2</pre><p>After successfully installing Apache, you can verify its version.</p><pre>apache2 -v</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vdz1LHGetTt9o0k0" /></figure><p>If Apache is installed correctly, you can access it from your VM’s IP address or your redirected port, which in my instance is <strong>localhost:8085</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TshKf2e9kD6ocLqG" /></figure><p>You can obtain your VM’s IP with the command:</p><pre>ip addr</pre><p>Services in Ubuntu normally start immediately. However, you can also check if Apache is running using:</p><pre>sudo systemctl status apache2</pre><p>You may also start Apache with:</p><pre>sudo systemctl start apache2</pre><h4>STEP 3: Install MySQL</h4><p>Command to install MySQL</p><pre>sudo apt -y install mysql-server</pre><p>Command to check the version of the installed MySQL</p><pre>mysql --version</pre><p>The command should produce the results shown in the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DoPyKwWJnEzXaRlK" /></figure><p>To see if MySQL is running, use the following command:</p><pre>sudo systemctl status mysql</pre><p>Mysql should start automatically when it is installed, but you can use the following instructions to start and enable it.</p><pre>sudo systemctl start mysql<br>sudo systemctl enable mysql</pre><h4>STEP 4: Install PHP</h4><p>Laravel works smoothly with PHP version 8.2, so be sure to install it.</p><p>I have noticed that Ubuntu 23.10 automatically installs PHP8.2, but older versions do not. To address this, we must add a PPA repository for PHP. This repository would provide us with access to many versions of PHP, including the desired version 8.2.</p><pre>sudo add-apt-repository ppa:ondrej/php -y</pre><p>After that, it’s common practice to update your package list to ensure that the system is completely aware of your new repository.</p><pre>sudo apt update</pre><p>Then you can specify which version of PHP you wish to download, which in our instance is version 8.2.</p><pre>sudo apt -y install php8.2</pre><p>Check the version of PHP to ensure you have the correct one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/0*HZk-N4lIyL3Ysk8r" /></figure><h4>STEP 5: Install git</h4><p>Since we wish to clone a Laravel repository on GitHub, we need git installed on our virtual machine. Some virtual machines come with git pre-installed, whereas others do not.</p><p>To check if git is available on your system, run;</p><pre>which git</pre><p>If you have git installed on your machine, you should see an executable path similar to the one shown in the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*STyC0Sli8K_nBUk5VLAbyg.png" /></figure><p>If you don’t already have Git, install it.</p><pre>sudo apt -y install git</pre><p>Also, check the version after installation.</p><pre>git --version</pre><h4>STEP 6: Install dependencies</h4><p>There are a few dependencies that are not included with the installation of PHP8.2, but are essential for the Laravel application and its accompanying components to function properly on the lamp stack. Below are the dependencies and what they accomplish:</p><ul><li>unzip: Is not a PHP dependency; it is a system utility for extracting files from compressed archives, also known as zip files. In the case of our Laravel repository, unzip will unpack the downloaded archive before placing it in the proper directory on our VM.</li><li>php-zip: Is used to manage zip archives within PHP. Laravel can use this dependency for file extraction and zip file creation (compression) during downloads and uploads.</li><li>php-mysql: Enables PHP scripts to interact with the MySQL database. This dependency is required since Laravel utilizes MySQL by default to organize and store data.</li><li>Php-xml: To clarify, XML is a markup language similar to HTML that defines a set of rules for encoding content in a format that humans and machines can understand. Laravel uses XML to parse configuration files and interface with web services that return XML data. So the php-xml extension is what enables Laravel to access such functions.</li><li>php-curl: Enables PHP to make HTTP requests via the curl library in its scripts. This allows Laravel to connect with external APIs and services while also retrieving data from remote URLs.</li><li>Php-mbstring: This extension provides PHP with extra tools for dealing with non-ASCII characters and multibyte character encodings (characters that are beyond the standard English alphabet). These include characters that can take up more than one byte of memory and are used in languages such as Chinese and Japanese. Laravel relies on these functions for activities such as string manipulation, text encoding, and decoding.</li></ul><p>Now that we understand why we are installing these dependencies, we can proceed to install them all at the same time.</p><pre>sudo apt install -y unzip php8.2-zip php8.2-mysql php8.2-xml php8.2-curl php8.2-mbstring</pre><p>You may encounter difficulties while attempting to install these dependencies, but as long as you have a stable internet connection and have followed the steps religiously, you should be fine. Just run the command again.</p><p>Other dependencies you might require are php8.2-dom, php8.2-json, php8.2-fileinfo, php8.2-filter, php8.2-hash, php8.2-openssl, php8.2-pcre, php8.2-pcre, php8.2-PDO, php8.2-session, php8.2-tokenizer and php8.2-ctype. I didn’t install these because they came with the PHP version I installed.</p><p>Run this command to confirm that all required dependencies have been successfully installed. This command would generate a list of your VM’s PHP dependencies.</p><pre>php -m</pre><h4>STEP 7: Download and setup Composer</h4><p>In the context of our goal, the composer is critical for managing the dependencies of Laravel apps, as well as extra PHP packages and libraries.</p><ul><li>It ensures that when new libraries and dependencies are released, they are compatible and up to date.</li><li>It simplifies code usage by creating an autoloader file (typically called vendor) that allows PHP to automatically locate and load classes from installed packages.</li><li>Makes it easier to configure projects and execute scripts.</li></ul><p>First, grab a copy of the composer setup file.</p><pre>php -r &quot;copy(&#39;http://getcomposer.org/installer&#39;, &#39;composer-setup.php&#39;);&quot;</pre><p>Command output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rS71BWRlWjtwvAEu" /></figure><p>Then, download the composer app from the setup file.</p><pre>php composer-setup.php</pre><p>This should be the outcome of downloading the app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2JYT0J21OALq4OCF" /></figure><p>Move the app to an executable path and rename it composer, so that you may run it from any directory or path, just like a normal Linux command.</p><pre>sudo mv composer.phar /usr/local/bin/composer</pre><p>Running the command in the figure below should confirm that the composer is now in an executable path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/0*gGjxJbMDQSHHJ0dV" /></figure><h4>STEP 8: Setup appropriate permissions and clone Laravel</h4><p>To ease the complexity of the next sets of steps, I would divide them into sub-steps.</p><p><strong>Step 8a: Remove the existing apache file been served on the browser</strong></p><p>I’d like to save my cloned Laravel page in the <strong>html</strong> directory. Let’s move there.</p><pre>cd /var/www/html</pre><p>For the time being, I don’t want anything to be in my <strong>html</strong> directory, so let’s completely empty it.</p><pre>sudo rm -r *</pre><p>The previous webpage supplied by that directory will no longer be available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EesLdGbxUae4Alit" /></figure><p><strong>Step 8b: change ownership of the html directory to your present user</strong></p><pre>sudo chown -R $USER:$USER /var/www/html</pre><p>In my situation, my user is vagrant, so as seen in the figure below, my html directory now has vagrant as user and group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/0*asfp-JrAWWaDkh9V" /></figure><p><strong>Step 8c: clone and configure laravel</strong>The first step is to ensure that you are on the correct Apache route where you intend to clone your Laravel repository. In my case, it’s <strong>var/www/html</strong>; therefore, I’ll change my directory accordingly.</p><pre>cd /var/www/html</pre><p>While working on this project, I initially had difficulty downloading my Laravel dependencies with composer. I then realized that it’s critical to clone your repo without using the sudo command, especially if you’re not using the <strong>root</strong> account. Using sudo automatically changes ownership of your cloned repository to root. Remember that we changed the ownership of the html directory to our current user, which in my case is <strong>vagrant</strong>. However, if your current user is <strong>root</strong>, simply add <strong>sudo</strong> to the beginning of the command below.</p><p>Here’s a link to the <a href="https://github.com/laravel/laravel.git">laravel repository</a>.</p><pre>git clone https://github.com/laravel/laravel.git</pre><p>Here’s what you should expect to see.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J0UcII0ENvsP9Xk7" /></figure><p>Certain versions of the Laravel repository include the composer.lock file, while others do not.</p><p>The <strong>composer.lock</strong> file locks down the precise versions of dependencies indicated in the <strong>composer.json</strong> file. So, every time you run composer install, you get the precise version of the dependencies, ensuring that there are no differences between the development and production environments.</p><p>If you’ve completed all of the procedures so far; running the next command will install both the <strong>composer.lock</strong> and <strong>vendor </strong>files.</p><p>Execute the next command within the Laravel directory.</p><pre>composer install</pre><p>The contents of the figure below should correspond to what you see now that composer.lock and the vendor file are available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UVQu-lP_gVNw5vQ4" /></figure><p>Loading our Laravel page in the browser should result in a directory listing like the one below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*92Bm0fcNDjUnhYhD" /></figure><p>Access the Laravel directory on the webpage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/0*IP0M2xy-DTlq1OAN" /></figure><p>Access the public directory. The resulting content, depending on your web browser, should show either a “blank page” or a “page not working” message.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*saJx7A74KmEazyCe" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IpBrbPKiAHk1L4-m" /></figure><p>This is due to your page’s inability to read environment variables. It’s not that the Laravel we’ve setup lacks environment variables; it’s just that they’re in the <strong>.env.example</strong> file, and this application has most likely been configured to read variables from the <strong>.env</strong> file.</p><p>The <strong>.env.example</strong> file serves as a template for the <strong>.env</strong> file. It includes setups and comments to help you set up environment variables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LBlC74XCpai4GRjQ" /></figure><p>We can use the <strong>mv</strong> command to rename <strong>.env.example</strong> to <strong>.env</strong>, or we can copy the contents of <strong>.env.example</strong> into <strong>.env</strong>. I would use the latter since it allows us to keep the original example file that comes with Laravel in case you want to compare the differences.</p><pre>cp .env.example .env</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9uFbUQgcGm24vav8" /></figure><p>With the correct file in there, our webpage would display different content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IGJ25da2SyB6Mmn7" /></figure><p>Hurray! We’ve got another problem.</p><p>Our web server (APACHE) is unable to view the Laravel logs due to permissions being refused. Laravel logs are kept in the storage directory, which is owned and managed by our current user. If the webserver needs to access the file, it must do so as the default Apache user and group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UAc9vqd5k0xGF3Qg" /></figure><p>Apache has a default user and group named <strong>www-data</strong>. We need to alter the ownership of the storage directory to the default for Apache.</p><pre>sudo chown -R www-data:www-data /var/www/html/laravel/storage</pre><p>The command above changes ownership of the storage directory, including its files and directories.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KSt7yE3WKUx3wQYM" /></figure><p>Let’s refresh our webpage and observe what happens next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6t2R6RiDqEG8_eaq" /></figure><p>Our software is missing a key, and it has provided us with the command to produce one.</p><p>Yes, we can generate the key, but why do we need to generate it?</p><p>The artisan key is an essential security component in Laravel. It protects the Laravel application’s data by encrypting and decrypting it, performing user authentication, and improving the overall integrity of the application.</p><p>The key is stored in the .env file when generated.</p><pre>vi .env</pre><p>You’ll notice the section designated for the key is empty.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/0*RX7C_a-Ylf91XeP4" /></figure><p>Running the command:</p><pre>php artisan key:generate</pre><p>Should generate the key.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/0*tg-s0w-LfDj_Ukew" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/0*GCyhEPbKAqlCV9pu" /></figure><p>Reloading the webpage should produce a different result now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wk7MLSUW2AJ8ChTX" /></figure><p>We are encountering this error because we have not set up the database for our Laravel application, which we will accomplish in the next session.</p><h4>STEP 9: Setup Database</h4><p>Accessing the <strong>.env </strong>file reveals that the database configuration has been commented out, and still needs to be configured properly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/0*GPgmB3XD2M5fEXKY" /></figure><p><strong>Step 9a: Create database and database user</strong></p><p>Run the following commands one after the other, replacing the database user, password, and database name with your preferred values.</p><pre>sudo mysql -uroot<br>CREATE USER &#39;goodness_user&#39;@&#39;localhost&#39; IDENTIFIED BY &#39;123456&#39;;<br>GRANT ALL PRIVILEGES ON *.* TO &#39;goodness_user&#39;@&#39;localhost&#39;;<br>CREATE DATABASE goodness_db_name;<br>SHOW DATABASES;<br>exit;</pre><p>If you did everything correctly, your database name will be available when you run the command to display the list of available databases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v82s1OlQeeJ6tvn7" /></figure><p><strong>Step 9b: populate your database details</strong></p><p>Replace the database, name, username, and password with those you created. Uncomment the configuration parameters and change the database connection to <strong>mysql</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/0*o4d4B1-L6gEY2WH0" /></figure><p><strong>Step 9c: Synchronize database schema to laravel application</strong></p><p>To confirm that the.env file and the Laravel application are in sync, use the command below.</p><pre>php artisan migrate</pre><p>If everything is done correctly, this should be the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VshlpyhfvexqDKgy" /></figure><p>Restart Apache.</p><pre>sudo systemctl restart apache2</pre><p>Your Laravel page should now be fully served.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J9GJ76-whiB0bQET" /></figure><h4>STEP 10: Reconfigure Apache2</h4><p>The Laravel application has been successfully deployed, however, there is one issue: the page is accessed through the directory listing <strong>/laravel/public</strong>. I prefer to access the page directly from my local host, virtual machine IP address, or assigned domain.</p><p>Navigate to the sites-available directory to find virtual host configuration files for individual Apache-hosted websites.</p><pre>cd /etc/apache2/sites-available/</pre><p>Create a new virtual host configuration file for the Laravel website.</p><pre>sudo vi laravel.conf</pre><p>Populate it with the following.</p><pre>&lt;VirtualHost *:80&gt;<br>ServerAdmin webmaster@localhost<br>DocumentRoot /var/www/html/laravel/public<br>ErrorLog ${APACHE_LOG_DIR}/error.log<br>CustomLog ${APACHE_LOG_DIR}/access.log combined<br>&lt;/VirtualHost&gt;</pre><p>Let’s break it down;</p><p><strong>&lt;VirtualHost *:80&gt;: </strong>This block simply indicates that the virtual host being established listens on port <strong>80</strong>, and the <strong>*</strong> sign specifies that the virtual host should reply to requests from any available IP address on our VM. <strong>&lt;/VirtualHost&gt;</strong> is simply the closing block.</p><p><strong>ServerAdmin webmaster@localhost: </strong>​​This specifies the email address of the server administrator, which is <strong>webmaster@localhost</strong> in our case. It serves as a contact for server notifications, including errors related to the server or our website.</p><p><strong>Document Root /var/www/html/laravel/public :</strong> This specifies the root directory from which Apache will load files for this virtual host. In this case, it comes from <strong>/var/www/html/laravel/public</strong>. When this is enabled, we will be able to view our site straight from the virtual machine’s IP address or the localhost I established.</p><p><strong>ErrorLog \${APACHE_LOG_DIR}/error.log: </strong>The entire directive specifies the path of the virtual host’s error log file. <strong>{APACHE_LOG_DIR}</strong> is an Apache’s environment variable that is used to represent the directory where Apache log files are stored. It’s all defined in Apache configuration.</p><p><strong>CustomLog \${APACHE_LOG_DIR}/access.log combined:</strong> This directive specifies the path to our virtual host’s access log file.</p><p>Exit the file and disable the current virtual host configuration file, which in my case is 000-default.conf.</p><pre>sudo a2dissite 000-default.conf</pre><p>Enable your newly configured virtual host.</p><pre>sudo a2ensite laravel.conf</pre><p>Then restart Apache.</p><pre>sudo systemctl reload apache2</pre><p>The generated webpage can now be visited via your virtual machine’s IP address, localhost, or your chosen domain, with no directory listing required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FHm6wA2Z3yl2mqtF" /></figure><h3>Conclusion</h3><p>We have gotten to the end of the tutorial; I hope it was helpful, and you now have a working Laravel website. I’ve attached the <a href="https://github.com/busybrainx99/laravel_task.git">GitHub</a> repository where I automated the entire process with bashscript and developed an Ansible playbook to run it remotely on my slave server. If you have any questions that need clarification, please reach out to me on <a href="http://linkedin.com/in/goodness-eboh">Linkedin</a> or <a href="https://twitter.com/busybrainx99">Twitter</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18bd6c3c6c92" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>