<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amina lawal on Medium]]></title>
        <description><![CDATA[Stories by Amina lawal on Medium]]></description>
        <link>https://medium.com/@aminalawal?source=rss-b72c689e97c0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Sbdt2_4CQlVyLQ1Au8QOOA.jpeg</url>
            <title>Stories by Amina lawal on Medium</title>
            <link>https://medium.com/@aminalawal?source=rss-b72c689e97c0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 15:53:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aminalawal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GCP Network Reliability: Going Beyond the Basics with Cloud Armor and Private Service Connect]]></title>
            <link>https://medium.com/google-cloud/gcp-network-reliability-going-beyond-the-basics-with-cloud-armor-and-private-service-connect-af2637d8aa72?source=rss-b72c689e97c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/af2637d8aa72</guid>
            <category><![CDATA[private-service-connect]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[gcp-security-operations]]></category>
            <category><![CDATA[google-cloud-armor]]></category>
            <dc:creator><![CDATA[Amina lawal]]></dc:creator>
            <pubDate>Wed, 28 Jan 2026 08:01:04 GMT</pubDate>
            <atom:updated>2026-01-29T01:12:11.568Z</atom:updated>
            <content:encoded><![CDATA[<p>Standard firewall rules and basic load balancing are often the first line of defense, but for enterprise-grade workloads, they are rarely the last. In a landscape where a single Layer 7 DDoS attack can result in massive egress costs and hours of downtime, <em>“good enough”</em> networking becomes a liability.</p><p>To achieve true business continuity, architects must move beyond simple connectivity and toward a <strong>Zero-Trust Network Posture</strong>. This involves securing two distinct traffic flows:</p><ol><li><strong>North-South (External):</strong> Protecting your public entry points from sophisticated web exploits and botnets.</li><li><strong>East-West (Internal):</strong> Eliminating the risks of the public internet and complex VPC peering when connecting to internal APIs and Google services.</li></ol><p>In this guide, we explore how to integrate <strong>Google Cloud Armor</strong> and <strong>Private Service Connect (PSC)</strong> to create a unified, high-reliability architecture that removes reliance on public IPs and legacy peering, ensuring your infrastructure is both invisible to attackers and resilient under pressure.</p><ul><li><strong>Cloud ArmorExternal Security Perimeter:</strong> Protects public-facing applications against Layer 7 DDoS attacks and common web exploits (WAF).Eliminates expensive, exhausting attacks <em>before</em> they hit your application layer, ensuring uptime and reducing operational cost.</li><li><strong>Private Service Connect (PSC)Internal Connectivity &amp; Security:</strong> Provides private, VPC-less access to Google APIs and other services. Drastically reduces your internal attack surface by removing the need for legacy VPNs, VPC Peering, and reliance on shared Google API VIPs.</li></ul><h3>Prerequisites</h3><p><strong>Before You Begin: The Baseline Architecture</strong></p><p>To follow this guide, you should have the infrastructure from our <a href="https://medium.com/google-cloud/cloud-native-reliability-101-building-an-auto-healing-web-app-on-gcp-with-migs-and-a-global-load-f71fa2c21d6c">previous tutorial</a> already deployed. Here is the technical state we are starting from:</p><ul><li><strong>VPC Network:</strong> A custom-mode VPC (e.g., web-app-vpc) with a subnet in your target region.</li><li><strong>Regional Managed Instance Group (MIG):</strong> A group of VMs (e.g., web-app-mig) running your web server, distributed across multiple zones for auto-healing.</li><li><strong>Global External Application Load Balancer:</strong> An HTTP(S) Load Balancer using the MIG as a backend service.</li><li><strong>Cloud NAT &amp; Router:</strong> Configured to allow your private VMs to access the internet for updates without having public IP addresses.</li><li><strong>IAM Permissions:</strong> Ensure you have Compute Network Admin and Security Admin roles assigned to your GCP account.</li></ul><h3>Part 1: Fortifying the Perimeter with Cloud Armor</h3><h3>The Problem: L7 DDoS and WAF Gaps</h3><p>Traditional network security focuses on Layers 3 and 4 (IPs and Ports). However, modern threats target the <strong>Application Layer (Layer 7)</strong>. Attackers use sophisticated, low-volume requests to bypass simple firewalls, probing for vulnerabilities like <strong>SQL Injection (SQLi)</strong> or <strong>Cross-Site Scripting (XSS)</strong>.</p><p>Google Cloud Armor acts as your distributed, global-scale defense, integrated directly into the Global Load Balancing infrastructure to stop these threats at the edge before they ever reach your servers</p><h3>Use Case: Protecting a High-Traffic Login API</h3><p>Imagine a critical login API exposed via a GCP Global External Application Load Balancer. This API is vulnerable to:</p><ol><li><strong>Exploitation:</strong> An attacker sending malicious SQL code in the username field. (WAF needed)</li><li><strong>Abuse/DDoS:</strong> A botnet overwhelming the API with 10,000 requests per minute to slow down legitimate users. (Rate Limiting needed)</li></ol><h3>Implementation Guide: Cloud Armor Policy Setup (Steps 1–4)</h3><p>Cloud Armor policies are attached directly to a Backend Service, providing granular control over the traffic destined for that service.</p><h4>Step 1: Policy Creation and Attachment</h4><p>The first step is creating the container for your rules and immediately attaching it to the application’s backend service. This ensures the rules are evaluated before traffic reaches your Compute Engine VMs or GKE pods.</p><ol><li><strong>Navigate to Cloud Armor:</strong> In the Google Cloud Console, use the search bar or the navigation menu to go to <strong>VPC Network Security</strong> &gt; <strong>Cloud Armor</strong>.</li><li>Select the <strong>Policies</strong> tab.</li><li>Click the blue button <strong>CREATE POLICY</strong>.</li></ol><ul><li>Policy Name: enterprise-ddos-waf-policy</li><li>Policy Type: Backend Service (Selected by default). Cloud Armor policies protect Backend Services.</li><li>Scope: Global (This is required to protect the Global External Application Load Balancer we set up in the previous article.)</li><li>Default action: Allow (Leave this as Allow. We will use specific rules to Deny bad traffic.)</li></ul><p>4. Click <strong>Next</strong> <strong>to</strong> <strong>Attach the Policy to the Backend Service</strong></p><p>This links the new policy to the traffic entry point in your application.</p><ol><li>On the <strong>Target</strong> section (Step 3 of 4 in the wizard):</li><li>Click <strong>Add target</strong>.</li><li>In the pop-up:</li></ol><ul><li><strong>Backend service:</strong> Select your Load Balancer’s backend service, which we named web-backend-service (or similar).</li></ul><p>4. Click <strong>Done</strong>, then click <strong>CREATE</strong> to finalize the policy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dViXz15xFsrWfeaQ.png" /></figure><h4>Step 2: Defense Against Exploits (WAF Rules)</h4><p>Instead of writing complex regex for every threat, Cloud Armor provides pre-configured WAF rules based on Google’s threat intelligence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pd8iFr69lneNNY1jTOx5pg.png" /></figure><p>The expression is simple but powerful, leveraging Google’s constantly updated threat intelligence. We will now add the first critical rule to protect against known application exploits (like SQL Injection).</p><ol><li><strong>Navigate to the Policy:</strong> After creation, you should be on the policy details page. If not, go back to the <strong>Policies</strong> tab and click on enterprise-ddos-waf-policy.</li><li>Select the <strong>Rules</strong> tab.</li><li>Click <strong>Add rule — </strong>While still in the <strong>Rules</strong> tab of your enterprise-ddos-waf-policy, click <strong>Add rule</strong>.</li></ol><ul><li><strong>Rule Name:</strong> deny-sqli-v33</li><li><strong>Description:</strong> “Blocks common SQL Injection patterns.”</li><li><strong>Priority:</strong> 1000</li><li><strong>Condition: </strong><br>- Under <strong>Mode</strong>, select <strong>Advanced mode</strong>.<br>- In the <strong>Match</strong> text area, paste the expression:</li></ul><pre>evaluatePreconfiguredWaf(&#39;sqli-v33-stable&#39;)</pre><p><strong>6. Action:</strong> Deny</p><p>7. <strong>Response Code:</strong> 403 Forbidden</p><p>8. Click <strong>Done</strong>, then <strong>Add rule</strong>.</p><p>This is the easiest way to leverage Google’s managed intelligence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*44gCd4bmRX1tPtL0.png" /></figure><h4>Step 3: Defense Against Volumetric Abuse (Custom Rate Limiting)</h4><p>This is the key to L7 DDoS defense. Rate limiting is the “secret sauce” for stopping Layer 7 DDoS attacks where a botnet tries to overwhelm your login API. We’ll set a strict limit to prevent a single abusive client from impacting the whole service.</p><ol><li>Now, let’s add the rate-limiting rule.<br>1. While still in the <strong>Rules</strong> tab, click <strong>Add rule</strong> again.<br>2. <strong>Rule Name:</strong> rate-limit-abuse <br>3. <strong>Description:</strong> “Rate limits abusive traffic based on IP address.” <br>4. <strong>Priority:</strong> 2000 <br>5. <strong>Condition:<br>- </strong>Under <strong>Mode</strong>, select <strong>Advanced mode</strong>.<br>- In the <strong>Match</strong> text area, paste the expression: true<br><em>(This means the rule will apply to all requests that reach this priority)<br></em>6. <strong>Action:</strong> Select <strong>Rate Limit</strong>.<br>- <strong>Rate Limit Threshold: <em>Requests</em>:</strong> 100<strong>, <em>Interval</em>:</strong> 60 seconds<br>- <strong>Enforcement Key:</strong> Select <strong>IP address</strong> from the dropdown.<br>- <strong>Rate Limit Action:</strong> Deny<strong><br>- Response Code:</strong> 429 Too Many Requests <br>7. Click <strong>Done</strong>, then <strong>Add rule</strong> — <strong>Priority:</strong> 2000</li><li><strong>Action:</strong> <strong>Rate Limit</strong> (set response code to 429 Too Many Requests).<br><strong><em>Note: Why 429?</em></strong><em> Using a 429 status code is a standard practice that tells legitimate clients they are moving too fast, while simply dropping 403s can sometimes confuse monitoring tools.</em></li><li><strong>Condition:</strong> true (applies to all traffic for this service).</li><li><strong>Threshold:</strong> 100 requests per 60 seconds.</li><li><strong>Enforcement Key:</strong> <strong>IP address</strong>.</li></ol><blockquote>💡Tip: The Enforcement Key</blockquote><blockquote>For public APIs, using IP Address is standard. For authenticated applications, consider using a HTTP Header (like a session token) or a Cookie as the enforcement key. This prevents an attacker from using many source IPs (e.g., a VPN) to bypass the limit while ensuring a single user session remains protected from abuse.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8Pw3iO51UY3r4dYG.png" /></figure><h4>Step 4: Geopolitical Controls</h4><p>For compliance or high-risk scenarios, you can block traffic from entire regions with a single rule. For global applications, you may need to block traffic from specific regions based on business needs or security mandates.</p><p>While still in the <strong>Rules</strong> tab, click <strong>Add rule</strong>.</p><ul><li><strong>Description:</strong> “Denies traffic from Russia (RU).”</li><li><strong>Priority:</strong> 3000</li><li><strong>Condition:<br>- </strong>Under <strong>Mode</strong>, select <strong>Advanced mode</strong>.<br>- In the <strong>Match</strong> text area, paste the expression:</li></ul><pre>origin.region_code == &#39;RU&#39;</pre><ul><li><strong>Action:</strong> Deny</li><li><strong>Response Code:</strong> 403 Forbidden</li><li>Click <strong>Done</strong>, then <strong>Add rule</strong>.</li></ul><p><strong>Example for blocking high-risk countries</strong></p><pre>gcloud compute security-policies rules create 3000 \<br>    --security-policy=enterprise-ddos-waf-policy \<br>    --expression=&quot;origin.region_code == &#39;RU&#39; || origin.region_code == &#39;CN&#39;&quot; \<br>    --action=&quot;deny-403&quot; \<br>    --description=&quot;Geographical Block based on security mandate&quot;</pre><p>By stacking high-priority <strong>WAF</strong> and <strong>rate-limiting</strong> rules, you create a robust, layered security perimeter that absorbs malicious traffic at the edge, reducing the load and risk on your core application infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bmfTkk6UW5JBATIT.png" /></figure><h3>Part 2: Securing Private Connectivity with Private Service Connect (PSC)</h3><h3>The Problem: Legacy Private Access is Suboptimal</h3><p>Historically, private access to Google APIs (like Cloud Storage or BigQuery) relied on <strong>Private Google Access (PGA)</strong>, which uses special, shared Google VIPs (199.36.153.4/30). While traffic stays internal, this still relies on a general internet gateway route, and firewall rules must be carefully managed.</p><p>Furthermore, connecting to a <em>partner</em> VPC or a service in another organization required complex, often IP-colliding <strong>VPC Peering</strong>.</p><p><strong>PSC is the modern solution for both.</strong> It adheres to a <strong>zero-trust</strong> networking model by allowing you to consume services using a standard, dedicated, internal IP address within <em>your</em> VPC.</p><h3>Use Case: VPC-less Access to Google APIs (Consumer Mode)</h3><p>The most common enterprise use case is to provide private, high-security access to Google services.</p><h4>Implementation Guide: PSC Endpoint for Cloud Storage</h4><p>We will create a PSC Endpoint that targets the Google-managed Service Attachment for Cloud Storage, effectively creating a dedicated internal conduit.</p><h4>Step 1: Reserve a Regional Internal IP Address</h4><p>Unlike PGA, PSC requires a standard, internal IP address within your VPC to act as the service gateway. This makes the Google API appear as just another private service in your network, like any other internal resource.</p><ol><li>Go to <strong>VPC Network</strong> &gt; <strong>IP addresses</strong>.</li><li>Click <strong>RESERVE INTERNAL STATIC ADDRESS</strong>.</li></ol><ul><li><strong>Name:</strong> psc-storage-ip</li><li><strong>IP version:</strong> IPv4</li><li><strong>Type:</strong> Select <strong>Internal</strong>.</li><li><strong>Network:</strong> Select your VPC network (e.g., default).</li><li><strong>Subnet:</strong> Select the subnet where your VMs are running.</li><li><strong>Purpose:</strong> Non-shared</li></ul><p>3. Click <strong>RESERVE</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Nnwj5QcwDEeoSRPJ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7Ypn5M_NwjHtbYDL.png" /></figure><h4>Step 2: Create the PSC Endpoint (Forwarding Rule)</h4><p>The endpoint is a special forwarding rule that maps your reserved internal IP to the Google-managed Service Attachment for Cloud services.</p><ol><li><strong>Target Service:</strong> Select <strong>“All Google APIs”</strong> or a specific service (e.g., Cloud Storage).</li><li><strong>IP Address:</strong> Use the internal IP reserved in Step 1.</li></ol><p><strong>Navigate:</strong> Go to <strong>VPC Network</strong> &gt; <strong>Private Service Connect</strong>.</p><ol><li>Select the <strong>Connected endpoints</strong> tab.</li><li>Click <strong>CONNECT ENDPOINT</strong></li><li><strong>Name: </strong>pscstorageendpoint</li><li><strong>Target: </strong>Select <strong>“Google APIs”</strong></li><li><strong>Service: </strong>Select <strong>Cloud Storage</strong> (<a href="http://storage.googleapis.com/">storage.googleapis.com</a>).</li><li><strong>Network: </strong>Select your VPC network.</li><li><strong>Subnet: </strong>Select your subnet.</li><li><strong>IP address: </strong>Select the reserved IP: psc-storage-ip. This connects the IP from Step 1 to the Google service.</li><li>Enable Global Access</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SDUHKJ7F6Fk-oVlv.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/0*Ko0q9BEjawCTu2x5.png" /></figure><p><strong>Step 3: Create the Private DNS Zone (DNS Redirection)</strong></p><p>This crucial step tells your VPC network to internally resolve the public API domain name (e.g., <a href="http://storage.googleapis.com/">storage.googleapis.com</a>) to the internal IP address of your PSC Endpoint. This allows your VMs to use the standard API name while keeping all traffic private.</p><p><strong>1. Create the Managed Private Zone: </strong>The Private Zone intercepts all requests destined for the Google API domain namespace within your VPC.</p><ul><li><strong>Navigate:</strong> Go to <strong>Cloud DNS</strong> in the GCP Console.</li><li>Click <strong>CREATE ZONE</strong>.</li><li><strong>Zone type:</strong> Select <strong>Private zone</strong>.</li><li><strong>Zone name:</strong> psc-google-apis-zone (Use a simple, descriptive name).</li><li><strong>DNS name:</strong> <a href="http://googleapis.com/">googleapis.com</a>. (This is the domain you are intercepting. <strong>Don&#39;t forget the trailing dot!</strong>).</li><li><strong>Networks:</strong> Select your VPC network</li><li>Click <strong>CREATE</strong>.</li></ul><p><strong>2. Add the DNS Record</strong></p><p>Since we created a PSC Endpoint specifically for <strong>Cloud Storage</strong>, we must map its fully qualified domain name (<a href="http://storage.googleapis.com/">storage.googleapis.com</a>) to the reserved internal IP address.</p><ul><li>Stay on the Cloud DNS page and click into your newly created zone: psc-google-apis-zone.</li><li>Click <strong>ADD STANDARD RECORD SET</strong>.</li><li><strong>DNS Name:</strong> storage (Since the zone is <a href="http://googleapis.com/">googleapis.com</a>., the full record will be <a href="http://storage.googleapis.com/">storage.googleapis.com</a>.).<br>- <em>💡 Note: If you had created a PSC Endpoint targeting </em><strong><em>“All Google APIs”</em></strong><em>, you would use the wildcard record </em>* or *.<a href="http://googleapis.com/">googleapis.com</a>. instead.</li><li><strong>Record type:</strong> <strong>A</strong></li><li><strong>TTL:</strong> Leave as default (e.g., 5 minutes).</li><li><strong>IPv4 Address:</strong> Enter the internal IP address of your PSC Endpoint (the one you reserved earlier)</li><li>Click <strong>CREATE</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*E_nIezKdCTFx5bgJ.png" /></figure><h4>Step 4: Verification (The “Aha!” Moment)</h4><p>We need to prove that an instance that cannot access the internet can now access Cloud Storage privately using the PSC endpoint.</p><ol><li><strong>Launch a Test VM (Optional but Best Practice):</strong> Launch a new VM in your network with <strong>NO External IP</strong> assigned.</li><li><strong>SSH into the VM.</strong></li><li><strong>Confirm Internal Resolution (The Proof):</strong> Run a DNS lookup for the storage API domain. The output must show the PSC Endpoint’s internal IP address, confirming the private DNS setup is working.</li></ol><pre># The output of this command is the true &quot;Aha!&quot; moment. <br>nslookup storage.googleapis.com</pre><p><strong>Expected Result:</strong> The Address field in the response should be your <strong>internal PSC IP</strong> (e.g., 10.10.10.5), not a public Google IP.</p><p>4. <strong>Test the Connectivity:</strong> Run the following command. The request should succeed because the traffic is internally routed via the PSC Endpoint.</p><pre># This command tries to access the Cloud Storage API endpoint. <br>curl -s -i https://storage.googleapis.com/</pre><p><strong>Expected Result:</strong> Since this endpoint is designed for API calls, you won’t get a website, but you should receive an HTTP response code that is <strong>NOT</strong> a timeout or a network error (e.g., HTTP/2 400 Bad Request or 404 Not Found). A 400-level code confirms the <strong>Cloud Storage API received your request</strong>.</p><h3>Advanced PSC: Service Attachment (Producer Mode)</h3><p>For the most advanced enterprise architectures, PSC is used to share services between organizations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*1lNPU1ygRWLN5mRXV0owvg.png" /></figure><p><strong>Key Benefit:</strong> These two VPCs communicate securely and privately without shared IP ranges (no peering) and with zero cross-organization administrative dependencies.</p><h3>Part 3: The Enterprise Network Security Posture: PSC and Cloud Armor Combined</h3><p>The true power of these features is realized when they are architected together.</p><ul><li><strong>Cloud Armor</strong> provides a high-security fence around the <strong>North-South (External)</strong> traffic.</li><li><strong>PSC</strong> locks down the <strong>East-West (Internal)</strong> and <strong>Outbound API</strong> traffic.</li></ul><h3>The Unified Zero-Trust Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oJV0pKa7s63T_8s4.png" /></figure><ul><li><strong>VPC A (Production):</strong> Hosts the main application.<br>- <strong>External Access (Inbound):</strong> Secured by <strong>Cloud Armor</strong> via a Global Load Balancer.<br>- <strong>Internal Access (Outbound):</strong> Connects privately to Google APIs via a <strong>PSC Endpoint</strong>.</li><li><strong>VPC B (Partner/Data Service):</strong> Hosts a data service shared with VPC A.<br>- <strong>External Access (Inbound):</strong> <em>No external access</em>.<br>- <strong>Internal Access (Inbound):</strong> Exposed only to VPC A via a <strong>Service <br>Attachment</strong> and consumed by VPC A using a <strong>PSC Endpoint</strong>.</li></ul><p>The shift from a “functioning” architecture to an “enterprise-ready” one is defined by how you handle the traffic you <em>don’t</em> expect. By integrating <strong>Cloud Armor</strong> as your global security perimeter and <strong>Private Service Connect</strong> as your internal backbone, you’ve done more than just connect services , you’ve built a <strong>Zero-Trust Network Posture</strong>.</p><p>This architecture removes nearly all reliance on public IPs, internet gateways, and complex peering configurations, leaving only highly secured, policy-enforced entry and exit points. This is the definition of <strong>next-generation network reliability</strong> on GCP.</p><h3>Next Steps: Moving Up the Maturity Model</h3><p>Building this infrastructure manually is the best way to understand the flow, but to truly scale, consider these next milestones:</p><ul><li><strong>Infrastructure as Code (IaC):</strong> Use <strong>Terraform</strong> to codify your Cloud Armor policies and PSC endpoints. This ensures that your security posture is version-controlled and reproducible across Dev, Staging, and Production environments.</li><li><strong>ML-Driven Defense:</strong> Explore <strong>Cloud Armor Adaptive Protection</strong>. It uses machine learning to analyze your unique traffic patterns and automatically suggests WAF rules to block potential L7 DDoS attacks before they escalate.</li><li><strong>Service Directory Integration:</strong> For complex environments, integrate PSC with <strong>Service Directory</strong> to manage service discovery at scale, making it even easier for your internal consumers to find and connect to private endpoints.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af2637d8aa72" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/gcp-network-reliability-going-beyond-the-basics-with-cloud-armor-and-private-service-connect-af2637d8aa72">GCP Network Reliability: Going Beyond the Basics with Cloud Armor and Private Service Connect</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud-Native Reliability: Building an Auto-Healing Web App on GCP with MIGs and a Global Load…]]></title>
            <link>https://medium.com/google-cloud/cloud-native-reliability-101-building-an-auto-healing-web-app-on-gcp-with-migs-and-a-global-load-f71fa2c21d6c?source=rss-b72c689e97c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f71fa2c21d6c</guid>
            <category><![CDATA[reliability]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[load-balancing]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Amina lawal]]></dc:creator>
            <pubDate>Mon, 17 Nov 2025 08:02:15 GMT</pubDate>
            <atom:updated>2025-11-20T17:38:49.730Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cloud-Native Reliability: Building an Auto-Healing Web App on GCP with MIGs and a Global Load Balancer.</h3><p>In this tutorial, we will build a <strong>production-ready, highly available</strong> web application on Google Cloud Platform. Our primary focus is achieving <strong>fault tolerance</strong> and <strong>auto-healing</strong> — meaning if a VM fails, it is automatically replaced, and traffic is instantly rerouted, ensuring minimal downtime.</p><p><strong>Why is this level of reliability necessary?</strong> In the real world, single points of failure are inevitable:</p><ul><li><strong>Zone Failures:</strong> A rare, localized power outage or network issue in a single data center <strong>(Zone)</strong> could take down all your services instantly. By distributing our app across multiple zones using a <strong>Regional Managed Instance Group (MIG)</strong>, we ensure the service remains operational even if an entire zone goes offline.</li><li><strong>Software Crashes/Corruptions:</strong> If a bug in your application or an automated configuration change causes the web server on a VM to crash or hang, the <strong>Load Balancer’s Health Check</strong> detects the failure and the <strong>MIG’s Auto-Healing</strong> feature immediately provisions a replacement. This prevents manual intervention and restores service in minutes.</li><li><strong>Traffic Spikes:</strong> While our focus here is on <em>healing</em>, a highly available setup is the foundation for <em>scaling</em>. When a flash sale or viral news post drives a sudden surge of users, having the infrastructure ready to handle the load across multiple instances ensures your application remains responsive and doesn’t buckle under pressure.</li></ul><p><strong>Key Components:</strong></p><ul><li><strong>VPC Firewall:</strong> Secure ingress for health checks.</li><li><strong>Instance Template (with Startup Script):</strong> Defines the application installation process.</li><li><strong>Managed Instance Group (MIG):</strong> Handles auto-healing and scaling.</li><li><strong>Global External Application Load Balancer:</strong> Provides a single, global entry point.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yJMMj3foOupGkwbtUjtFgQ.png" /><figcaption>Architectural diagram illustrating a highly available web application on Google Cloud Platform</figcaption></figure><h3>Prerequisites and Code Setup</h3><h4>The Demo Application</h4><p>We will use a simple Python Flask application. Clone the code from the dedicated repository.</p><blockquote><strong><em>Code Repository:</em></strong><em> </em><a href="https://github.com/Amiynarh/gcp-network-demo-app"><em>https://github.com/Amiynarh/gcp-network-demo-app</em></a></blockquote><p>The repository contains:</p><ul><li>main.py: The Flask app.</li><li>requirements.txt: Specifies dependencies.</li><li>startup-script.sh: The critical script that installs and runs the app on the VM.</li></ul><h3>Part 1: Configuring the Network and Compute</h3><h4>Step 1: Create the Firewall Rule for Health Checks</h4><p>The Load Balancer needs to ensure your VMs are alive. We must allow traffic from Google’s health check probe ranges on the app port (80). GCP’s load balancer probes use specific IP ranges (130.211.0.0/22 and 35.191.0.0/16) for health checks. You must explicitly allow ingress traffic on the application port (80) from these ranges.</p><p>To create the firewall rule via console, navigate to <strong>VPC Network</strong> &gt; <strong>Firewall</strong> &gt; <strong>Create firewall rule</strong></p><ul><li><strong>Name:</strong> fw-allow-lb-health</li><li><strong>Targets:</strong> Specified target tags</li><li><strong>Target tags:</strong> allow-health-check</li><li><strong>Source filter:</strong> IP ranges</li><li><strong>Source IP ranges:</strong> 130.211.0.0/22, 35.191.0.0/16</li><li><strong>Protocols and ports:</strong> Check <strong>Specified protocols and ports</strong> and enter tcp:80</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZLBPHz18YvlTmRpEBpdTuA.png" /><figcaption>Screenshot 1: The Firewall Rule Configuration, highlighting Source Ranges and Target Tags.</figcaption></figure><p>Or use the <strong>gcloud command</strong></p><pre>gcloud compute firewall-rules create fw-allow-lb-health \<br>    --network=default \<br>    --action=ALLOW \<br>    --direction=INGRESS \<br>    --source-ranges=130.211.0.0/22,35.191.0.0/16 \<br>    --target-tags=allow-health-check \<br>    --rules=tcp:80</pre><h4>Step 2: Create the Instance Template with Startup Script</h4><p>The template defines the VM configuration, including the software and startup script, allowing the MIG to create identical replicas. The template is the blueprint for all VMs in our deployment.</p><p>To set up the instance template — Navigate to <strong>Compute Engine</strong> &gt; <strong>Instance templates</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nSBHlESKCeJEpFITCLIv3g.png" /><figcaption>Screenshot 2: Instance Template Management Tab, showing the pasted startup script.</figcaption></figure><ol><li>Click <strong>Create instance template</strong>.</li><li><strong>Name:</strong> lb-backend-template</li><li><strong>Machine type:</strong> e2-micro (sufficient for the demo).</li><li><strong>Firewall:</strong> Check <strong>Allow HTTP traffic</strong> (or rely on the tags below).</li><li><strong>Network tags (under Networking):</strong> Add http-server, allow-health-check</li><li><strong>Management Section:</strong></li></ol><ul><li><strong>Automation:</strong> Paste the contents of startup-script.sh into the <strong>Startup script</strong> box.</li></ul><p><strong>gcloud Command</strong></p><p><em>(Note: Using </em><em>--metadata-from-file is cleaner for real-world setups)</em></p><pre>gcloud compute instance-templates create lb-backend-template \<br>    --machine-type=e2-micro \<br>    --image-family=debian-11 \<br>    --image-project=debian-cloud \<br>    --tags=http-server,allow-health-check \<br>    --metadata=startup-script=&quot;$(cat startup-script.sh)&quot;</pre><h4>Step 3: Create the Regional Managed Instance Group (MIG)</h4><p>The MIG uses the template to create and manage the VMs, ensuring auto-healing and high availability across zones.</p><p>Navigate to <strong>Compute Engine</strong> &gt; <strong>Instance groups</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7bmpb4p-jkJq3nHZur56zw.png" /><figcaption>Screenshot 3: MIG Configuration, focusing on the Metadata section.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O1AYNJkXelOopaNxX0ziNw.png" /><figcaption>Screenshot 4: MIG Configuration, focusing on the Port Mapping.)</figcaption></figure><ol><li>Click <strong>Create instance group</strong>.</li><li><strong>Group name:</strong> lb-backend-group</li><li><strong>Location:</strong> Select <strong>Regional</strong> (e.g., us-central1).</li><li><strong>Instance template:</strong> Select lb-backend-template.</li><li><strong>Autoscaling:</strong> Set minimum number of instances to <strong>2</strong>.</li><li><strong>Autohealing:</strong> Under <strong>Initial delay</strong>, set to 300 seconds (to allow app installation).</li><li><strong>Port Mapping (Crucial for LB):</strong> Under <strong>Configure ports</strong>, click <strong>Add item</strong> and map Port Name http to Port Number 80.</li></ol><p><strong>Note: Why 300 seconds?</strong> The <em>Initial delay</em> gives your application time to fully install and start the web server defined in your startup-script.sh before the health checker starts probing it. If the check begins too soon, the instance will be marked as <em>UNHEALTHY</em> (even though it&#39;s still starting up) and immediately replaced, wasting resources and causing an endless repair loop. We set it high enough to guarantee a successful first start.</p><p><strong>gcloud Command</strong></p><pre>gcloud compute instance-groups managed create lb-backend-group \<br>    --base-instance-name=lb-backend \<br>    --size=2 \<br>    --template=lb-backend-template \<br>    --region=us-central1 \<br>    --named-ports=http:80</pre><h3>Part 2: The Global External Application Load Balancer</h3><p>This component ties everything together, providing a single, reliable entry point.</p><h4>Step 4: Reserve a Global Static IP Address</h4><p>A global IP is required for the front-end of the global load balancer. In the console — Navigate to <strong>VPC network</strong> &gt; <strong>IP addresses</strong>.</p><ol><li>Click <strong>Reserve external static address</strong>.</li><li><strong>Name:</strong> lb-ipv4-1</li><li><strong>Network Service Tier:</strong> Premium (required for Global LB)</li><li><strong>Type:</strong> <strong>Global</strong></li></ol><blockquote><strong>Note: Why Premium Tier?</strong> The Premium Tier offers the <strong>highest performance</strong> and lowest latency because it uses Google’s global, private fiber network for the majority of the connection path, starting right from the user’s entry point. This is <strong>required</strong> for a <strong>Global External Load Balancer</strong> because you want a single IP that serves users worldwide using the fastest path possible. The Standard Tier uses the public internet for part of the path, which is not suitable for a global entry point.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wLBkPv6KM3uyyedndOm2zw.png" /><figcaption>Screenshot 5: Reserving a global static IP address</figcaption></figure><p><strong>gcloud</strong> <strong>Command</strong></p><pre>gcloud compute addresses create lb-ipv4-1 \<br>    --ip-version=IPV4 \<br>    --global</pre><h4>Step 5: Configure the Load Balancer</h4><p>Use the Load Balancing wizard to configure the three parts: Backend, Host Rules, and Frontend. Navigate to <strong>Network Services</strong> &gt; <strong>Load balancing</strong> &gt; <strong>Create load balancer</strong>.</p><ol><li>Select <strong>Application Load Balancer (HTTP/S)</strong> &gt; <strong>Public facing (external)</strong> &gt; <strong>Global External Application Load Balancer</strong>.</li><li>Click <strong>Configure</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dT-apCF78YTgu6j57fvs8g.png" /><figcaption>Loadbalancer Configuration</figcaption></figure><p><strong>A. Backend configuration</strong></p><ul><li><strong>Backend services &amp; backend buckets:</strong> Create a new <strong>Backend service</strong>.</li><li><strong>Name:</strong> web-backend-service</li><li><strong>Backend type:</strong> Instance group</li><li><strong>Instance group:</strong> Select lb-backend-group.</li><li><strong>Port numbers:</strong> 80</li><li><strong>Health check:</strong> Create a new check.</li><li><strong>Name:</strong> http-health-check</li><li><strong>Protocol:</strong> HTTP</li><li><strong>Request path:</strong> /healthz (This is the endpoint we wrote in the Flask app!)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TQySWTvus6NuDHzbcGIXDA.png" /><figcaption>Screenshot 6: The Load Balancer Health Check configuration, showing the use of the /healthz path.</figcaption></figure><p><strong>B. Frontend configuration</strong></p><ul><li><strong>Protocol:</strong> HTTP</li><li><strong>IP address:</strong> Select lb-ipv4-1</li><li><strong>Port:</strong> 80</li></ul><blockquote><strong>NOTE: </strong>Port 80 is the <strong>standard port for all HTTP (unencrypted web) traffic</strong>. When a user types a public IP address into their browser without specifying a port, the browser defaults to port 80. This is the global entry point for all client requests.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GHnQlWJ9VrtKiOxLtOWmHA.png" /><figcaption>Screenshot 7: The Load Balancer Health Check configuration, showing the use of the /healthz path.</figcaption></figure><p>Click <strong>Create and give your loadbalancer a name.</strong></p><h4>Step 6: Final Verification</h4><p>Wait 5–10 minutes for the load balancer to provision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Donz4Ocd_Xi_b-1BaVPLAQ.png" /><figcaption>Screenshot 8: The provisioning of Loadbalancer is successfuly and Backend healthy</figcaption></figure><p>Find the global IP address of the Load Balancer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lExxFbV7k7gUVyJwumRi5A.png" /><figcaption>Screenshot 9: The frontend page displaying the public IP</figcaption></figure><p>Paste the IP into a browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nx7gj1HUXjjfQYa6KXTu9Q.png" /><figcaption>Screenshot 10: The final web page result</figcaption></figure><h4>Expected Result</h4><p>You should see:</p><blockquote><em>“Hello from the </em><strong><em>GCP Network Reliability Demo Backend</em></strong><em>! Instance Name: </em><em>&lt;VM-HOSTNAME&gt;&quot;</em></blockquote><p>If you refresh the page, the VM hostname should change, confirming traffic is being load-balanced across your MIG instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MQf4MrwRpyPlYXpkFGxLJA.png" /><figcaption>Screenshot 11: The final web page result, showing another instance name</figcaption></figure><p>When we route /healthz for the health check request path. Using a dedicated health check endpoint is a standard practice for modern web apps. The /healthz endpoint in our Flask app is designed to return a simple <strong>HTTP 200 OK</strong> status code only if the application is fully running and ready to serve traffic. This tells the Load Balancer, &quot;Yes, I am alive and healthy,&quot; allowing the MIG to only direct traffic to ready instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EKJRSGNPCojsSaux27O7UA.png" /><figcaption>Screenshot 11: Displaying the healthy status of our endpoint</figcaption></figure><p>Congratulations! You have successfully deployed a <strong>highly available, auto-healing web application</strong> on Google Cloud Platform. By combining the power of the <strong>Global External Application Load Balancer</strong> and a <strong>Regional Managed Instance Group (MIG)</strong>, you achieved a resilient architecture. Traffic is distributed globally, and the built-in health check and auto-healing features ensure that if any backend VM fails, it is automatically replaced and traffic is immediately redirected, establishing a strong foundation for cloud-native reliability.</p><h3>Next Steps to Production</h3><p>To enhance this setup for a real-world production environment, consider these steps:</p><ul><li><strong>Implement HTTPS:</strong> Use a <strong>Target HTTPS proxy</strong> and <strong>Google-managed SSL certificates</strong> in your load balancer configuration to encrypt traffic.</li><li><strong>Enable Autoscaling:</strong> Configure the MIG to scale dynamically based on CPU utilization or HTTP load to handle unpredictable traffic spikes.</li><li><strong>Use GKE/Cloud Run:</strong> For more complex microservice deployments, migrate your application to <strong>Google Kubernetes Engine (GKE)</strong> or the serverless <strong>Cloud Run</strong> platform.</li></ul><h3>References</h3><ul><li>GCP documentation on Load Balancer Health Check IP Ranges: <a href="https://cloud.google.com/load-balancing/docs/health-check-concepts">https://cloud.google.com/load-balancing/docs/health-check-concepts</a></li><li>The Flask Demo Application Code Repository: <a href="https://github.com/Amiynarh/gcp-network-demo-app">https://github.com/Amiynarh/gcp-network-demo-app</a></li><li>GCP documentation on Instance Groups: <a href="https://docs.cloud.google.com/compute/docs/instance-groups">https://docs.cloud.google.com/compute/docs/instance-groups</a></li><li>GCP documentation on Instance Templates: <a href="https://docs.cloud.google.com/compute/docs/instance-templates">https://docs.cloud.google.com/compute/docs/instance-templates</a></li><li>GCP documentation on Load Balancing: <a href="https://cloud.google.com/load-balancing">https://cloud.google.com/load-balancing</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f71fa2c21d6c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/cloud-native-reliability-101-building-an-auto-healing-web-app-on-gcp-with-migs-and-a-global-load-f71fa2c21d6c">Cloud-Native Reliability: Building an Auto-Healing Web App on GCP with MIGs and a Global Load…</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Kubernetes with HKE: A Hands-on Journey]]></title>
            <link>https://medium.com/hostspaceng/exploring-kubernetes-with-hke-a-hands-on-journey-d40c54bd2502?source=rss-b72c689e97c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d40c54bd2502</guid>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Amina lawal]]></dc:creator>
            <pubDate>Fri, 06 Dec 2024 10:43:01 GMT</pubDate>
            <atom:updated>2024-12-06T10:43:01.341Z</atom:updated>
            <content:encoded><![CDATA[<p>Managing Kubernetes clusters can be complex and time-consuming. HKE <a href="https://hostspaceng.com/services/hke">(HostSpace Kubernetes Engine)</a> aims to simplify this process with automated cluster management, built-in monitoring, and intelligent optimization recommendations. Let’s explore how HKE streamlines the Kubernetes experience while providing robust management features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j3Nl9_PBTea1hPPPvdLyYA.png" /><figcaption>HostSpace Kubernetes Engine Console Page</figcaption></figure><h3>Prerequisites</h3><p>Before creating your first cluster, ensure you have:</p><ul><li>A server/instance ready for use</li><li>SSH key for secure access</li><li>Instance username</li><li>Basic understanding of Kubernetes concepts</li></ul><p><strong>Pro tip:</strong> Start by visiting the HKE Settings page to add your instances and SSH keys. This is particularly useful if you manage multiple infrastructure components.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vCVmpm0FHrIDHCBUH7Hyng.png" /></figure><h3>Creating Your First Cluster</h3><p>HKE’s approach to cluster creation is refreshingly straightforward. Unlike traditional Kubernetes setups that often require extensive configuration, HKE streamlines the process to a few essential steps:</p><ol><li>Choose your Kubernetes version</li><li>Select or add your instance IP</li><li>Provide SSH credentials</li><li>Name your cluster</li><li>Select your Kubernetes version (1.31 in my case)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1RGuWqA3t9xysduf7HUrCQ.png" /><figcaption>Cluster Home Page</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zzi_r17WpPDbTYdtzcB-tA.png" /><figcaption>Creating a Cluster using Saved Credentials (Instance IP &amp; SSH Keys)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U8_WJVWDEJd8h03y01wyzg.png" /><figcaption>Cluster Creation Completed</figcaption></figure><p>What stands out is the minimal input required — HKE handles the complex underlying configuration automatically.</p><h3>Connecting to Your Cluster</h3><p>Once your cluster is up, HKE provides straightforward connection instructions:</p><ol><li>Download the kubeconfig file</li><li>Configure kubectl using the provided commands</li><li>Verify connection with kubectl get nodes</li></ol><p>A useful tip: Make sure to use the complete context name when switching contexts, as it includes a unique identifier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jda7pcrumpWltwajOExfvQ.png" /><figcaption>Connection instructions from HKE UI</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fXF40AOn2TQfhg-bDOJOlA.png" /><figcaption>Running the commands &amp; Verifying connection by getting nodes in the cluster</figcaption></figure><h3>Deploying Applications from the Marketplace</h3><p>HKE comes with a built-in marketplace offering various applications. For this demonstration.</p><p>Before deploying ArgoCD, we first needed to set up the Nginx Ingress Controller from HKE’s Marketplace. This is a crucial component because:</p><p>— It acts as the traffic manager for your cluster<br>— Enables domain-based routing using .nip.io<br> — Allows external access to applications running in the cluster</p><ol><li>Navigate to the Marketplace tab</li><li>Locate Nginx Ingress Controller &amp; ArgoCD in the available applications</li><li>Click Install — HKE handles all the underlying complexity</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*atVkdMo16rRNKQA5U6-tbw.png" /><figcaption>HKE Marketplace</figcaption></figure><h3>Quick Verification with Port Forwarding</h3><p>Before setting up external access, it’s good practice to verify your deployment using port forwarding:</p><pre>kubectl port-forward svc/argocd-server -n argocd 8080:443</pre><p>This allows you to quickly access the UI at <a href="https://localhost:8080/">https://localhost:8080</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EyAc-pPslUqUPQ2__7NZRQ.png" /><figcaption>Generating password for ArgoCD &amp; Port Forwarding</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J7qOO3DIKiZT5M1B0nAO_g.png" /><figcaption>Local Host ArgoCD UI accessed with Port-forwarding</figcaption></figure><h3>Setting Up External Access</h3><p>In our case:</p><ol><li>HKE Marketplace automatically deployed the Nginx Ingress Controller</li><li>Created ingress rules for ArgoCD</li><li>Configured domain access using .nip.io for convenient URL access</li></ol><p>Since we’re running in an environment without a LoadBalancer and we didn’t set up MetalLb, we needed to switch to NodePort:</p><pre>kubectl patch svc nginx-ingress-ingress-nginx-controller -n default -p &#39;{&quot;spec&quot;: {&quot;type&quot;: &quot;NodePort&quot;}}&#39;</pre><p>This change allows external traffic to reach our ingress controller through a specific port on the node.</p><p>After configuring NodePort, access ArgoCD through.</p><pre>http://argocd.YOUR-NODE-IP.nip.io:NODEPORT</pre><p>Where <strong>NODEPORT</strong> is the port assigned by Kubernetes (in our case, 31043)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sGG8OdEms9r_4KGQ5PyMig.png" /><figcaption>Successfully deployed ArgoCD interface</figcaption></figure><h3>Security Considerations</h3><p>For production environments, consider these additional security enhancements:</p><ul><li>Enable SSL redirect</li><li>Configure backend protocol as HTTPS</li><li>Implement proper certificate management</li></ul><h3>What Makes HKE Stand Out</h3><ol><li><strong>Simplified Management</strong>: The one-click deployment of complex applications like ArgoCD shows how HKE abstracts away Kubernetes complexity</li><li><strong>Intelligent Monitoring</strong>: Built-in resource tracking, security monitoring, and advisory systems provide comprehensive cluster insights</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e0h2lr8XTwKBkxKV27dudw.png" /><figcaption>Security Events</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dBp5vncYg6lNq6tt_XW_WQ.png" /><figcaption>HKE Advisory</figcaption></figure><p><strong>3. Automated Best Practices</strong>: From ingress configuration to resource optimization recommendations, HKE guides you toward Kubernetes best practices</p><p><strong>4. Clear Visibility</strong>: The dashboard provides immediate insights into cluster health, resource usage, and potential optimizations</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yE9bnTwRGZ72KrsVe5eRjA.png" /><figcaption>General Cluster Dashboard</figcaption></figure><p>HKE simplifies Kubernetes deployment and management while maintaining flexibility for customization. The marketplace feature particularly stands out, making application deployment a streamlined process.</p><p>Key takeaways:</p><ul><li>Easy cluster creation and management</li><li>Rich marketplace of applications</li><li>Automated ingress configuration</li><li>Flexibility to adapt to different infrastructure requirements</li></ul><p>The platform’s ability to automatically handle complex tasks while providing comprehensive monitoring and optimization recommendations makes it a compelling choice for teams looking to streamline their Kubernetes operations.</p><p>Get started <a href="https://ui.hostspacecloud.com/">here</a> today!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d40c54bd2502" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hostspaceng/exploring-kubernetes-with-hke-a-hands-on-journey-d40c54bd2502">Exploring Kubernetes with HKE: A Hands-on Journey</a> was originally published in <a href="https://medium.com/hostspaceng">HostSpace Cloud Solutions</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Advanced Secrets Management for CI/CD Pipelines with Doppler and GitHub Actions]]></title>
            <link>https://medium.com/@aminalawal/advanced-secrets-management-for-ci-cd-pipelines-with-doppler-and-github-actions-24685d0e6620?source=rss-b72c689e97c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/24685d0e6620</guid>
            <category><![CDATA[doppler]]></category>
            <category><![CDATA[secrets]]></category>
            <category><![CDATA[doppler-secret-manager]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <dc:creator><![CDATA[Amina lawal]]></dc:creator>
            <pubDate>Tue, 16 Apr 2024 10:14:20 GMT</pubDate>
            <atom:updated>2024-04-16T10:14:20.986Z</atom:updated>
            <content:encoded><![CDATA[<p>In the dynamic field of software development, integrating effective tools is crucial for maintaining a secure and efficient workflow. <strong>Doppler</strong> is a secrets management platform that helps teams securely store and manage their environment variables and other sensitive data. <strong>GitHub Actions</strong> is a CI/CD solution that automates workflows to run tasks, such as testing and deploying code, directly from GitHub. Together, they provide a robust solution for automating deployments and managing secrets securely across multiple repositories. This integration not only simplifies the management of secrets but also enhances the security and consistency of your deployment processes.</p><h3>Why Use Doppler with GitHub Actions?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sg6rEN-pAhagkHOtJzotUA.png" /></figure><p>Doppler provides a seamless way to integrate secrets management into CI/CD processes, significantly enhancing the security and efficiency of these pipelines. By centralizing secret storage and tightly controlling their distribution, Doppler prevents the accidental exposure of sensitive data across development stages, from testing to production. This integration ensures that sensitive data, such as API keys and credentials, are managed securely and are only accessible to authorized systems and personnel.</p><h3>Getting Started with Doppler and GitHub Actions</h3><h3>Prerequisites</h3><p>Before you begin integrating Doppler with GitHub Actions, ensure you have the following:</p><ul><li><strong>A GitHub Account: </strong>You should have owner access to one or more repositories. <a href="https://github.com/join">Create a GitHub account</a> if you don’t have one.</li><li><strong>A Doppler Account:</strong> Necessary for managing and syncing secrets across your projects. Sign up for Doppler.</li><li><strong>Familiarity with GitHub Actions:</strong> Basic understanding of how to configure and manage workflows. For a quick overview, refer to the <a href="https://docs.github.com/en/actions">GitHub Actions documentation</a>.</li></ul><h3>1. Integrating Your Repositories with Doppler</h3><h3>Step 1: Doppler Setup</h3><p>For each repository you wish to integrate, follow these steps:</p><ul><li>Log into your Doppler dashboard.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CSmwQihNDTsIKKZXorar0g.png" /></figure><ul><li><strong>Create a Project:</strong> This will be your central space for managing secrets. Set up a new project for each application or service you deploy using GitHub Actions. Organize your secrets within this project by environment (development, staging, production) to streamline access and enhance security.</li><li><strong>Select “Integrations”</strong> and choose GitHub as your integration service.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yp0NlussfPAtZ8NWsYf-zw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yrhqFp4tTnvmxM-fbeYc3A.png" /></figure><ul><li>Follow the prompts to authorize Doppler to access your GitHub repositories.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9MRUmg22k-rgb5Gk2IBxLA.png" /></figure><ul><li>Choose a repository to integrate and configure the secrets you want to sync.</li></ul><h3>2. Configuring GitHub Actions Workflows</h3><p>For each repository, create or ensure a .github/workflows directory exists at the root with a YAML file defining your CI/CD workflow. This file should specify the actions to be taken (e.g., build, test, deploy) and configure triggers for repository_dispatch or workflow_dispatch events, depending on your operational needs.</p><p><strong>Repository Dispatch: </strong>Triggers workflows on custom repository events.</p><p><strong>File:</strong><strong>repository_dispatch</strong>.<strong>yaml</strong></p><pre>name: Dispatch Event for Doppler<br>on:<br>  repository_dispatch:<br>    types: [deploy_event]<br>    <br>jobs:<br>  process_dispatch:<br>    runs-on: ubuntu-latest<br>    steps:<br>    - name: Checkout code<br>      uses: actions/checkout@v4<br><br>    - name: Run Custom Script<br>      run: |<br>        echo &quot;I worked!&quot;</pre><p><strong>Workflow Dispatch:</strong> Allows you to run workflows manually or via API calls.</p><p><strong>File:</strong><strong>workflow_dispatch.yaml</strong></p><pre>name: Handle Dispatch Event for Doppler<br>on:<br>  workflow_dispatch:<br><br>jobs:<br>  process_dispatch:<br>    runs-on: ubuntu-latest<br>    steps:<br>    - name: Checkout code<br>      uses: actions/checkout@v4<br><br>    - name: Run Custom Script<br>      run: |<br>        echo &quot;I worked!&quot;</pre><ul><li><strong>repository_dispatch </strong>is ideal for automated scenarios where external events (like a tool or service hook) trigger the workflow.</li><li><strong>workflow_dispatch</strong> provides flexibility to trigger workflows manually or programmatically, offering control when deployments are more sensitive or require timing.</li></ul><h3>3. Setting Up Webhooks in Doppler</h3><p>Doppler webhooks will trigger your GitHub Actions workflows upon changes to your secrets.</p><h3>Creating a Webhook for Each Repository</h3><ul><li>In Doppler, navigate to the “Webhooks” section within your project’s settings.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4hLB6tf1v9UDWuv3DgXuUg.png" /></figure><h3><strong>Configuration Steps:</strong></h3><ol><li><strong>URL: </strong>Set this to the GitHub Actions trigger endpoint.</li></ol><ul><li>For repository_dispatch, use:</li></ul><pre>https://api.github.com/repos/OWNER/REPO_NAME/dispatches</pre><ul><li>For workflow_dispatch, adjust the URL to target the specific workflow within the repository. Please note that the WORKFLOW_ID is the name of the workflow; eg main.yaml</li></ul><pre>https://api.github.com/repos/OWNER/REPO/actions/workflows/WORKFLOW_ID/dispatches</pre><p>2. <strong>Authentication:</strong> Ensure each webhook is authenticated with a GitHub Personal Access Token (PAT) that has repo workflow permissions.</p><ul><li>Choose “Bearer Token” for authentication.</li><li>Input the PAT you generated.</li></ul><p><strong>3. JSON Payload:</strong></p><ul><li>The payload varies based on the event type (repository_dispatch or workflow_dispatch) and the data your workflow expects.</li><li>For repository_dispatch:</li></ul><pre>{<br>  &quot;event_type&quot;: &quot;deploy_event&quot;,<br>  &quot;client_payload&quot;: {<br>    &quot;unit&quot;: false,<br>    &quot;integration&quot;: true<br>  }<br>}</pre><ul><li>For workflow_dispatch:</li></ul><pre>{<br>  &quot;ref&quot;: &quot;develop&quot;<br>  }<br>}</pre><blockquote>Adjust the &quot;ref&quot;, and&quot;event_type&quot; as needed for your specific workflows. The “ref” is to reference the branch name the workflow is on</blockquote><p><strong>Common Issues:</strong></p><ul><li><strong>Webhook Failures:</strong> Ensure the PAT has appropriate permissions.</li><li><strong>Workflow Errors:</strong> Verify YAML syntax and ensure all required secrets are correctly referenced in Doppler.</li></ul><h3>4. Testing Your Integration</h3><p>After configuring webhooks in Doppler, test the integration by updating a secret in Doppler and verifying that it triggers the appropriate workflow in GitHub Actions. Check the “Actions” tab in your GitHub repository for execution details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b2Bct3CMuBUY81NYUHhN2g.png" /></figure><p>Actions triggered with both workflow &amp; repository triggers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dg_C8ZUydiZPtT44prrjGg.png" /></figure><h3>Conclusion</h3><p>Integrating Doppler with GitHub Actions across multiple repositories enhances your CI/CD pipelines by automating deployments and securing secrets. By following the steps outlined above, you can ensure consistent, secure, and efficient workflows across all your development projects.</p><p>Remember to maintain security best practices by managing access permissions carefully and regularly reviewing your integration setup for any potential improvements.</p><h3><strong>Resources</strong></h3><ol><li><a href="https://docs.github.com/en/rest/actions/workflows?apiVersion=2022-11-28#create-a-workflow-dispatch-event">Create a workflow dispatch event</a></li><li><a href="https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#create-a-repository-dispatch-event">Create a repository dispatch event</a></li><li><a href="https://docs.doppler.com/docs/webhooks">Creating Doppler Webhooks</a></li><li><strong>Doppler GitHub Actions Documentation:</strong> <a href="https://docs.doppler.com/docs/github-actions">https://docs.doppler.com/docs/github-actions</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=24685d0e6620" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying Actions Runner Controller for Cost-Efficient GitHub Actions]]></title>
            <link>https://medium.com/@aminalawal/deploying-actions-runner-controller-for-cost-efficient-github-actions-bdbd7bf0549e?source=rss-b72c689e97c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdbd7bf0549e</guid>
            <category><![CDATA[helm]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Amina lawal]]></dc:creator>
            <pubDate>Sun, 21 Jan 2024 14:02:17 GMT</pubDate>
            <atom:updated>2024-01-21T14:02:17.091Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GCjDkLLKvR63gZTV368BUw.png" /></figure><p><strong>Introduction</strong></p><p><em>GitHub Actions</em> is a powerful tool for automating workflows within your software development lifecycle. While GitHub offers hosted runners to execute these workflows, they can incur significant costs, especially for frequent or resource-intensive jobs. <em>Actions Runner Controller (ARC)</em> emerges as a solution to manage <a href="http://Differences between GitHub-hosted and self-hosted runners">self-hosted runners</a> on your Kubernetes cluster, offering cost savings, autoscaling capabilities, and seamless integration with GitOps practices.</p><p><strong>Why Use ARC?</strong></p><ul><li>Cost-Effectiveness: Self-hosted runners often cost less than hosted runners, especially for long-running or high-resource jobs. ARC allows you to leverage your existing infrastructure, potentially saving you money.</li><li>Autoscaling: ARC automatically scales runners up and down based on demand, ensuring efficient resource utilization and avoiding unnecessary costs.</li><li>Increased Security: GitHub self-hosted runners can be configured to run on your servers, which provides an extra layer of security compared to using shared runners. This helps to protect sensitive information and data in your workflows.</li><li>Greater Control: Self-hosted runners give you complete control over the resources and environment used for your workflows, which can help you optimize performance and ensure that your builds and deployments run smoothly.</li></ul><p>In this tutorial, we will show you how to set up GitHub’s ARC on Kubernetes using Helm, then create and run a workflow that uses runner scale sets. For more information about runner scale sets, see “<a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#runner-scale-set">Deploying runner scale sets with Actions Runner Controller</a>.”</p><p><em>If you are looking to secure your Actions Runner Controller (ARC) environment, </em><a href="https://hostspaceng.com/"><em>get in touch</em></a><em> with Hostspace.</em></p><h3>Step-by-Step Guide</h3><p><strong>Prerequisites</strong></p><ul><li>Basic understanding of Kubernetes, Git, and GitHub Actions</li><li>A Kubernetes cluster with ArgoCD installed</li><li>Helm installed on your local machine</li></ul><ol><li><strong>Prepare the ARC Charts Repository:</strong></li></ol><ul><li><strong>Create a New Repository<br></strong>- You need to create a dedicated repository to store the Helm charts for this specific deployment. You can create this repository on GitHub within your organization or on any other Git hosting platform you prefer.<br>- For clarity, let’s assume you call this new repository gh-arc-scale-set-charts.</li><li><strong>Clone the Base Repository<br></strong>- Next, clone the <a href="https://github.com/actions/actions-runner-controller/tree/master">actions/actions-runner-controller</a> repository to your local machine. This repository contains the base files needed for deploying ARC. We only need the below folders.</li></ul><pre>#Link to the gha-runner-scale-set-controller folder<br>https://github.com/actions/actions-runner-controller/tree/master/charts/gha-runner-scale-set-controller<br><br>#Link to gha-runner-scale-set-controller folder<br>https://github.com/actions/actions-runner-controller/tree/master/charts/gha-runner-scale-set</pre><p>- Now, you need to copy the contents of the cloned actions-runner-controller directory into the local directory of your newly created gh-arc-scale-set-charts repository.<br>- Finally, push the local gh-arc-scale-set-charts repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CQyLOBhOGKgJJ1p0ZeI9Dg.png" /></figure><p><strong>2. Create ArgoCD Applications:</strong></p><p>Once you’ve prepared your custom gh-arc-scale-set-charts repository, it’s time to tell ArgoCD how to deploy and manage your ARC components. This involves creating two ArgoCD applications, one for the gha-runner-scale-set-controller and one for the gha-runner-scale-set.</p><p><strong>Recommended Approach: Using a Connected Git Repository</strong></p><p>Use the provided manifest files (gha-runner-scale-set-controller.yaml and gha-runner-scale-set.yaml), adjusting values as needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M9KLxsbTAPaK1S6NqQ5Iow.png" /><figcaption>Create the gha-runner-scale-set-controller Application</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9n5-OikL154BWN16QtHCtQ.png" /></figure><ul><li>name: This defines the name of your ArgoCD application, in this case, gha-runner-scale-set-controller.</li><li>namespace: This specifies the namespace where the application resources will be deployed. In this example, it&#39;s argocd.</li><li>destination: This defines the Kubernetes cluster and namespace where the resources will be deployed. Here, it points to the arc-systems namespace in the default Kubernetes cluster.</li><li>project: This defines the ArgoCD project the application belongs to. Usually, you&#39;ll use the default project.</li><li>source: This section specifies the location of the manifest files.</li><li>repoURL: Replace this with the URL of your gh-arc-scale-set-charts repository.</li><li>targetRevision: This specifies the Git revision to use. Typically, you&#39;ll use main.</li><li>path: This points to the specific directory within the repository containing the manifest files. In this case, it&#39;s ./gha-runner-scale-set-controller.</li><li>syncPolicy: This defines how ArgoCD will manage the application&#39;s state.</li><li>automated: Enables automatic syncing with the Git repository.</li><li>prune: Removes resources that are no longer present in the repository.</li><li>selfHeal: Attempts to fix any discrepancies between the desired and actual state.</li><li>syncOptions: Additional options for syncing. In this case, CreateNamespace=true ensures the arc-systems namespace is created if it doesn&#39;t exist.</li><li>githubConfigUrl: Specifies the URL of your GitHub repository, organization, or enterprise where the runners will operate.<br>- Example: <a href="https://github.com/your-organization">https://github.com/your-organizatio</a>n</li><li>githubConfigSecret: References a Kubernetes secret containing credentials for authenticating with GitHub.<br>- To enable ARC to authenticate to GitHub, generate a personal access token (classic) or GitHub App. For more information, see <a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#deploying-using-personal-access-token-classic-authentication">Authenticating to the GitHub API</a>.</li><li>maxRunners: Sets the maximum number of runners that can be scaled up in the cluster. Example: 5</li><li>minRunners: Specifies the minimum number of runners that should always be available. Example: 1</li><li>controllerServiceAccount: Configures the service account used by the ARC controller to manage runners.<br>- namespace: Identifies the namespace where the service account resides (usually arc-systems).<br>- name: Specifies the name of the service account (usually actions-runner-controller-gha-rs-controller).</li></ul><p><strong>Key Point:</strong></p><ul><li>Carefully adjust these values to match your specific environment and requirements.</li><li>Ensure the githubConfigSecret exists in your Kubernetes cluster with the appropriate credentials.</li></ul><p><strong>3. Authentication Configuration:</strong></p><p>In the gha-runner-scale-set Helm values file, choose your authentication method:</p><ul><li><strong>PAT:</strong> Use a personal access token (PAT) with appropriate permissions.</li><li><strong>GitHubConfigSecret:</strong> Create a Kubernetes secret containing GitHub app credentials</li></ul><p><em>To enable ARC to authenticate to GitHub, generate a personal access token (classic) or GitHub App. For more information, see </em><a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#deploying-using-personal-access-token-classic-authentication"><em>Authenticating to the GitHub API</em></a><em>.</em></p><ul><li>Reference the secret in the githubConfigSecret field of the Helm values.</li></ul><p><strong>5. Deploy the Applications:</strong></p><p>If you’ve wisely connected your gh-arc-scale-set-charts &amp; your manifests repository to ArgoCD, the deployment process is remarkably streamlined! ArgoCD will automatically detect changes in your repository and initiate syncs without requiring manual commands.</p><p>Here’s a brief overview for those who haven’t connected their repository yet:</p><ul><li>Manual Sync (If Not Using GitOps):</li><li>Use the following commands to manually trigger the initial sync:</li></ul><pre>argocd app sync gha-runner-scale-set-controller<br>argocd app sync gha-runner-scale-set</pre><p><strong>6. Verify Deployment:</strong></p><ul><li>Check if the ARC controller and runner pods are running in your cluster.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o6hwwMfF0kwLQK8ja4fGNA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3xszgSPK9hY6Vsb_relMXw.png" /></figure><p>7. <strong><em>Actions Workflows to Test Custom Runners:</em></strong></p><p>Once you’ve successfully deployed your Actions Runner Controller (ARC) with custom runners using ArgoCD, it’s essential to verify that everything works as expected. This section guides you through creating a sample workflow to test your setup.</p><p>Before that, let’s <a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/monitoring-and-troubleshooting-self-hosted-runners#checking-the-status-of-a-self-hosted-runner">Check the status of our deployed self-hosted runner</a>.</p><p>A self-hosted runner can be located in either your repository, organization, or enterprise account settings on GitHub. To manage a self-hosted runner, you must have the following permissions, depending on where the self-hosted runner was added:</p><ul><li><strong>User repository:</strong> You must be the repository owner.</li><li><strong>Organization: </strong>You must be an organization owner.</li><li><strong>Organization repository: </strong>You must be an organization owner, or have admin access to the repository.</li></ul><ol><li>In your organization or repository, navigate to the main page and click Settings.</li><li>In the left sidebar, click Actions, then click Runners.</li><li>Under <strong>“Runners”</strong>, you can view a list of registered runners, including the runner’s name, labels, and status.<br>The status can be one of the following:<br>- <strong>Idle</strong>: The runner is connected to GitHub and is ready to execute jobs.<br>- <strong>Active:</strong> The runner is currently executing a job.<br>- <strong>Offline:</strong> The runner is not connected to GitHub. This could be because the machine is offline, the self-hosted runner application is not running on the machine, or the self-hosted runner application cannot communicate with GitHub.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U4kwHU9Jzv16pRVWAV4Bjw.png" /></figure><p>Since our runner is online, let’s test it by creating a workflow to test our setup.</p><p><strong>Here’s the process:</strong></p><ul><li><strong>Access Your Workflow File: <br></strong>- Navigate to the GitHub repository where you want to create the test workflow.<br>- Create a new file named .github/workflows/test-custom-runners.yml (or any preferred name).</li><li><strong>Define the Workflow:</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JHQlltnHNaRnyp57tGvIMw.png" /></figure><ul><li><strong>Monitor the Workflow:<br></strong>The workflow will automatically execute on the specified runner.<br>You can track its progress and output in the “Actions” tab of your repository.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nhjv90YZ3jRdlG-QEaEIAA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OSomVOiOp7XHkC4Ec6mobg.png" /></figure><p><strong>Tips and Troubleshooting for Deploying Actions Runner Controller with ArgoCD</strong></p><p><strong>Tips:</strong></p><ul><li><strong>Use Secrets for Credentials:</strong> Store GitHub access tokens or App credentials in Kubernetes Secrets for enhanced security and easier management.</li><li><strong>Test and Validate your Manifests</strong>: Ensure your manifests are correct before syncing them to your cluster. Consider using tools like Helm lint or Kustomize for validation.</li><li><strong>Monitor ArgoCD and Kubernetes Logs:</strong> Keep an eye on ArgoCD UI and Kubernetes logs for deployment progress and potential issues.</li><li><strong>Start Small and Scale Gradually:</strong> Begin with a minimal runner scale set and gradually increase based on your workload demands.</li><li><strong>Utilize GitOps Best Practices:</strong> Leverage the power of Git for version control, collaboration, and automated deployments.</li></ul><p><strong>Troubleshooting:</strong></p><ul><li><strong>Connection Errors:<br></strong>- Verify that ArgoCD can access your Kubernetes API server.<br>- Double-check the URL and namespace settings in your manifests.</li><li><strong>Resource Creation Failures:<br></strong>- Ensure the required Kubernetes resources (namespaces, service accounts, etc.) exist.<br>- Check for permission errors related to the service account used by the controller.</li><li><strong>ArgoCD Sync Failures:<br></strong>- Review the ArgoCD UI for detailed error messages.<br>- Consult the ArgoCD documentation for specific troubleshooting steps.</li></ul><p><strong>Conclusion</strong>:</p><p>ARC empowers you to take control of GitHub Actions runner costs and management while benefiting from autoscaling, declarative configuration, and GitOps integration. By following these steps and understanding the key concepts, you can successfully deploy ARC in your Kubernetes environment and optimize your GitHub Actions workflows.</p><p>This doesn’t end here, <strong>there will be next steps</strong> after the successful deployment</p><ul><li><strong><em>Creating custom images for actions-runner-controller:</em></strong> Now that we have <a href="https://github.com/actions/actions-runner-controller">actions-runner-controller</a> up and running, we need to think through the runner image some. You would want to install dependencies on the image eg Docker, Yarn, etc. — <strong><em>Article coming soon</em></strong></li></ul><p><strong>Resources:</strong></p><ul><li><strong>Actions Runner Controller (ARC) Repository:</strong> <a href="https://github.com/actions/actions-runner-controller/releases">https://github.com/actions/actions-runner-controller/releases</a></li><li><strong>Actions Runner Controller (ARC) Documentation:</strong> <a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller">https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller</a></li><li><strong>ArgoCD Documentation:</strong> <a href="https://argocd-operator.readthedocs.io/en/latest/reference/api.html/">https://argocd-operator.readthedocs.io/en/latest/reference/api.html/</a></li><li><strong>Helm Documentation:</strong> <a href="https://helm.sh/docs/helm/">https://helm.sh/docs/helm/</a></li><li><strong>GitHub App Authentication:</strong> <a href="https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app">https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bdbd7bf0549e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Triggering Workflows in Another Repository with GitHub Actions]]></title>
            <link>https://medium.com/@aminalawal/triggering-workflows-in-another-repository-with-github-actions-4f581f8e0ceb?source=rss-b72c689e97c0------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f581f8e0ceb</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[workflow]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Amina lawal]]></dc:creator>
            <pubDate>Tue, 03 Oct 2023 03:12:56 GMT</pubDate>
            <atom:updated>2024-03-30T00:09:54.860Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QwAHQqPqY0-9npm_pCefzg.jpeg" /></figure><p>GitHub Actions provides a powerful automation framework that allows you to orchestrate and automate various tasks within your repositories. One of its features is the ability to trigger workflows in one repository from another repository. This can be useful for managing cross-repository actions, such as updating dependencies or deploying services.</p><p>In this guide, we will walk you through the steps to create a workflow in a source repository that triggers a workflow in a target repository.</p><h3>Prerequisites</h3><p>Before you begin, ensure you have the following:</p><ul><li>Access to the source repository (the repository where you want to trigger the workflow).</li><li>Access to the target repository (the repository where the triggered workflow will run).</li><li>Basic knowledge of GitHub Actions.</li></ul><h3>Step 1: Configure the Source Repository</h3><h3>1.1. Set Up the Trigger Workflow</h3><p>In the source repository (the repository that will trigger the workflow in the target repository), create a new GitHub Actions workflow file or use an existing one. This file should define the workflow responsible for triggering the target workflow.</p><p>Example workflow file (target_workflow.yml):</p><pre>name: Trigger Target Workflow<br><br>on:<br>  workflow_dispatch:<br>    inputs:<br>      target_service:<br>        description: &#39;Input a service name (e.g., demo-app)&#39;<br>        required: true<br>      target_version:<br>        description: &#39;Input a version (e.g., v1.0.0)&#39;<br>        required: true<br><br>jobs:<br>  trigger:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>      - name: Trigger Workflow in Another Repository<br>        run: |<br>          # Set the required variables<br>          repo_owner=&quot;REPOSITORY OWNER&quot; <br>          repo_name=&quot;REPOSITORY NAME&quot;  <br>          event_type=&quot;trigger-workflow&quot; <br>          service=${{ github.event.inputs.target_service }}&quot;<br>          version=&quot;${{ github.event.inputs.target_version }}&quot;<br>  <br>          curl -L \<br>            -X POST \<br>            -H &quot;Accept: application/vnd.github+json&quot; \<br>            -H &quot;Authorization: Bearer ${{ secrets.PAT }}&quot; \<br>            -H &quot;X-GitHub-Api-Version: 2022-11-28&quot; \<br>            https://api.github.com/repos/$repo_owner/$repo_name/dispatches \<br>            -d &quot;{\&quot;event_type\&quot;: \&quot;$event_type\&quot;, \&quot;client_payload\&quot;: {\&quot;service\&quot;: \&quot;$service\&quot;, \&quot;version\&quot;: \&quot;$version\&quot;, \&quot;unit\&quot;: false, \&quot;integration\&quot;: true}}&quot;</pre><p>In this example, we’ve defined a workflow named “Trigger Target Workflow” that allows you to manually trigger the target workflow in another repository. This workflow is responsible for triggering another workflow in a different repository (target repository) based on user input. It allows you to specify a service name and a version (e.g., demo-app and v1.0.0) and then triggers the target workflow in the specified repository.</p><h3>1.2. Configure the Trigger Logic</h3><p>Inside your source workflow, define the logic and inputs required to trigger the target workflow. The key part is setting the target_repository input to specify the target repository where the workflow should be triggered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*peAjC5a5K76Lcsd0f6JH0Q.png" /></figure><h4>Trigger Logic and Inputs:</h4><ul><li><strong>on:</strong> This workflow is triggered manually using the workflow_dispatch event, which allows users to specify inputs.</li><li><strong>Inputs:</strong> Users are prompted to provide two inputs:<br>-target_service: The service name (e.g., demo-app) that identifies the target repository where the workflow should be triggered.<br>-target_version: The version (e.g., v1.0.0) that will be passed to the target workflow.</li></ul><h4>Job: trigger</h4><ul><li><strong>runs-on:</strong> The job runs on an Ubuntu-based runner.</li><li><strong>Steps:</strong></li></ul><ol><li><strong>Trigger Workflow in Another Repository:</strong> This step performs the actual triggering of the target workflow in the specified repository.</li></ol><ul><li>It sets the required variables:<br>- repo_owner: The owner or organization name of the target repository.<br>- repo_name: The name of the target repository.<br>- event_type: The event type that the target workflow listens for (e.g., trigger-workflow).<br>- service: The service name provided as an input.<br>- version: The version provided as an input.</li><li>It uses the curl command to send a POST request to the GitHub API to trigger the workflow.</li><li>The request includes necessary headers such as authorization, content type, and API version.</li><li>It passes the event type and a client payload with the service name, version, and other optional flags.</li></ul><p><strong>Note:</strong> You should generate your own Personal Access Token (PAT) with the required permissions to enable you to trigger workflows in other repositories. You can refer to this <a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens">guide</a> for instructions on how to create a PAT.</p><h3>1.3. Commit and Push</h3><p>Commit the changes to your source repository and push them to the main branch. This ensures that the source workflow is set up and ready to trigger the target workflow.</p><h3>Step 2: Configure the Target Repository</h3><h3>2.1. Set Up the Target Workflow</h3><ol><li>In the target repository (the one where you want to run the triggered workflow), create a new GitHub Actions workflow file or use an existing one. This file should define the workflow you want to run when triggered.</li></ol><p>Example workflow file (target_workflow.yml):</p><pre>name: My Target Workflow<br><br>on:<br>  repository_dispatch:<br>    types: [trigger-workflow]<br>    <br>jobs:<br>  build:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v4<br>        with:<br>        ref: ${{ github.event.client_payload.sha }}<br><br>      # Add your workflow steps here</pre><p>In this example, the “My Target Workflow” workflow is designed to respond to a repository_dispatch event with the trigger-workflow type. When this event occurs, it initiates the workflow</p><h3>2.2. Configure the Workflow Logic</h3><p>Inside your target workflow, define the logic and actions you want to perform when the workflow is triggered. This could include tasks like updating dependencies, deploying applications, or running tests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sJsCnHRAUDvaxxnTXEd31w.png" /></figure><ol><li><strong>Checkout code:</strong></li></ol><ul><li>This step uses the actions/checkout@v4 action to check out the source code of the repository.</li><li>Important customization is applied here by specifying the ref parameter with the value ${{ github.event.client_payload.sha }}. This means that the workflow will check out the specific commit or branch referenced in the sha field of the client_payload object of the triggering event. This allows the workflow to operate on the exact code state associated with the triggering event.</li></ul><p><strong>2. Extract Service Name &amp; Version:</strong></p><ul><li>This step is responsible for extracting and displaying the service name and version values from the client_payload object of the triggering event. These values were passed when the workflow was manually triggered and are stored in the payload for use in downstream steps.</li><li>The extraction process involves the following actions:<br>- Parsing the service field from the client_payload object and assigning it to the service_name variable.<br>- Displaying the extracted service name using echo.<br>- Parsing the version field from the client_payload object and assigning it to the version variable.<br>- Displaying the extracted version using echo.</li></ul><h3>2.3. Commit and Push</h3><p>Commit the changes to your target repository and push them to the main branch. This ensures that the target workflow is set up and ready to be triggered.</p><h3>Step 3: Trigger the Workflow</h3><p>Now that both the source and target workflows are set up, you can trigger the target workflow in the target repository from the source repository.</p><h3>3.1. Manual Trigger</h3><ol><li>In the source repository, go to the “Actions” tab.</li><li>Find and select the “Trigger Target Workflow” workflow.</li><li>Click the “Run workflow” button.</li><li>Provide the target_repository input with the name of the target repository (e.g., owner/repo).</li><li>Click the “Run workflow” button to trigger the target workflow in the specified repository.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_UV1NahIu7RWdxNQB2-foA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2m5urHSQ8gJBxPv0209mmw.png" /></figure><p>Once this has successfully completed, it then triggers the other workflow and passes the inputs from the previous workflow to the current one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-yr05xPPYONy1kuL0zaPbg.png" /></figure><h3>3.2. Automated Trigger (Optional)</h3><p>You can also automate the trigger process by scheduling the source workflow or integrating it with other events such as code pushes or pull requests. This allows you to set up automatic workflows that trigger the target workflow based on specific conditions or schedules.</p><h3>Conclusion</h3><p>You’ve successfully configured a GitHub Actions workflow in one repository to trigger another workflow in a different repository. This powerful feature allows you to streamline automation and collaboration across multiple repositories, making it easier to manage complex workflows and tasks.</p><p>With this knowledge, you can automate various processes and actions, improving your development and deployment workflows.</p><h3>REFERENCES</h3><ul><li><a href="https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows">Events that trigger workflows</a></li><li><a href="https://docs.github.com/en/rest/apps/webhooks?apiVersion=2022-11-28#update-a-webhook-configuration-for-an-app">GitHub App webhooks</a></li><li><a href="https://docs.github.com/en/actions">GitHub Actions documentation</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f581f8e0ceb" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>