<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vipul Raja on Medium]]></title>
        <description><![CDATA[Stories by Vipul Raja on Medium]]></description>
        <link>https://medium.com/@vipulraja?source=rss-dd01c5137cb0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*D7eQgtS3gy8iXf8D_idUjg.jpeg</url>
            <title>Stories by Vipul Raja on Medium</title>
            <link>https://medium.com/@vipulraja?source=rss-dd01c5137cb0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 17:12:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vipulraja/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Gemma 4 31B IT Self-hosting Options & Gotchas]]></title>
            <link>https://medium.com/google-cloud/gemma-4-31b-it-self-hosting-options-gotchas-de1111dc36b4?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/de1111dc36b4</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Mon, 13 Jul 2026 12:24:38 GMT</pubDate>
            <atom:updated>2026-07-13T12:24:38.040Z</atom:updated>
            <content:encoded><![CDATA[<h3>Captured from moments when I got burned, so you don’t have to! 😅</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wHRMb5lOqLzr9T01tPvjEA.png" /><figcaption>Nano Banana generated image that summarizes this article 🤯</figcaption></figure><h3>Context</h3><p><strong><em>Gemma 4 31B IT</em></strong> is an instruction-tuned, open-weight model (read open source) from Google DeepMind, released in April 2026 as part of the Gemma 4 family. It is a dense, multimodal model with approximately *30.7 billion parameters, engineered to deliver frontier-level reasoning, coding, and agentic performance in a footprint that runs on a single high-memory GPU.</p><p><strong>Gemma 4 31B IT — Naming Convention Breakdown</strong></p><blockquote>Gemma 4 31B IT = <em>Gemma family, 4th generation, 31 billion parameters, Instruction-Tuned variant</em></blockquote><p>For context, there are other Gemma and Gemma 4 model variants available that are all open-weight, Im listing some of them below for reference.</p><p><strong>Current Generation</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ALd-AV5RIO2Wg3KE-OUxgw.png" /><figcaption>Gemma 4 Model Family (current as of July 2026)</figcaption></figure><p><strong>Key Capabilities</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ydGqiySb2uN_0sewiNhB-g.png" /><figcaption>Gemini created table for us</figcaption></figure><p><strong>Key Highlights</strong></p><ul><li><strong><em>Advanced Reasoning</em></strong>: Features a configurable “thinking” mode that enables step-by-step problem-solving for complex tasks.</li><li><strong><em>Agentic &amp; Developer-Ready</em></strong>: Native support for function calling and structured output, making it well-suited for tool use and agentic workflows.</li><li><strong><em>Accessible Deployment</em></strong>: Optimized to run on consumer workstations and single-GPU setups (the unquantized version fits on an 80GB GPU), while scaling to cloud and enterprise environments.</li><li><strong><em>Open &amp; Flexible</em></strong>: Its Apache 2.0 license allows developers and researchers to freely deploy, fine-tune, and build on the model.</li></ul><p><strong>Ideal Use Cases</strong></p><p>Complex reasoning, multilingual applications, code generation, multimodal understanding, and building autonomous, tool-using agents.</p><h3>Why is Gemma 4 31B IT oozing relevance?</h3><ol><li><strong>Best-in-Class Reasoning Density</strong></li></ol><p>While many top-tier competitors (like Llama 4 and Qwen 3.6 Max) have shifted to large <strong>Mixture-of-Experts (MoE)</strong> architectures, Gemma 4 31B remains a <strong>dense</strong> model. This delivers consistent, predictable, high-quality reasoning without the routing complexity of MoE — punching well above its weight class with strong scores on demanding benchmarks like <strong>AIME 2026 (~89%)</strong> and <strong>LiveCodeBench v6 (~80%)</strong>.</p><ol><li><strong>The “Sweet Spot” for Local, Workstation Hardware</strong></li></ol><p>Its ~30.7B dense design is deliberately engineered to fit on <strong>high-end consumer GPUs</strong> (down to 24GB VRAM class with quantization). This makes it uniquely accessible for:</p><ul><li>Developers running frontier-quality reasoning <strong>locally</strong>.</li><li>Researchers who need reproducible, single-GPU deployments.</li><li>Teams that can’t justify the multi-GPU footprint of massive MoE models</li></ul><p>Given this context, hope y’all can relate as to why Gemma 4 is a great choice for enterprises in today’s day and age. If there’s still some grey spots, let me help, with Gemma 4 we get:</p><p>- Fully Private, self-contained hosting and inference<br>- Ability to fine-tune, train on enterprise specific problems<br>- Does not need to run on a Rolls-Royce of a machine, a Toyota will do!<br>- You can change and adapt the model architecture to test-drive theories and experiments while keeping the base-weights frozen by performing model surgery (deserves a little bit attention below)</p><blockquote><strong>Model surgery</strong> is the practice of selectively modifying a model’s internal weights, layers, or expert routing (e.g., pruning MoE experts, merging LoRA adapters, or swapping attention heads) post-training — and on Gemma 4 it can be performed in numerous ways but the most native one being, direct checkpoint editing using JAX since the weights are openly available.</blockquote><p>Note: I am building out a comprehensive guide on performing model surgery on Gemma 4 as that is worthy of an article of its own, stay tuned.</p><p>Now, with this in mind let’s move ahead to see what GCP-native options we have for self-hosting Gemma 4; 2 are highly recomended: Vertex AI Endpoints (now GEAP endpoints) and GKE. We will review both of these here and compare them against each other. Another common choice is CloudRun and I have included documentation for it in the footnotes for reference.</p><p>The article covers the recommended common steps agnostic of your deployment choice and then moves into deployment pattern-specific recommendations.</p><h3>Common Step:🚰 Model Weight Hydration (GCS-to-GCS) with Model Garden</h3><p>Instead of using Hugging Face (which is throttled and requires secure token management; clunky imho) and Cloud Build (which requires complex workspaces), we suggest copying the weights directly from Google’s Vertex Model Garden GCS bucket.</p><p>Since this is a GCS-to-GCS transfer, it is executed entirely on the Google Cloud internal backbone. It will take less than 2 minutes!</p><h3>1. Copy the Model Weights (Model Garden -&gt; Your GCS)</h3><p>Run this command to trigger a server-side copy:</p><blockquote>bash</blockquote><blockquote>gcloud storage cp -r \</blockquote><blockquote>gs://vertex-model-garden-restricted-us/gemma4/gemma-4–31B-it \</blockquote><blockquote>gs://$BUCKET_NAME/</blockquote><h3><strong>2. GCS Fuse (Mitigating the 30-Min Cold Start)</strong></h3><p>Gemma 4 31B weights are ~58GB. Downloading this during pod scale-up will cause extremely long cold starts (often 30+ minutes due to throttling).</p><ul><li><strong>Best Practice:</strong> Download the model weights once and store them in a <strong>Google Cloud Storage (GCS) bucket</strong> in the same region as your cluster.</li><li><strong>GKE Configuration:</strong> Mount the GCS bucket to your GKE pods using <strong>Cloud Storage FUSE</strong> with caching enabled, or use a <strong>Local SSD cache</strong>. This reduces pod startup time from 30 minutes to <strong>under 2 minutes</strong>.</li></ul><h3>☸️ Deploying on GKE (Self-Managed)</h3><h3>2. Hardware Selection &amp; Memory Planning</h3><p>Gemma 4 31B is a dense model. Memory planning must account for both the <strong>static base weights</strong> and the <strong>dynamic KV Cache</strong> (which grows with context length and concurrency).</p><h3>Memory Requirements</h3><ul><li><strong>BF16 (16-bit):</strong> ~58.3 GB (Base weights only)</li><li><strong>FP8 (8-bit quantized):</strong> ~30.4 GB (Base weights only)</li><li><strong>INT4 (4-bit quantized):</strong> ~17.4 GB (Base weights only)</li></ul><h3>Recommended GPU Configurations</h3><h4>Option A: Single-GPU</h4><p>Deploying on a single GPU avoids inter-GPU communication overhead (Tensor Parallelism) and simplifies Kubernetes pod scaling.</p><ul><li><strong>NVIDIA RTX PRO 6000 Blackwell (96GB VRAM) [G4 Instances]:</strong> <em>Top Recommendation.</em></li><li><strong>Weights (BF16):</strong> ~58.3 GB</li><li><strong>Headroom for KV Cache:</strong> ~37.7 GB (Ample room for large context windows and high concurrency).</li><li><strong>NVIDIA H100 (80GB VRAM) [A3 Instances]:</strong></li><li><strong>Weights (BF16):</strong> ~58.3 GB</li><li><strong>Headroom for KV Cache:</strong> ~21.7 GB (Sufficient for medium-to-large context).</li></ul><h4>Option B: Multi-GPU Cost-Optimized</h4><p>If H100s or RTX 6000 Blackwells are constrained, use cost-effective <strong>NVIDIA L4 (24GB VRAM) [G2 Instances]</strong> with Tensor Parallelism (TP).</p><ul><li><strong>4x NVIDIA L4 (96GB total VRAM):</strong> Run with <em>tensor-parallel-size=4</em> for BF16.</li><li><strong>2x NVIDIA L4 (48GB total VRAM):</strong> Run with <em>tensor-parallel-size=2</em> and <strong>FP8 Quantization</strong>.</li></ul><h3>3. Serving Architecture: vLLM Config for Agents</h3><p>Use <strong>vLLM</strong> as the inference engine, leveraging the Google Cloud Model Garden optimized image: <em>us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:gemma4</em></p><h3>🤖 Deploy on Vertex AI Endpoints (Managed)</h3><p>For hosting on <strong>Vertex AI Prediction Endpoints</strong> (the core model hosting infrastructure of the Gemini Enterprise Agent Platform — GEAP), deploying a custom <strong>vLLM</strong> container provides a fully managed API with low operational overhead while preserving advanced optimization parameters.</p><h4>1. Vertex AI Deployment Architecture Overview</h4><p>Vertex AI Endpoints allow you to host models inside custom Docker containers. For Gemma 4, you should package <strong>vLLM</strong> inside a custom container and configure Vertex AI to route prediction and health requests directly to vLLM’s OpenAI-compatible API.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X-SfQy6HKZddyAPPk5hxmg.png" /><figcaption>Birds-eye view</figcaption></figure><p>Gemma 4 31B is a 58GB model. Deserializing weights into GPU memory takes time.</p><ul><li><strong>Requirement:</strong> Set serving_container_deployment_timeout to 1800 seconds (30 minutes) to prevent Vertex AI from flagging the deployment as failed while the weights are still loading.</li></ul><h4>3. Hardware Selection on Vertex AI</h4><p>Match the Vertex AI machine types to your serving requirements:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9CwM_M_vGWtjBUJ0bPq2aQ.png" /></figure><h4>4. Implementation: Python SDK Deployment Code</h4><p>The recommended way to deploy to GEAP / Vertex AI is using the official google-cloud-aiplatform Python SDK.</p><h4>5. Operational Best Practices on Vertex AI</h4><p><strong>Weight Hydration from Cloud Storage (GCS)</strong></p><ul><li>Do not pull directly from Hugging Face. During autoscaling events, Hugging Face’s bandwidth limits will throttle download speeds, stretching cold starts to 45+ minutes.</li><li><strong>Best Practice:</strong> Download the model using a scratch instance, upload it to a <strong>GCS bucket in the same region</strong> as your Vertex AI Endpoint, and grant the endpoint’s Service Account the roles/storage.objectViewer role. vLLM natively reads model weights directly from gs:// paths at maximum GCP network speeds.</li></ul><p><strong>Private VPC Connectivity</strong></p><ul><li>In production environments, you may want to configure <strong>Private Service Connect (PSC)</strong> or <strong>VPC Network Peering</strong> for Vertex AI Online Prediction. This allows your backend application or agent harness to query the endpoint via private IPs, reducing network hops and securing model traffic.</li></ul><h3>Comparison &amp; Operational Summary</h3><h3>1. Deployment Platform Choice: GKE vs. Vertex AI</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0GR0VjNY1xwlL2bu-39yxw.png" /><figcaption>Quick comparison</figcaption></figure><blockquote><strong><em>TIP:</em></strong> Use <strong>GKE</strong> if you need to maximize throughput, minimize latency via advanced vLLM configs, and implement custom autoscaling based on queue depth. Use <strong>Vertex AI Model Garden</strong> for rapid prototyping or if you prefer a low-maintenance, fully managed service.</blockquote><h3>1. Sample Configuration &amp; Setup</h3><ul><li><strong>GKE (Self-Managed):</strong> Manual orchestration using Kubernetes deployment manifests. Serving engine (vLLM) is configured via container arguments such as — tensor-parallel-size, — enable-prefix-caching, and — dtype=bfloat16.</li><li><strong>Vertex AI (Managed):</strong> Uses a Python-based deployment script to upload models to the Model Registry and deploy to managed endpoints via aiplatform.Model.upload and model.deploy.</li></ul><h3>2. Auto-scaling under High Concurrency</h3><ul><li><strong>GKE:</strong> Supports custom, event-driven autoscaling using KEDA. Scaling can be based on vLLM-specific metrics like queue depth (num_requests_waiting) or GPU cache utilization.</li><li><strong>Vertex AI:</strong> Fully managed autoscaling triggered by CPU/GPU utilization or QPS, with configurable target duty cycles and replica counts.</li></ul><h3>3. Latency &amp; Throughput Benchmarks</h3><ul><li>Both platforms utilize a Python benchmarking script to measure p50/p99 latency and tokens per second.</li><li><strong>Optimizations:</strong> Both support prefix caching (reducing TTFT) and chunked prefill. Speculative decoding can provide up to a 2x speedup in decode latency.</li></ul><h3>4. Known Limitations &amp; Operational Gotchas</h3><ul><li><strong>Cold Starts:</strong> Large weight downloads (58GB) can cause 45-minute cold starts; using Cloud Storage FUSE reduces this to under 2 minutes.</li><li><strong>GPU Quotas:</strong> High-end GPUs often require manual quota requests.</li><li><strong>Shared Memory:</strong> Vertex AI requires explicit shared memory allocation to prevent container crashes.</li></ul><h3>5. Infra/Maintenance Overhead Comparison</h3><ul><li><strong>GKE:</strong> High control and flexibility but higher maintenance overhead (managing clusters and manifests).</li><li><strong>Vertex AI:</strong> Low maintenance as Google handles infrastructure and load balancing, but offers less configuration control.</li></ul><h3>6. References</h3><p><a href="https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/serve-gemma-gpu-vllm">https://docs.cloud.google.com/kubernetes-engine/docs/tutorials/serve-gemma-gpu-vllm</a></p><p><a href="https://cloud.google.com/vertex-ai/docs/generative-ai/open-models/use-gemma">https://cloud.google.com/vertex-ai/docs/generative-ai/open-models/use-gemma</a></p><p><a href="https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemma4">https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemma4</a></p><p><a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/resources/locations#united-states_2">https://docs.cloud.google.com/gemini-enterprise-agent-platform/resources/locations#united-states_2</a></p><p><a href="https://docs.cloud.google.com/run/docs/run-gemma-on-cloud-run">https://docs.cloud.google.com/run/docs/run-gemma-on-cloud-run</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=de1111dc36b4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/gemma-4-31b-it-self-hosting-options-gotchas-de1111dc36b4">Gemma 4 31B IT Self-hosting Options &amp; Gotchas</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exposing GEAP Endpoints Privately]]></title>
            <link>https://medium.com/google-cloud/exposing-geap-endpoints-privately-188002843481?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/188002843481</guid>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Sat, 11 Jul 2026 08:05:38 GMT</pubDate>
            <atom:updated>2026-07-11T08:05:38.566Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c3pkW0BT6x_et5qEHsK5lA.png" /><figcaption>Nano Banana generated image that is kinda all you need for knowing how expose GEAP Endpoints privately</figcaption></figure><p>This guide provides a comprehensive, step-by-step blueprint and architectural best practices for configuring Gemini Enterprise Agent Platform (GEAP) online prediction endpoints to be securely reachable over private networks, including Virtual Private Clouds (VPCs) and on-premises environments.</p><p>Pls note that I had originally written this article in my notes before Vertex AI became GEAP and thus, you’ll find me use Vertex AI in lieu of GEAP throughout.</p><h3>1. Overview of Private Endpoints in GEAP (pka Vertex AI)</h3><p>To design a secure, low-latency machine learning architecture, it is critical to understand how Google Cloud isolates and connects Vertex AI resources.</p><h3>The Tenant Project Architecture</h3><p>Vertex AI online prediction runs in a <strong>Google-managed tenant project</strong>. When you deploy a model to an endpoint, the actual compute resources (Google Kubernetes Engine clusters or Virtual Machines) are provisioned inside this tenant project. Because customers do not have direct access to the tenant project, network connectivity must be explicitly established between the customer’s VPC and the tenant VPC.</p><h3>Recommended Pattern: Private Service Connect (PSC) for GEAP Endpoints</h3><p><strong>Private Service Connect (PSC)</strong> is the modern, recommended pattern for secure, private connectivity to GEAP. It completely eliminates the complexities of VPC peering, IP address block allocation, and transitive routing.</p><h3>Architecture: How PSC Works</h3><p>Instead of peering networks, PSC uses a <strong>forwarding rule</strong> inside your VPC that points to a <strong>Service Attachment</strong> exposed by the GEAP online prediction service. This forwarding rule instantiates a single, local private IP address inside your subnet.</p><p>To your network, the GEAP endpoint looks like a local resource (e.g., a local VM interface).</p><h3>Step-by-Step PSC Endpoint Configuration</h3><h4>Step 1: Create the Private Endpoint in Vertex AI</h4><p>When creating the endpoint, you must enable PSC and specify the consumer project IDs that are allowed to connect.</p><blockquote>gcloud beta ai endpoints create \</blockquote><blockquote>— display-name=vertex-psc-endpoint \</blockquote><blockquote>— region=us-central1 \</blockquote><blockquote>— project=my-gcp-project-id \</blockquote><blockquote>— enable-private-service-connect \</blockquote><blockquote>— project-allowlist=my-gcp-project-id</blockquote><p>Once created, describe the endpoint to retrieve the <strong>Service Attachment URI</strong>:</p><blockquote>gcloud beta ai endpoints describe [ENDPOINT_ID] — region=us-central1</blockquote><blockquote>Look for the serviceAttachment field in the output (e.g., projects/vtxd-prod-us-central1/regions/us-central1/serviceAttachments/sa-xxxx)</blockquote><h4>Step 2: Reserve an Internal IP in Your VPC</h4><p>Reserve a single IP address in a subnet of your choice:</p><blockquote>gcloud compute addresses create psc-vertex-prediction-ip \</blockquote><blockquote>— subnet=my-subnet \</blockquote><blockquote>— addresses=10.0.0.100 \</blockquote><blockquote>— region=us-central1 \</blockquote><blockquote>— project=my-gcp-project-id</blockquote><h4>Step 3: Create the Forwarding Rule (PSC Endpoint)</h4><p>Create a regional forwarding rule pointing to the Service Attachment URI retrieved in Step 1.</p><blockquote>gcloud compute forwarding-rules create psc-geap-forwarding-rule \</blockquote><blockquote>— region=us-central1 \</blockquote><blockquote>— network=my-custom-vpc \</blockquote><blockquote>— address=psc-vertex-prediction-ip \</blockquote><blockquote>— target-service-attachment=projects/vtxd-prod-us-central1/regions/us-central1/serviceAttachments/sa-xxxx \</blockquote><blockquote>— project=my-gcp-project-id</blockquote><h4>Step 4: Configure Private DNS</h4><p>Create a Private DNS Zone in Cloud DNS so that your application code can resolve the GEAP service domain to the local PSC IP address (10.0.0.100).</p><blockquote># 1. Create a Private DNS Zone for the regional endpoint</blockquote><blockquote>gcloud dns managed-zones create vertex-dns-zone \</blockquote><blockquote>— description=”Private zone for Vertex AI PSC” \</blockquote><blockquote>— dns-name=us-central1-aiplatform.googleapis.com. \</blockquote><blockquote>— networks=my-custom-vpc \</blockquote><blockquote>— visibility=private \</blockquote><blockquote>— project=my-gcp-project-id</blockquote><blockquote># 2. Add an A record pointing to the PSC IP</blockquote><blockquote>gcloud dns record-sets create us-central1-aiplatform.googleapis.com. \</blockquote><blockquote>— rrdatas=10.0.0.100 \</blockquote><blockquote>— type=A \</blockquote><blockquote>— ttl=300 \</blockquote><blockquote>— zone=vertex-dns-zone \</blockquote><blockquote>— project=my-gcp-project-id</blockquote><h3>Key Architectural Clarification: PSC Endpoints vs. PSC Interfaces</h3><p>It is vital to distinguish between the two PSC directions:</p><p>1. <strong>PSC Endpoints (Ingress / Consumer-to-Producer)</strong>: Used to allow your applications (in VPC or on-premises) to call the GEAP online prediction service. You create a forwarding rule in your VPC.</p><p>2. <strong>PSC Interfaces (Egress / Producer-to-Consumer)</strong>: Used when a Google-managed service (like GEAP Custom Training, Ray on Vertex AI, or GEAP Pipelines) needs to initiate connections <em>outbound</em> to reach databases, API endpoints, or secure web proxies inside your private network. This uses a <strong>Network Attachment</strong> resource.</p><h3>References &amp; further reading</h3><ul><li><em>Vertex AI Networking Overview</em>: <a href="https://cloud.google.com/vertex-ai/docs/general/netsec-overview">https://cloud.google.com/vertex-ai/docs/general/netsec-overview</a></li><li><em>Vertex AI Private Service Connect Endpoints</em>: <a href="https://cloud.google.com/vertex-ai/docs/general/psc-endpoints">https://cloud.google.com/vertex-ai/docs/general/psc-endpoints</a></li><li><em>VPC Network Peering with Vertex AI</em>: <a href="https://cloud.google.com/vertex-ai/docs/general/vpc-peering">https://cloud.google.com/vertex-ai/docs/general/vpc-peering</a></li><li><em>Service Networking / Private Services Access</em>: <a href="https://cloud.google.com/vpc/docs/configure-private-services-access">https://cloud.google.com/vpc/docs/configure-private-services-access</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=188002843481" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/exposing-geap-endpoints-privately-188002843481">Exposing GEAP Endpoints Privately</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gemini Enterprise Agent Platform Pipelines: Private & On-Premises Network Connectivity Guide]]></title>
            <link>https://medium.com/google-cloud/gemini-enterprise-agent-platform-pipelines-private-on-premises-network-connectivity-guide-044496cb4bad?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/044496cb4bad</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Fri, 10 Jul 2026 07:39:26 GMT</pubDate>
            <atom:updated>2026-07-10T07:39:26.580Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9pPICJKF6IGy2H05zekCxg.png" /><figcaption>Nano Banana generated visual which manages to capture the contents of this post!</figcaption></figure><p>This article provides a comprehensive, step-by-step technical playbook for configuring <strong>Vertex AI Pipelines</strong> (now the <strong>Gemini Enterprise Agent Platform Pipelines aka GEAP</strong>) to securely resolve and reach internal IP addresses on your private or on-premises networks.</p><h3>Summary &amp; Architecture Overview</h3><p>GEAP Pipelines runs workloads in a secure, Google-managed <strong>tenant project</strong> and <strong>tenant VPC network</strong>. By default, these pipeline containers run in isolation, meaning they cannot reach resources in your Virtual Private Cloud (VPC) network or your on-premises datacenters.</p><p>Historically, organizations connected these networks using <strong>VPC Network Peering (Private Services Access / PSA)</strong>. However, VPC Peering introduces significant constraints, notably the lack of transitive routing and heavy IP address consumption.</p><p>Google Cloud’s modern, recommended approach is <strong>Private Service Connect (PSC) Interfaces</strong>. Rather than peering entire VPCs, the Google-managed tenant VPC initiates a targeted connection to your VPC via a <strong>Network Attachment</strong>. This creates a multi-NIC interface directly on the pipeline container, allowing it to leverage your VPC’s native routing tables, resolve internal DNS, and transitively access hybrid networks (VPN/Interconnect) and internet egress.</p><h3>Architectural Comparison: PSA vs. PSC Interface</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ozK_IXV-WKULlGhko_zwew.png" /></figure><h3>Section 1: Foundations of Private Service Connect (PSC) Interfaces</h3><ul><li>A Private Service Connect interface is a secondary virtual network interface (vNIC) deployed inside the producer GEAP-VM or GKE pod.</li><li>A <strong>Network Attachment</strong> is created in the consumer (your) VPC network.</li><li>When the pipeline runs, GEAP requests a connection to this Network Attachment.</li><li>The pipeline container is provisioned with an IP address allocated directly from your designated consumer subnet.</li><li>Because the interface resides inside your subnet, it inherits the routing, DNS, and firewall rules of your VPC.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SgCocxUXRaVsNA-E6kR93w.png" /><figcaption>Architectural representation of PSC Interfaces on GEAP Pipelines</figcaption></figure><h3>Section 2: Step-by-Step Configuration: Basic Private Connectivity (RFC 1918)</h3><p>This section details how to configure GEAP Pipelines to reach an RFC 1918 destination (e.g., an internal VM or database inside your VPC) using a PSC Interface.</p><h3>Step 2.1: Enable Required APIs in your Project</h3><p>Ensure the following APIs are enabled in your Google Cloud project:</p><blockquote>gcloud services enable \</blockquote><blockquote>compute.googleapis.com \</blockquote><blockquote>aiplatform.googleapis.com \</blockquote><blockquote>servicenetworking.googleapis.com \</blockquote><blockquote>dns.googleapis.com</blockquote><h3>Step 2.2: Set Up the Consumer VPC and Subnets</h3><p>Create your VPC network and define two subnets:</p><p>1. A subnet for your target resources (e.g., test-subnet-1).</p><p>2. A dedicated subnet reserved exclusively for the PSC Network Attachment (e.g., intf-subnet).</p><blockquote># Create VPC</blockquote><blockquote>gcloud compute networks create consumer-vpc — subnet-mode=custom</blockquote><blockquote># Create Subnet for Target VM / Service</blockquote><blockquote>gcloud compute networks subnets create test-subnet-1 \</blockquote><blockquote>— network=consumer-vpc \</blockquote><blockquote>— range=192.168.20.0/28 \</blockquote><blockquote>— region=us-central1</blockquote><blockquote># Create Subnet for PSC Network Attachment</blockquote><blockquote>gcloud compute networks subnets create intf-subnet \</blockquote><blockquote>— network=consumer-vpc \</blockquote><blockquote>— range=192.168.10.0/28 \</blockquote><blockquote>— region=us-central1 \</blockquote><blockquote>— enable-private-ip-google-access</blockquote><h3>Step 2.3: Create the Private Service Connect Network Attachment</h3><p>The Network Attachment acts as the landing point for the PSC interface.</p><blockquote>gcloud compute network-attachments create psc-network-attachment \</blockquote><blockquote>— region=us-central1 \</blockquote><blockquote>— connection-preference=ACCEPT_MANUAL \</blockquote><blockquote>— subnets=intf-subnet</blockquote><p>Note: Keep track of the Network Attachment URI. It will look like:<em> “projects/[PROJECT_ID]/regions/us-central1/networkAttachments/psc-network-attachment”</em></p><h3>Step 2.4: Configure Service Agent IAM Permissions</h3><p>Vertex AI Pipelines operates on your behalf using a Service Agent. This Service Agent must have the <strong>Compute Network Admin</strong> role to dynamically provision network interfaces in your VPC.</p><ol><li>Retrieve your project number:</li></ol><blockquote>Bash PROJECT_NUMBER=$(gcloud projects describe $(gcloud config get-value project) — format=’value(projectNumber)’)</blockquote><ol><li>Grant the role to the Vertex AI Service Agent:</li></ol><blockquote>Bash gcloud projects add-iam-policy-binding $PROJECT_ID \ — member=”serviceAccount:service-$PROJECT_NUMBER@gcp-sa-aiplatform.iam.gserviceaccount.com” \</blockquote><blockquote>— role=”roles/compute.networkAdmin”</blockquote><h3>Step 2.5: Configure Firewalls to Allow Ingress</h3><p>Since the pipeline traffic enters your VPC originating from intf-subnet (192.168.10.0/28), you must allow this source range to reach your target resources.</p><blockquote># Example: Allow ICMP and TCP traffic from the PSC subnet to your target VMs</blockquote><blockquote>gcloud compute firewall-rules create allow-ingress-from-vertex-pipelines \</blockquote><blockquote>— network=consumer-vpc \</blockquote><blockquote>— action=ALLOW \</blockquote><blockquote>— direction=INGRESS \</blockquote><blockquote>— priority=1000 \</blockquote><blockquote>— source-ranges=192.168.10.0/28 \</blockquote><blockquote>— rules=tcp:22,tcp:80,tcp:443,tcp:5432,icmp</blockquote><h3>Step 2.6: Configure the Pipeline Job in Python</h3><p>When submitting your GEAP Pipeline run using the Google Cloud Pipeline Components (GCPC) SDK or the REST API, you must explicitly supply the psc_interface_config.</p><p>Below is an example of how to define and run a pipeline with this configuration:</p><blockquote>from google.cloud import aiplatform</blockquote><blockquote>from kfp import dsl</blockquote><blockquote>from kfp import compiler</blockquote><blockquote>PROJECT_ID = “your-project-id”</blockquote><blockquote>REGION = “us-central1”</blockquote><blockquote>NETWORK_ATTACHMENT_URI = f”projects/{PROJECT_ID}/regions/{REGION}/networkAttachments/psc-network-attachment”</blockquote><blockquote>BUCKET_URI = “gs://your-pipeline-bucket”</blockquote><blockquote># Initialize SDK</blockquote><blockquote>aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)</blockquote><blockquote># Define a simple task that pings an internal database</blockquote><blockquote>@dsl.container_component</blockquote><blockquote>def ping_internal_db(db_ip: str):</blockquote><blockquote>return dsl.ContainerSpec(</blockquote><blockquote>image=”ubuntu:22.04&quot;,</blockquote><blockquote>command=[“bash”, “-c”],</blockquote><blockquote>args=[</blockquote><blockquote>f”apt-get update &amp;&amp; apt-get install -y inetutils-ping &amp;&amp; ping -c 3 {db_ip}”</blockquote><blockquote>]</blockquote><blockquote>)</blockquote><blockquote>@dsl.pipeline(name=”private-network-pipeline”)</blockquote><blockquote>def my_pipeline(db_ip: str):</blockquote><blockquote>ping_internal_db(db_ip=db_ip).set_caching_options(False)</blockquote><blockquote># Compile Pipeline</blockquote><blockquote>compiler.Compiler().compile(pipeline_func=my_pipeline, package_path=”private_pipeline.yaml”)</blockquote><blockquote># Run Pipeline with PSC Config</blockquote><blockquote>job = aiplatform.PipelineJob(</blockquote><blockquote>display_name=”private-pipeline-psc-run”,</blockquote><blockquote>template_path=”private_pipeline.yaml”,</blockquote><blockquote>parameter_values={“db_ip”: “192.168.20.2”}, # Target IP inside the VPC</blockquote><blockquote>pipeline_root=f”{BUCKET_URI}/pipeline_root”</blockquote><blockquote>)</blockquote><blockquote># Submit with the PSC Interface Configuration</blockquote><blockquote>job.submit(</blockquote><blockquote>psc_interface_config={</blockquote><blockquote>“network_attachment”: NETWORK_ATTACHMENT_URI</blockquote><blockquote>}</blockquote><blockquote>)</blockquote><h3>Section 3: Setting Up DNS Resolution</h3><p>To allow your pipeline containers to resolve Fully Qualified Domain Names (FQDNs) hosted within your VPC (such as <em>database.internal.demo.com</em> or on-premises DNS names), you must configure <strong>DNS Peering</strong> between the Vertex AI tenant project and your consumer VPC.</p><h3>Procedural flow of DNS Peering</h3><ol><li>The pipeline container queries an internal domain (e.g., database.internal.demo.com).</li><li>The Google-managed tenant DNS server forwards the query to your consumer VPC’s Cloud DNS.</li><li>Your Cloud DNS resolves the IP address (or forwards it on-premises via Outbound DNS Forwarding).</li></ol><h3>Step 3.1: Grant DNS Peer Permission to GEAP Service Agent</h3><p>The GEAP Service Agent needs permission to peer with your DNS zones:</p><blockquote>gcloud projects add-iam-policy-binding $PROJECT_ID \</blockquote><blockquote>— member=”serviceAccount:service-$PROJECT_NUMBER@gcp-sa-aiplatform.iam.gserviceaccount.com” \</blockquote><blockquote>— role=”roles/dns.peer”</blockquote><h3>Step 3.2: Configure DNS Peering in the Pipeline Run</h3><p>You must append the <em>dns_peering_configs</em> block to your <em>psc_interface_config</em>. This informs the tenant project’s DNS to forward queries for specific domains to your VPC.</p><blockquote># Submit the job with both Network Attachment and DNS Peering configuration</blockquote><blockquote>job.submit(</blockquote><blockquote>psc_interface_config={</blockquote><blockquote>“network_attachment”: NETWORK_ATTACHMENT_URI,</blockquote><blockquote>“dns_peering_configs”: [</blockquote><blockquote>{</blockquote><blockquote>“domain”: “demo.com.”, # Note the trailing dot</blockquote><blockquote>“target_project”: PROJECT_ID,</blockquote><blockquote>“target_network”: “consumer-vpc”</blockquote><blockquote>}</blockquote><blockquote>]</blockquote><blockquote>}</blockquote><blockquote>)</blockquote><h3>Section 4: Secure Network Isolation, VPC Service Controls, and Proxy Configurations</h3><h3>The Non-RFC 1918 &amp; VPC-SC Egress Challenge</h3><p>While PSC Interfaces easily route RFC 1918 traffic (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), they face two major constraints:</p><p>1. <strong>Non-RFC 1918 IP Blocks:</strong> Standard PSC Interfaces cannot natively route to non-RFC 1918 addresses (e.g., Class E ranges like 240.0.0.0/4, or public IPs) without additional routing mechanisms.</p><p>2. <strong>VPC Service Controls (VPC-SC):</strong> If your project is inside a VPC-SC security perimeter, the tenant project’s default direct internet access is <strong>blocked</strong> to prevent data exfiltration. However, your pipeline may still need to fetch packages (pip, apt), download ML models, or access public APIs (like GitHub or HuggingFace).</p><h3>The Solution: Egress via Explicit Proxy</h3><p>To resolve these routing and security issues, you must deploy an <strong>Explicit Proxy</strong> inside your consumer VPC. All pipeline internet and non-RFC 1918 traffic is routed to this proxy VM (which has an RFC 1918 address), and the proxy routes the traffic out to the internet via your VPC’s <strong>Cloud NAT</strong> or on-premises gateways. There are 2 ways that I can think of for this;</p><ol><li>Google Cloud Secure Web Proxy (SWP)</li><li>Self-managed Proxy (Squid et al)</li></ol><p>I am going to limit the scope of this article to only GCP Secure Web Proxy as that’s my preferred poison and what I recommend.</p><h3>Google Cloud Secure Web Proxy (SWP)</h3><p>Secure Web Proxy is a fully managed, scalable service that acts as an explicit proxy, allowing you to define granular URL filtering policies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ON-AxuzbbhEXR2WvCH71CA.png" /></figure><h4>Step A.1: Create a Proxy-Only Subnet</h4><p>Secure Web Proxy requires a dedicated active proxy-only subnet in the region.</p><blockquote>gcloud compute networks subnets create proxy-only-uscentral1 \</blockquote><blockquote>— purpose=REGIONAL_MANAGED_PROXY \</blockquote><blockquote>— role=ACTIVE \</blockquote><blockquote>— region=us-central1 \</blockquote><blockquote>— network=consumer-vpc \</blockquote><blockquote>— range=10.10.100.0/26</blockquote><h4>Step A.2: Configure Secure Web Proxy Policy and Rules</h4><p>Define the security policies and URL rules. Below, we create a rule allowing access to a specific external domain (huggingface.co) or private domain.</p><ol><li>Create a security policy:</li></ol><blockquote># policy.yaml</blockquote><blockquote>description: “Secure Web Proxy policy for Vertex AI Pipelines”</blockquote><blockquote>name: “projects/[PROJECT_ID]/locations/us-central1/gatewaySecurityPolicies/vertex-swp-policy”</blockquote><blockquote>gcloud network-security gateway-security-policies import vertex-swp-policy \</blockquote><blockquote>— source=policy.yaml — location=us-central1</blockquote><ol><li>Add an explicit allowance rule (e.g., allow pipelines to reach huggingface.co):</li></ol><blockquote># rule.yaml</blockquote><blockquote>name: “projects/[PROJECT_ID]/locations/us-central1/gatewaySecurityPolicies/vertex-swp-policy/rules/allow-huggingface”</blockquote><blockquote>description: “Allow HuggingFace access”</blockquote><blockquote>enabled: true</blockquote><blockquote>priority: 100</blockquote><blockquote>basicProfile: ALLOW</blockquote><blockquote>sessionMatcher: “host() == ‘huggingface.co’”</blockquote><blockquote>gcloud network-security gateway-security-policies rules import allow-huggingface \</blockquote><blockquote>— source=rule.yaml — location=us-central1 — gateway-security-policy=vertex-swp-policy</blockquote><h4>Step A.3: Deploy the Gateway</h4><p>Create the Secure Web Proxy gateway. Assign it a static RFC 1918 IP address within your standard VPC subnet.</p><blockquote># gateway.yaml</blockquote><blockquote>name: “projects/[PROJECT_ID]/locations/us-central1/gateways/vertex-swp-gateway”</blockquote><blockquote>type: “SECURE_WEB_GATEWAY”</blockquote><blockquote>addresses: [“10.10.10.5”] # The RFC 1918 Proxy IP</blockquote><blockquote>ports: [8080]</blockquote><blockquote>gatewaySecurityPolicy: “projects/[PROJECT_ID]/locations/us-central1/gatewaySecurityPolicies/vertex-swp-policy”</blockquote><blockquote>network: “projects/[PROJECT_ID]/global/networks/consumer-vpc”</blockquote><blockquote>subnetwork: “projects/[PROJECT_ID]/regions/us-central1/subnetworks/rfc1918-subnet1”</blockquote><blockquote>routingMode: “EXPLICIT_ROUTING_MODE”</blockquote><blockquote>gcloud network-services gateways import vertex-swp-gateway \</blockquote><blockquote>— source=gateway.yaml — location=us-central1</blockquote><h4>Step A.4: Configure Target Firewalls</h4><p>Because Secure Web Proxy initiates egress connections using IP addresses from the <strong>proxy-only subnet</strong> (10.10.100.0/26), you must configure your firewall rules to allow ingress traffic from this proxy-only subnet to your target networks.</p><blockquote>gcloud compute firewall-rules create allow-ingress-from-swp \</blockquote><blockquote>— network=consumer-vpc \</blockquote><blockquote>— action=ALLOW \</blockquote><blockquote>— direction=INGRESS \</blockquote><blockquote>— priority=1000 \</blockquote><blockquote>— source-ranges=10.10.100.0/26 \</blockquote><blockquote>— rules=tcp:80,tcp:443</blockquote><h3>Step 4.5: Configuring the Pipeline Code to Use the Proxy</h3><p>To route outbound HTTP/HTTPS requests through your proxy (whether SWP or VM-based), your Python pipeline component must configure the standard proxy environment variables or pass them explicitly to your client library.</p><p>Here is an example KFP component utilizing the proxy configuration:</p><blockquote>@dsl.container_component</blockquote><blockquote>def run_api_call_via_proxy():</blockquote><blockquote>return dsl.ContainerSpec(</blockquote><blockquote>image=”python:3.9-slim”,</blockquote><blockquote>command=[“python3”, “-c”],</blockquote><blockquote>args=[</blockquote><blockquote>“””</blockquote><blockquote>import requests</blockquote><blockquote>import os</blockquote><blockquote># Define the proxy URL pointing to the Secure Web Proxy (SWP) inside the VPC</blockquote><blockquote># We use the internal DNS address configured via DNS Peering</blockquote><blockquote>proxy_url = “http://explicit-swp.demo.com:8080&quot;</blockquote><blockquote>proxies = {</blockquote><blockquote>“http”: proxy_url,</blockquote><blockquote>“https”: proxy_url,</blockquote><blockquote>}</blockquote><blockquote># The request is sent to the proxy, which then resolves and fetches the resource</blockquote><blockquote>try:</blockquote><blockquote>response = requests.get(“https://huggingface.co/api/models&quot;, proxies=proxies)</blockquote><blockquote>print(“Outbound Status:”, response.status_code)</blockquote><blockquote>print(“Data sample:”, response.text[:200])</blockquote><blockquote>except Exception as e:</blockquote><blockquote>print(“Egress failed:”, e)</blockquote><blockquote>raise e</blockquote><blockquote>“””</blockquote><blockquote>]</blockquote><blockquote>)</blockquote><p>That’ll do it for now, I wanted to include more in this article particularly how to isolate your network and observability best practices but I’ll save them for another day as this is already dense.</p><p><strong>Recommended Reading:</strong></p><ul><li><em>GCP Documentation: </em><a href="https://cloud.google.com/vertex-ai/docs/general/vpc-psc-i-setup"><em>Set up a Private Service Connect interface for Vertex AI resources</em></a></li><li><em>GCP Documentation: </em><a href="https://cloud.google.com/vertex-ai/docs/general/psc-interfaces"><em>About accessing Vertex AI services through Private Service Connect interfaces</em></a></li><li><em>GCP Blog: </em><a href="https://cloud.google.com/blog/products/ai-machine-learning/extending-network-reachability-of-vertex-pipelines"><em>Extending network reachability of Vertex AI Pipelines</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=044496cb4bad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/gemini-enterprise-agent-platform-pipelines-private-on-premises-network-connectivity-guide-044496cb4bad">Gemini Enterprise Agent Platform Pipelines: Private &amp; On-Premises Network Connectivity Guide</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the Model: Dissecting Reusable KFP Components in Gemini Enterprise Agent Platform Pipelines]]></title>
            <link>https://medium.com/google-cloud/beyond-the-model-dissecting-reusable-kfp-components-in-gemini-enterprise-agent-platform-pipelines-850764e56de5?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/850764e56de5</guid>
            <category><![CDATA[ml-pipeline]]></category>
            <category><![CDATA[kubeflow-pipelines]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[vertex-ai]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Fri, 10 Jul 2026 04:24:58 GMT</pubDate>
            <atom:updated>2026-07-10T04:24:58.496Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*byHJjaSq9fJJBjlVZGjwCw.png" /></figure><p>In the rapidly evolving landscape of MLOps, efficiency isn’t just about how fast your model trains; it’s about how quickly you can iterate on your entire pipeline. One of the most significant bottlenecks in machine learning engineering is the “siloed” component — a script written for one specific task that requires a complete rewrite the moment a new model or data type is introduced. By focusing on component reusability, teams can drastically reduce technical debt and accelerate the transition from experimentation to production. At Google Cloud we natively use Kubeflow Pipelines (KFP) within Gemini Enterprise Agent Platform (GEAP) Pipelines, an alternative to KFP is Tensorflow Xtended (TFX). The scope of this article is how to think about writing reusable components with KFP ie. the philosophy behind them.</p><h3>The Anatomy of a Robust Training Component</h3><p>A well-structured Kubeflow Pipelines (KFP) component acts as a modular unit within a larger workflow. To understand how to make these components reusable, we must first look at the core structure of a standard Python-based training script.</p><ul><li><strong>Imports</strong>: These define the environment, pulling in libraries like argparse, google.cloud, pandas, and xgboost.</li><li><strong>Core Training Logic</strong>: Encapsulated in functions like train_xgboost_model, this logic handles the specific requirements of the library, such as creating a DMatrix for XGBoost efficiency.</li><li><strong>The Orchestration Function</strong>: The model_training function serves as the main entry point, managing data loading from BigQuery or GCS, hyperparameter handling, and the final upload of the trained model artifact to a storage bucket.</li><li><strong>Execution Entry Point</strong>: The KFP DSL executor links the Python functions to the pipeline infrastructure.</li></ul><h3>Swapping Engines: Adapting for scikit-learn</h3><p>What happens when your project requires a shift from gradient boosting to a bagging approach like RandomForestClassifier? A reusable component structure allows you to swap the underlying engine with minimal friction.</p><h3>1. Update Imports</h3><p>Replace XGBoost-specific imports with scikit-learn and joblib for model serialization.</p><p>from sklearn.ensemble import RandomForestClassifier</p><p>from sklearn.model_selection import train_test_split</p><p>import joblib</p><h3>2. Define the New Training Logic</h3><p>Create a localized training function tailored for the scikit-learn API.</p><p>def train_random_forest_model(X_train, y_train, params):</p><p>model = RandomForestClassifier(**params)</p><p>model.fit(X_train, y_train)</p><p>return model</p><h3>3. Update the Orchestrator</h3><p>Adjust the model_training function to handle standard pandas DataFrames instead of specialized data structures like DMatrix, and use joblib.dump for the final export.</p><h3>One Component, Many Tasks: Task Adaptation</h3><p>Often, you don’t need to change the model library, but the task itself. Whether you are performing multi-class classification, binary classification, or regression, the core XGBoost component can be adapted through simple parameter adjustments.</p><h3>For Regression Tasks</h3><p>To predict continuous values, shift the objective and evaluation metrics:</p><ul><li><strong>Objective</strong>: Change from multi:softprob to reg:squarederror.</li><li><strong>Metric</strong>: Use rmse (Root Mean Squared Error) instead of mlogloss.</li><li><strong>Cleanup</strong>: Remove the num_class parameter as it is only applicable to classification.</li></ul><h3>For Binary Classification</h3><p>For tasks with only two possible outcomes:</p><ul><li><strong>Objective</strong>: Set to binary:logistic.</li><li><strong>Metric</strong>: Use logloss or auc.</li></ul><h3>The Architect’s Secret: Maximizing Reusability</h3><p>The ultimate goal of a reusable component is to avoid code changes entirely when switching tasks. This is achieved by promoting internal parameters to component arguments. By making <em>model_objective</em> and <em>eval_metric</em> configurable inputs, a single component can serve multiple purposes across different pipelines.</p><blockquote>def model_training(</blockquote><blockquote>datasets_path: str,</blockquote><blockquote>model_objective: str = ‘multi:softprob’,</blockquote><blockquote>eval_metric: str = ‘mlogloss’,</blockquote><blockquote># …</blockquote><blockquote>):</blockquote><blockquote>xgb_params = {</blockquote><blockquote>‘objective’: model_objective,</blockquote><blockquote>‘eval_metric’: eval_metric,</blockquote><blockquote># …</blockquote><blockquote>}</blockquote><h3>Conclusion: The “Write Once, Run Anywhere” Philosophy</h3><p>Building reusable KFP components is more than a coding preference; it is a strategic approach to MLOps. By abstracting the training logic and parameterizing the task-specific details, you create a library of tools that are truly versatile. This “write once, run anywhere” philosophy ensures that your focus remains on solving business problems rather than managing boilerplate code. Embracing this modularity is the first step toward a mature, scalable machine learning practice.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=850764e56de5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/beyond-the-model-dissecting-reusable-kfp-components-in-gemini-enterprise-agent-platform-pipelines-850764e56de5">Beyond the Model: Dissecting Reusable KFP Components in Gemini Enterprise Agent Platform Pipelines</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Thought Leadership, anyway?]]></title>
            <link>https://medium.com/@vipulraja/what-is-thought-leadership-anyway-a752eae58d11?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a752eae58d11</guid>
            <category><![CDATA[innovation-management]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[thought-leadership]]></category>
            <category><![CDATA[organizational-change]]></category>
            <category><![CDATA[growth-mindset]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Sun, 15 Sep 2024 03:19:06 GMT</pubDate>
            <atom:updated>2024-09-15T03:19:55.069Z</atom:updated>
            <content:encoded><![CDATA[<p>And, can you get better at it?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zk0gIligHc22hu0XY1WsPw.jpeg" /><figcaption>img source: inc.com</figcaption></figure><p>We in the Tech world love our ever changing buzz-words and mental models, Thought Leadership being one such buzz word. I wonder why a lot is spoken about the various consequential breakthroughs in technology, but, so little light is shone on the ways of working and the psyche that it takes to be working with breakthrough technology. And to take a bold step further, the method that it takes to create an environment where more technological breakthroughs can happen, organically.</p><p>In my experience, observation and learnings (many), I have come to believe that Thought Leadership is one of the key ingredients to working with breakthrough technologies. And, it is also the reason you are reading this article.</p><p>I went hunting on Google Search for a great definition of Thought Leadership. And I found many, listing some below for reference.</p><p>“<strong>Thought leadership</strong> refers to the position of an individual or an organization as an expert in a specific field.” — <a href="https://www.linkedin.com/pulse/thought-leadership-meaning-strategies-examples-youlynq-me-1e/">Read the article here!</a></p><p>“A thought leader has been described as an individual or firm recognized as an authority in a specific field.” — Wikipedia</p><p>“Being a thought leader means having a unique point of view and consistently living it. You must demonstrate strength and discipline in all your actions. You must have clarity about your purpose. That’s how you become known for something, productize it and even monetize it.” — <a href="https://www.forbes.com/councils/forbesagencycouncil/2023/04/19/what-it-really-means-and-takes-to-become-a-thought-leader/">Forbes</a></p><p>All though very close, I don’t think that these attempts define accurately what Thought Leadership is to me. I like the Forbes one for it calls out uniqueness and hence, randomness. In general, most definitions seem to paint thought leadership in an objective light and kind of put it in a box, which is what it isn’t!</p><blockquote>“Thought leadership (IMHO) is an act of creativity. It is an act that can result in abundance, directly or consequentially. It is also an act that is conjectured and is kind of similar to a best guess i.e. it involves randomness and failure as such.” <br> —<em> Vipul Raja</em></blockquote><p>The attributes of thought leaders mentioned in the definitions like discipline, clarity of purpose, described as an authority figure in their field are attributes that are commonly found in thought leaders. But I feel their objectivity can be construed incorrectly and cause one to miss the critical component of thought leadership. Since, the one core objective of thought leadership is, to create abundance i.e. opportunities, shifts etc. for the broader world.</p><p>At this point I’d like to draw the readers’ attention to the fact that both technological breakthroughs and thought leadership have this trait of creating abundance in common. Probably technical breakthroughs get the abundance creation gene from thought leadership!</p><p>A culture of thought leadership in your organization or team, returns various benefits. And yes, I feel that there are actions and behaviors that can be implemented for enhancing thought leadership.</p><p>Some ways to improve thought leadership:</p><p>1. Promoting a blamelessness culture.<br>2. Promoting a culture of skepticism.<br>3. Reducing prescriptiveness and allowing freedom.<br>4. Promoting a culture of embracing conflicts.<br>5. Accepting your mistakes.</p><p>Here’s another property of thought leadership, it is contagious. It attempts to create more thought leaders.</p><p>Thanks for reading this article. I appreciate your curiosity!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a752eae58d11" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ultra Trail Marathons and Failing BIG]]></title>
            <link>https://medium.com/@vipulraja/ultra-trail-marathons-and-failing-big-0113a4d2da3f?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/0113a4d2da3f</guid>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Fri, 28 Jun 2024 01:44:24 GMT</pubDate>
            <atom:updated>2024-06-28T01:46:57.314Z</atom:updated>
            <content:encoded><![CDATA[<h4>A match made in heaven!</h4><p>Another Trail Ultra Marathon is in the books! Here’s my Ultra Trail Gaspésia 100 aka UTG 100 54k story. I decided to write this blog post because as I started journaling my experience and feelings, I started to realize that I have content here that past me would have loved to know at the time. So, if you are looking to get into the adventurous sport of trail running, or trail ultras, you’ll find some rookie inspirational nuggets in here. Im gonna start off by saying this controversial statement, “You don’t need to run an ultra marathon to prove that you are a serious trail runner. Everyone that runs trails is a trail runner!” However, it can be very satisfying to plan, prepare and successfully run a trail ultra marathon. With this little context, let’s dive in! It’s story time!</p><p>Those who know me, know that I come up with themes for each year, 2023 was <strong>“No bad addictions”</strong> and 2024 is <strong>“Fail BIG”</strong>. With that theme in mind I planned to fail at my longest ultra trail marathon till date in Q4 2024 (~75k). But then while talking to a trail running buddy, I got to know a bunch of my buddies were running this other Trail Ultra in Q2. This story wouldn’t be complete without mentioning my trail running crew — <a href="https://www.saintjohntrailrunning.com/">Saint John Trail Running aka SJTR</a>. SJTR is a non-profit Trail Running community that I found or that found me (cosmic!) when I was at a very low point in my life. This was right around the time that COVID had hit, I was a <em>very</em> amateur runner, who got into short distance road running to beat depression and had never imagined running trails. SJTR organizes weekly group trail runs (for beginners, women, kids and advanced runners) — we run, but are mostly there for joking around and post-run hangouts ;)! For anyone looking to get into trail running, joining a local trail running community would be the one biggest value-add IMHO. You’ll find that trail runners are just really fast hikers (lol) and some of the best people. <a href="https://gaspesia.org/en/ultra-trail-gaspesia-100/">Ultra Trail Gaspesia 100</a> is an annual trail running race series that happens in June and is set in the beautiful Gaspésia National Park, with its base in Percé. The series offers an adventure for all appetites from 160k (100miles), 100k, 80k, 54k, 35k, 35k, 15k, 5k and kids’ 2–3k. But is very much popular for its Ultra courses i.e. 50k+. The course is known for its rugged terrain, brutal hills, river crossings and, magnificent views of the Atlantic Ocean as well as the renowned “Rocher du Perce aka The Perce Rock”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f_pfj77kTO1sIcqPbokagQ.jpeg" /><figcaption>Rocher du Percé aka Percé rock</figcaption></figure><p>This Ultra was on my radar since I started long distance trail running. I got to know about it ~6 weeks prior to the race (that’s less than half the time I usually train for before running an Ultra distance. The recommended training time is at least 4mos), but when your trail running fam is calling on you and you’re aiming to “Fail BIG” it’s a no brainer. I said, you know what I’m in! If I fail, I achieve my goal twice this year and if I don’t, well that’s just great isn’t it?! (The goal being ‘fail BIG’. )</p><p>And guess what?! I didn’t fail! I ran with a heart full of love, a smile I couldn’t shake and legs that were ready to give it their all!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*prHvr9KU2mSNJqvf5bO-8w.jpeg" /><figcaption>At the finish line, with the medal!</figcaption></figure><p>Here’s my preparation leading up to race day. I created a fast track version of my training schedule, I don’t have a running coach. If you can afford to, getting an ultra marathon running coach is definitely worth it. But if you can’t afford it or don’t want to, there’s plenty of resources out there to create your own training plan. I recommend checking out <a href="https://www.youtube.com/channel/UCwJaLh1vVoPcEL4EHW7dxog">Ultra Marathon Trail Running with Simon</a>. Simon does exactly this, he gives you the knowledge you get from a coach but via his YouTube channel (for free!).</p><p>So, since this wasn’t my first Trail Ultra I had a training plan that’s worked for me. I shaved 2 weeks off it on both ends and designed it with the same basic algorithm; ramp up from your regular weekly mileage (35k/week in my case) to a weekly mileage greater than the distance of your race and then taper down and have a rest week leading up to the big race. Having a plan is great, but its no good if it isnt practiced. I followed it religiously, I mean hardcore(ly?!) religious! With one exception. I fell ill after my last long training run (~42k), after which I was supposed to have a taper week. Guess what? Let’s re-do the schedule, no taper week anymore! My already short schedule is now further shortened by a week. I’m going in blind, with 2 full weeks of just 1 run; an 8k.</p><p>I drove 17hrs with my family to the Gaspésia National Park, to the beautiful small village of Percé, Quebec. With a day to spare before the race. That day we hung out with my trail running fam, got my gear + drop bag sorted and picked up my bib + race package. To end the day, we did a ritual shakeout run (~4k). Highlight: One of my buddies announced they’re pregnant! ❤</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*MXI3mRlSTsJIKIyk73Selw.jpeg" /><figcaption>My bib</figcaption></figure><p>After close to 10 hours in bed, it was race day. Here are some minutes leading up to the start and finish of the race. The race starts at 10.30 but you’re supposed to be at the muster point for 10 am to board the bus that drives you to the trailhead. The night while sleeping I felt a blister on my right big toe, it’s the same one that flared up during training. I tape it up real good! And hope for the best but prepare for the worst! On race day, I’m surprisingly not anxious or scared. I’m smiling through and through, still ready to fail! And excited to do everything for myself to allow me to finish strong with ease. It gets even better when I meet up with my buddies next to the muster point. We help each other with last minute stuff, it takes a lot of gear for a Trail Ultra and it’s easy to miss the thing you’re gonna need the most during the run. We walked to the muster point, together. All smiles, cracking dad jokes and laughing really loud! Mostly about how we’ve hydrated so much that we’re now using the washroom every 5mins. (TMI?!, no, keeping it real!)</p><p>We started off at the trailhead with us getting a picture clicked with the entire SJTR (Saint John Trail Running) crew that was running the 54k.</p><p>In that moment as we begin the race, I ponder “I am here starting an Ultra with a half-training and a blister, I should be very anxious that I am not going to be able to finish. But I am not?! Why?” The answer was, my brain was, “Ready to Fail BIG!”. I was no longer afraid of failing, in fact, I was happy to be failing at something bigger than I had ever failed at before. Once this fear of failure was out of the way, I was <em>actually</em> able to focus on enabling myself to succeed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xUhr1sl1UetmeYwG6fU97g.jpeg" /><figcaption>2k in, ready to Fail BIG!</figcaption></figure><p>Thanks for reading thus far! Failing BIG has really helped me unleash my true self and maybe it can help you do the same at running or work, or as in my case, with both!</p><p>I look forward to continue to Fail BIG(GER) at longer Trail Ultras and at bigger goals in general!</p><p>Special shoutout to Aum for his continued inspiration (co-owner of <a href="https://runtrimag.com/">Run Tri Bike</a> and, an inspiring Ultra Trail Runner).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0113a4d2da3f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Sandbox Org on Google Cloud]]></title>
            <link>https://medium.com/google-cloud/a-sandbox-org-on-google-cloud-b5da203555d7?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/b5da203555d7</guid>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Fri, 03 May 2024 14:54:00 GMT</pubDate>
            <atom:updated>2024-05-06T02:20:41.046Z</atom:updated>
            <content:encoded><![CDATA[<h4>Options, Recommendations and Setup Steps</h4><p>As you mature your cloud practices on Google Cloud, you might be faced with a fork in the road to setup multiple organizations within Google Cloud. There could be a few motivations behind this, like a separate Org for different companies under a parent company, a spin-off or divesting. But the most common use case and the one which I personally feel is the most valid one; “A separate Google Cloud Organization for sandboxing and prototyping quickly.” As you may be familiar that each CO is tied to a Cloud Identity (CI), mandatorily so. This context is helpful for inferring the text below.</p><p>We acknowledge this as a valid use case for the below reasons:</p><ol><li>Google Cloud moves fast with new services and features launched almost every day and you might find yourself interested in them but want to test your use cases first. With a sandbox org you can iterate fast (like really fast) if you set up your sandbox org with less constraints and a fully self-serve model.</li><li>For enterprises in highly regulated industries like Healthcare and Financial Services, it can help reduce the blast radius for any mishaps during prototyping.</li><li>You can easily setup lifecycle policies and enable short-lived environments. That is, periodically and automatically destroy all environments in the sandbox org.</li><li>Separate your prototyping and testing costs from Development and Production.</li></ol><p>There are a few different possible patterns for having a multiple org setup on Google Cloud:</p><ol><li>Single Cloud Identity, Multiple Cloud Organizations</li><li>Multiple Cloud Identities, Multiple Organizations</li></ol><p>The scope of this post is limited to the 1st use case i.e. Single CI and Multiple CO. Let’s double click on the use case for a minute. The assumption here is that Company X is running a Google Cloud CO and CI with their primary (and possibly secondary domains), lets call this ‘companyx.com’. Company X has an external IdP like Azure AD, AD etc. that they have synced to their CI for their primary org. For their sandbox org, Company X has registered a subdomain ‘sandbox.companyx.com’ and does not intend to setup another IdP as they would like for their existing domain users to be able to access both the orgs from the same accounts. This is also the recommended pattern for setting up a sandbox organization. Setting up an explicit IdP or and IdP sync or federation just for the sandbox organization is an overkill and also contributes to a negative UX that is human error prone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qz2xqT2dM1ZcwtCtRwqAxg.png" /></figure><p>Let’s begin by acknowledging the elephant in the room, there is inherent complexity that comes from the need to set up the right combination of cloud organizations (CO) and cloud identity (CI) accounts. Cloud Identity accounts and GCP orgs come in pairs, but in cases where we wanna operate multiple orgs (child orgs) it is worthwhile to think of them separately. Each CO is associated with a distinct CI. Each CI is, in turn, associated with a specific domain and can only accept identities associated with the same domain.</p><p>The above constraints apply to this situation. Here are some useful Google-native concepts that solve for these constraints:</p><ol><li>Sub-domains can be registered as primary domains within CI.</li><li>Configuring an IdP with CI is not a pre-requisite to setup a CO.</li><li>Identities in Google are global by default.</li></ol><p>With the above important concepts in mind, let’s move to the next section where we will walk through the steps that are needed to setup the sandbox org on Google Cloud.</p><h4>Steps for setting up the ‘sandbox.companyx.com’ Org</h4><p>The pre-requisite for the steps is to have the primary CO associated with the CI which in turn is associated with the company’s primary domain (in this example, companyx.com). This CI is integrated with the client’s primary identity store and gets synchronized with all users and groups who will have access to both COs.</p><ol><li>Ensure that the ‘sandbox.companyx.com’ subdomain does not exist within the primary CI as secondary or alias domain. If it does, delete it and wait 24 hours for it to be purged on the Google backend before proceeding further.</li><li>Setup a greenfield Google Cloud CI using ‘sandbox.companyx.com’ as the primary domain. Navigate to the <a href="https://workspace.google.com/gcpidentity/signup?sku=identitybasic">Cloud Identity sign up page</a> to begin the process.</li><li>In the About you section, enter your first and last name in the Name field.</li><li>In the Current email address you use for work field, enter your working email address eg. ‘bob@companyx.com’.<br>This email address will be used as a recovery email address.<em> It must be different</em> from the address you create below that you’ll use as your admin account for CI.</li><li>In the About your business section, enter your company name in the Business or organization name field.</li><li>In the Country/Region field, choose the appropriate country or region from the pulldown list.</li><li>Click Next to set up your domain.</li><li>In the Your Cloud Identity Domain window you’ll add the secondary domain you’ve already purchased for your company. You’ll need to verify that you own it by creating a specific CNAME record or uploading an html file. eg. ‘sandbox.companyx.com’</li><li>In the Create your Cloud Identity account window, enter a username and password. This account is your Cloud Identity administrator account and must be different from the email address you entered in step 2 above. As a best practice, we recommend that you enter a username with a format similar to: <a href="/service/mailto:admin@gcp-sdbx.cvshealth.com"><em>admin@sandbox.companyx.com</em></a>. Treat this email as a username and not a functioning email. This will be used to login locally to your CI. Recommend to save the credentials in an enterprise vault.</li><li>Once the domain verification is complete move to the next steps in the Cloud Console. More detailed steps for it can be found <a href="https://support.google.com/cloudidentity/answer/7331243">here</a>.</li><li>The first thing you want to do is to navigate to IAM in your new org and remove the secondary domain from the access.</li><li>The next thing to do is to add your primary domain eg. ‘companyx.com‘ as Org viewer and Folder viewer (optional) in IAM. After this step is complete all the principals with Google Cloud access in your primary domain will start to see the sandbox org in their cloud console.</li><li>Now you can add relevant groups associated with your primary domain to setup your resource hierarchy (folders, projects). To do that, add the group(s) to IAM as Org Admin, Folder Admin and Folder Creator.</li></ol><p>There you have it, now you have a working secondary org (though there is technically no such thing) and you are able to leverage the identities that you setup as part of your primary org setup. There are additional capabilities that you can tap into here depending on your use case eg. running through the Google Cloud Setup steps and leveraging baked-in designs and configurations for resource hierarchy etc., you could export the IAM bindings from your primary CO into a .json or .yaml and import them into the new CO in case you want to replicate them.</p><h4>Additonal recommendations and gotchas</h4><ul><li>Which users should have access to which of the organizations can be indicated by membership in specially defined groups, e.g. PrimaryUsers and SandboxUsers, which do not have any permissions but indicate ability to use the corresponding org. The actual permissions will need to be derived from membership in other groups that need to be carefully named to avoid confusions and conflicts: e.g. gcp-primary-network-admin vs gcp-sandbox-network-admin, etc. And <a href="https://forsetisecurity.org/">forseti</a> can be set up to monitor that no group is granted permissions in the wrong organization. <a href="https://cloud.google.com/resource-manager/docs/organization-policy/restricting-domains">Domain restricted sharing org policy</a> can be used to limit resource sharing based on domain.</li><li>You can go even further and choose to create separate (e.g. bob-sandbox@companyx.com) or even one-time (e.g. bob-sandbox-pilot123@acme.com) identities for the sandbox to prevent same principals from having entitlements in both places.</li><li>Use nested groups: create a group sandbox-owners in acme-sandbox.com and include in it a single group named sandbox-project-owners@acme.com. Manage all project owners in the latter group in acme.com. So the purpose here is that if your same Principal has access to both orgs, then the chances of you making mistakes and / or unintended behavior are higher + the auditability is tougher and would take deeper analysis. If you have a separate principal that’s for sandbox org only, then these pain points are avoided.</li><li>Use individual ownership: e.g. bob@acme.com can own a project in the sandbox and individual owners should be ok for short-lived non-production PoCs.</li><li>Each CO is associated with a primary billing account. A single account <a href="https://cloud.google.com/billing/docs/concepts#relationships-between-resources">can be used across multiple organizations</a> and this is usually the best approach since it would maximize discounts and simplify setting up support. If required, separate billing accounts can be used as well.</li><li>Because of the complexity associated with project ownerships and managing commonalities such as Org policies, if more than 2 Orgs are created. I would advise to limit to 2 orgs. Instead, folder based org representation should be the go-to choice for such scenarios.</li><li>In case you need to move any existing projects under the sanbox CO, see <a href="https://cloud.google.com/resource-manager/docs/organization-setup#migrate_existing">Migrating projects into an organization</a>.</li><li>Some more relevant links:<br>- <a href="https://support.google.com/a/answer/1275816">‘Google Account already exists’</a> error.<br>- <a href="https://support.google.com/a/answer/80610?hl=en&amp;ref_topic=1687139&amp;sjid=15292191284107086571-NC">Domain sign up error</a> with CI.<br>- <a href="https://support.google.com/cloudidentity/answer/7389975?hl=en&amp;ref_topic=7555414&amp;sjid=15292191284107086571-NC">Steps to complete</a> from the console.</li></ul><p>Thanks for reading. Hope you found this useful! I’m always looking for ways to improve my writing skills and for topics that could use more clarity. Pls leave any feedback you have in comments. :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b5da203555d7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/a-sandbox-org-on-google-cloud-b5da203555d7">A Sandbox Org on Google Cloud</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Quota Monitoring and Management Options on Google Cloud]]></title>
            <link>https://medium.com/google-cloud/quota-monitoring-and-management-options-on-google-cloud-b94caf8a9671?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/b94caf8a9671</guid>
            <category><![CDATA[gcp-security-operations]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[google-cloud]]></category>
            <category><![CDATA[cloud-monitoring]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Wed, 06 Mar 2024 14:52:02 GMT</pubDate>
            <atom:updated>2024-03-07T02:56:20.217Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/388/1*jIgDkmKqsjX7DgF7aouyUw.png" /></figure><h3>Introduction</h3><p>This article is most relevant to enterprise that use Google Cloud at scale across various services and push GCP services beyond their threshold quota levels. If your pattern is to use shared projects for multiple teams, which further enhances the limits to which a GCP service might be used within a project, you are at the right place. This premise sets us up with the problem statement, which is the need to request Quota increases frequently to avoid service interruption / negative user experience.</p><p>In this article we will summarize some of the options that are available to use for monitoring and managing Quota / usage and the monitoring of these. The options range from simple to deploy and manage, to complex with overhead management needs.</p><p>I recommend evaluating all the options and following the “less is more” approach. Start with the OOB (Out Of Box) Quotas Page and the associated APIs, also, if you are an enterprise GCP customer then you likely also have a Google Cloud Technical Account Manager (TAM)working with you, they are a magic wand that’s available to you to assist with QIRs and Capacity management in general. If those no longer work for your use case, then evaluate the Quota Monitoring Solution.</p><h3>Quotas Page OOB</h3><p>This section describes the OOB (Out Of Box) capabilities that are present on the “all quotas” page within a project. This is of course the least implementation heavy option and can easily fulfill basic use cases. To reach the page, type “all quotas” in the top search bar within a project. This will bring you to “Quotas and system limits” page.</p><p>Some useful features that are available within the page are described below.</p><ol><li>OOB Service usage Dashboard: The page has a dashboard that is pre-populated with all of the Google Cloud services and their respective usage percentages. You can order the dashboard by descending order of usage percentage among other things.</li><li>Alerting policies: The page provides you with 2 options for creating service usage alerts; 1) The first one is per service, you can hover on any service on the service usage dashboard and configure an alerting policy at a threshold of your choice with a notification channel of your choice. 2) The second one is the ability to create a blanket alerting policy for multiple services at once. To set this up, click on “manage alert policies”, which will take you to the alerting setup, here you can set up your alerts using the “builder” with predefined metrics or use (Prom)MQL to write your own query for alerting. For eg. Below code is a sample code for setting an alert policy for service usage for all services within a project and it alerts when the service usage spikes above 80%. To tweak the threshold, modify the “condition gt(ratio, 0.8 ‘1’)”</li></ol><blockquote>{</blockquote><blockquote>“displayName”: “Quota Monitoring Alerting”,</blockquote><blockquote>“documentation”: {</blockquote><blockquote>“subject”: “Quota usage threshold exceeded 80%, proactive action recommended”</blockquote><blockquote>},</blockquote><blockquote>“userLabels”: {},</blockquote><blockquote>“conditions”: [</blockquote><blockquote>{</blockquote><blockquote>“displayName”: “Quota Monitoring Chart”,</blockquote><blockquote>“conditionMonitoringQueryLanguage”: {</blockquote><blockquote>“duration”: “3600s”,</blockquote><blockquote>“trigger”: {</blockquote><blockquote>“count”: 1</blockquote><blockquote>},</blockquote><blockquote>“query”: “fetch consumer_quota\n| filter resource.service =~ ‘.*’\n| { t_0:\n metric ‘serviceruntime.googleapis.com/quota/allocation/usage’\n | align next_older(1d)\n | group_by [resource.project_id, metric.quota_metric, resource.location],\n [value_usage_max: max(value.usage)]\n ; t_1:\n metric ‘serviceruntime.googleapis.com/quota/limit’\n | align next_older(1d)\n | group_by [resource.project_id, metric.quota_metric, resource.location],\n [value_limit_min: min(value.limit)] }\n| ratio\n| every 1m\n| condition gt(ratio, 0.8 ‘1’)”</blockquote><blockquote>}</blockquote><blockquote>}</blockquote><blockquote>],</blockquote><blockquote>“alertStrategy”: {</blockquote><blockquote>“autoClose”: “604800s”</blockquote><blockquote>},</blockquote><blockquote>“combiner”: “OR”,</blockquote><blockquote>“enabled”: true,</blockquote><blockquote>“notificationChannels”: [</blockquote><blockquote>“projects/$project-name$/notificationChannels/$notifcation-channel$”</blockquote><blockquote>],</blockquote><blockquote>“severity”: “SEVERITY_UNSPECIFIED”</blockquote><blockquote>}</blockquote><h3>Terraform Implementation</h3><p>A super neat capability that I personally love, is the ability to set all of this up via code (IaC). No need to manage this within your environment as an outlier from within the Google Cloud UI, you can make it live with your Terraform configuration and manage it centrally! Refer to the below sample Terraform configuration.</p><blockquote># Notification Channel</blockquote><blockquote>resource “google_monitoring_notification_channel” “default” {</blockquote><blockquote>display_name = “Quota Monitoring Alert”</blockquote><blockquote>type = “email”</blockquote><blockquote>project = var.project_id</blockquote><blockquote>labels = {</blockquote><blockquote>email_address = “example@yourenterprise.com”</blockquote><blockquote>}</blockquote><blockquote>}</blockquote><blockquote># Alert</blockquote><blockquote>resource “google_monitoring_alert_policy” “default” {</blockquote><blockquote>display_name = “[${upper(var.environment)}] ${title(“Quota Monitoring”)}”</blockquote><blockquote>enabled = true</blockquote><blockquote>project = var.project_id</blockquote><blockquote>combiner = “OR”</blockquote><blockquote>conditions {</blockquote><blockquote>display_name = “main-condition”</blockquote><blockquote>condition_monitoring_query_language {</blockquote><blockquote>query = “fetch consumer_quota| filter resource.service =~ ‘.*’| { t_0: metric ‘serviceruntime.googleapis.com/quota/allocation/usage’ | align next_older(1d) | group_by [resource.project_id, metric.quota_metric, resource.location], [value_usage_max: max(value.usage)] ; t_1: metric ‘serviceruntime.googleapis.com/quota/limit’ | align next_older(1d) | group_by [resource.project_id, metric.quota_metric, resource.location], [value_limit_min: min(value.limit)] }| ratio| every 1m| condition gt(ratio, 0.8 ‘1’)”</blockquote><blockquote>duration = “60s”</blockquote><blockquote>trigger { count = “1” }</blockquote><blockquote>}</blockquote><blockquote>}</blockquote><blockquote>alert_strategy {</blockquote><blockquote>auto_close = “86400s”</blockquote><blockquote>}</blockquote><blockquote>notification_channels = [</blockquote><blockquote>google_monitoring_notification_channel.default.name</blockquote><blockquote>]</blockquote><blockquote>}</blockquote><p>Documentation reference for more details: <a href="https://cloud.google.com/docs/quota/monitor#quota_alerts">Monitor and alert with quota metrics | Documentation | Google Cloud</a></p><p>3. Quota Increase Requests: Another natively supported capability on the page is to submit and manage Quota Increase requests associated with a project. You can switch to the “Increase requests” tab to create new requests and manage previous ones.</p><h3>Quota Monitoring Solution (QMS)</h3><p>If you are looking for an automated and comprehensive way to manage quotas over a large number of projects, we have an internally built Quota Monitoring Solution. It has been adopted by more than a 100 customers and is supported by an implementation and management team at Google Cloud.</p><p>This solution is more complex and demands additional overhead. This is a more aggregated and comprehensive solution that should be managed by a team that is responsible for managing multiple folders, projects and likely an organization within GCP.</p><p>Some key features:</p><ul><li>A dashboard for real-time quota visibility.</li><li>Aggregated folder / organization quotas.</li><li>Alerting via emails or other tools.</li><li>Easy and scalable deployment.</li></ul><p><a href="https://cloud.google.com/blog/products/gcp/google-cloud-introduces-quota-monitoring-solution">Read more about Quota Monitoring Solution.</a></p><p><a href="https://github.com/google/quota-monitoring-solution/tree/main">Check out the code to deploy it here.</a></p><p><a href="https://github.com/google/quota-monitoring-solution/issues">Solution roadmap.</a></p><h3>Quotas associated API’s</h3><h3>Service Limit / Quota Recommender</h3><p>The service limit recommender analyzes usage of service quotas by projects in your organization and provides recommendations that help you identify resources that may be getting close to their quota limits. Service limit recommender analyzes your quota utilization and provides you with the following features to help you catch potential bottlenecks before they become an issue:</p><ul><li>Recommendations to review quotas with high utilization</li><li>Usage insights for each quota with high utilization</li></ul><p>The service limit recommender analyzes usage over rate, allocation, and concurrent quotas over the last 30 days. If at any point during those 30 days your utilization has hit 80% of your current limit, a recommendation will be generated.</p><p>Before you can view the insights and recommendations, you must do the following:</p><ul><li>You must enable the Recommender API. You only need to enable the API on a single project. You can then use this same project to examine recommendations and insights for other projects by using the <a href="https://cloud.google.com/sdk/gcloud/reference#--billing-project">— billing-project</a> functionality of gcloud/API.</li><li>Make sure that you have one of these required roles assigned:<br>View recommendations: recommender.serviceLimitViewer<br>View and update recommendations: recommender.serviceLimitAdmin</li></ul><p><a href="https://cloud.google.com/recommender/docs/service-limit-quota-recommender">Learn more about the Service limit (quota) recommender.</a></p><h3>Quota Adjuster</h3><p>Note that Quota Adjuster is Pre-GA (at the time of writing) and is subject to the “Pre-GA Offerings terms”.</p><p>Once you enable quota adjuster on a project, it begins monitoring all applicable quotas and applies the following logic:</p><ul><li>Quota adjuster checks if the peak usage has approached the quota value during a specified duration.</li><li>If so, quota adjuster attempts to increase the quota value (typically around 10–20%).</li></ul><p>If it’s possible to increase the quota value, the increase is approved and the value adjusted. It is a good call out that the QIR’s made by the Adjuster can be rejected similar to the ones you make manually. You can still manually request increases to quota values at any time, whether or not quota adjuster is enabled. The history of all Quota Increase Requests made by quota adjuster is available in the Quotas page of Google Cloud console. You can also set alerts to monitor any changes initiated by Quota Adjuster.</p><p>During this preview, the following Compute Engine quotas are available for automated increases using quota adjuster:</p><ul><li>CPUs</li><li>N2 CPUs</li><li>N2D CPUs</li><li>C2 CPUs</li><li>C2D CPUs</li><li>Persistent Disk Standard (GB)</li></ul><p>To enable quota adjuster on your Google Cloud project:</p><ol><li>Navigate to the IAM &amp; Admin &gt; Quotas page.</li><li>Click the Configurations tab.</li><li>Click the Enable switch.</li></ol><p>After the preview period, Quota Adjuster is planned to be a default configuration in all new projects, with an option to opt-out.</p><h3>Cloud Quotas API</h3><p>The Cloud Quotas API lets you programmatically adjust quotas and automate quota adjustments in your Google Cloud projects, folders, or organizations. Adjustments can be made to increase or decrease quota values.</p><p>The Cloud Quotas API can be used to:</p><p><strong>Automate quota adjustments</strong></p><p>You can use the Cloud Quotas API to request quota increases when certain conditions are met. For example, to avoid quota exceeded errors, you can use the API to programmatically request a quota increase when Compute Engine resources reach 80% of the available quota.</p><p><strong>Scale quota configurations across projects</strong></p><p>The Cloud Quotas API can clone your quota configurations from project to project. If there is a known set of quotas that need to be increased for every new Google Cloud project, you can integrate the API into the creation logic of your project to automatically issue a QIR. All quota increases are subject to Google Cloud review and approval.</p><p><strong>Serve customer quota requests</strong></p><p>If you are a SaaS provider integrated with Google Cloud, you may receive quota increase requests through a customer-facing portal other than Google Cloud console. These requests must be forwarded to Google Cloud for processing. The Cloud Quotas API can automatically forward such customer requests.</p><p><strong>Enable client configuration version control</strong></p><p>The Cloud Quotas API is declarative, you can treat quota configs as code and store configurations in your own version controlled system for history and rollback.</p><p><a href="https://cloud.google.com/docs/quotas/api-overview">Learn more about the Cloud Quotas API and managing quotas using API.</a></p><p>In conclusion, there are quite a few Quota Monitoring and Management options like I described above. If you are just attempting to solve for this problem, start with the basics — All quotas page, Quotas API and your TAM (Technical Account Manager), then work your way up as and when needed to the Quota Monitoring Solution with which your Google Cloud TAM can help out as well.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b94caf8a9671" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/quota-monitoring-and-management-options-on-google-cloud-b94caf8a9671">Quota Monitoring and Management Options on Google Cloud</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When to allow your environment to drift? and other edge cases for Terraform.]]></title>
            <link>https://medium.com/google-cloud/when-to-allow-your-environment-to-drift-and-other-edge-cases-for-terraform-a45b290bf013?source=rss-dd01c5137cb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a45b290bf013</guid>
            <category><![CDATA[gcp-app-dev]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[continuous-deployment]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Vipul Raja]]></dc:creator>
            <pubDate>Mon, 21 Nov 2022 03:13:41 GMT</pubDate>
            <atom:updated>2022-11-21T03:13:41.567Z</atom:updated>
            <content:encoded><![CDATA[<p>Terraform, the industry standard for provisioning Infrastructure in the cloud is great for many reasons — its Open Source, cloud agnostic, enables single click instantiations and immutable infrastructures. This brings us to the problem at hand, which goes hand-in-hand with one of the key benefits of Terraform i.e. immutability and environmental drift.</p><p>Let’s start by talking about environment drift. Environment drift is the phenomenon where the Infrastructure code and the real-time environment state do not match. Thereby causing a drift. If you work in the Cloud-native space you know this is frowned upon and rightly so, because it goes against the Cloud-native principle that your Infrastructure Code should be the single source of truth for your environment. But is there an anti-pattern to this?</p><p>Let’s look at a scenario where we wouldn’t want environment drift to be corrected. Let’s say our Infrastructure code states that we have 3 labels — LOB name, Cost centre and Billing ID. <em>Imagine that you are running a self-serve platform on a public cloud and use these labels to charge the appropriate department for their cloud consumption.</em> These labels are persistent and are created as part of the automated infrastructure instantiation pipelines. And are therefore tracked in the “terraform.tfstate” file which is Terraform’s map of real world resources to your configuration, keeps track of metadata; in simpler terms it captures the real state that your environment was in when Terraform “apply” or “refresh” was last run. If one of these labels’ key-value pairs were to be updated from the console or CLI, the environment state would not match the “terraform.state” - causing a drift.</p><p>As a result, if you re-run terraform for the environment, it will re-write these values to match the Infrastructure code. <em>This is indubitably one of the strongest abilities of Terraform and why it is so widely accepted.</em> And it also brings us to the edge case where let’s say a re-org happens (Does happen more than we’d like, right?) and the cost centre and Billing ID have been changed as a result. This is a critical update that will determine who to bill for the respective resources and it could also be time sensitive if your billing cycle is ending soon. There is also another factor to consider that these changes might be frequent.</p><p>We have 2 options, go and update the Terraform code to reflect the change or go and update the key-value pair via the console or CLI.</p><p>Updating the values outside of Terraform might be favourable in the above scenarios where getting the code updated is an overhead and / or the resources that manage the Cost Centre and Billing ID’s want the freedom to to manage these resources in the cloud without engaging the team that manages Terraform.</p><p>Luckily, Terraform thought of and accepted this edge case and built a feature for it.</p><p>Enter (drum roll, please) “The lifecycle Meta-Argument”. lifecycle is a nested block that can appear within a resource block. The lifecycle block and its contents are meta-arguments.</p><p><strong><em>Note: It is available for all </em></strong><strong><em>resource blocks regardless of type.</em></strong></p><p>The argument within the lifecycle resource block that is specific to our edge case here is the ignore_changes meta-argument.</p><p>“By default, Terraform detects any difference in the current settings of a real infrastructure object and plans to update the remote object to match configuration.”</p><p>The ignore_changes meta-argument is intended to be used when a resource is created with references to data that may change in the future, but should not impact said resource after its creation. In some rare cases, settings of a remote object are modified by processes outside of Terraform, which Terraform would then attempt to &quot;fix&quot; on the next run. In order to make Terraform share management responsibilities of a single object with a separate process, the ignore_changes meta-argument specifies resource attributes that Terraform should ignore when planning updates to the associated remote object.</p><p>The arguments corresponding to the given attribute names are considered when planning a <em>create</em> operation, but are ignored when planning an <em>update</em>. The arguments are the relative address of the attributes in the resource. Map and list elements can be referenced using index notation, like tags[&quot;Name&quot;] and list[0] respectively.” <em>— from Terraform webpage</em></p><p>The text above means that the attributes that are added to the ignore_changes meta argument will still be created and destroyed on “terraform applies and destroys” but will not be updated (re-written or fixed) on subsequent “terraform apply”. This exactly solves our problem. Whenever a business event leads to a change in the “Cost center” and “Billing ID”, the associated FinOps Team(s) can go and update the key:value pairs of these labels without having to worry about Terraform and the associated delay(s).</p><p>Here’s what that would like in action:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zbJtZ5yBX42LFiiQ" /><figcaption>ignore_changes meta-argument</figcaption></figure><p>The other arguments available within a lifecycle block are create_before_destroy, prevent_destroy and replace_triggered_by. All of these arguments have unique edge cases that they are apt for. Lets look at them on a high level.</p><p>create_before_destroy : By default, when Terraform must change a resource argument that cannot be updated in-place due to remote API limitations, Terraform instead destroys the existing object and then creates a new replacement object with the new configured arguments. The create_before_destroy meta-argument changes this behaviour so that the new replacement object is created <em>first,</em> and the prior object is destroyed after the replacement is created. This can be useful for edge cases where we do not want downtime i.e. always want a copy of the resource to be running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8XvleNiESw2PJypH" /><figcaption>create_before_destroy meta-argument</figcaption></figure><p>prevent_destroy: This meta-argument, when set to true, will cause Terraform to reject with an error any plan that would destroy the infrastructure object associated with the resource, as long as the argument remains present in the configuration. This can be used as a measure of safety against the accidental replacement of objects that may be costly to reproduce, such as business critical compute. prevent_destroy provides an added layer of protection against accidental deletion, or for resources that you want managed outside of Terraform. The destruction of resources that have the meta-argument set to “true” requires an additional step confirming that you “really” want the resource deleted. You would either make a change to the Terraform code that would go through a PR process before it gets committed or you would have to go to the console and delete the resource manually (which ideally only a handful people would have permissions to do).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l7VxQPtzeEMyRDYuz6YkSg.png" /><figcaption>prevent_destroy meta-argument</figcaption></figure><p>replace_triggered_by: Replaces a resource when any of the referenced resources are changed. Note that change here includes modification and deletion. You have the ability to supply a list of expressions referencing multiple resources.</p><p>References trigger replacement in the following conditions:</p><ul><li>If the reference is to a resource with multiple instances, a plan to update or replace any instance will trigger replacement.</li><li>If the reference is to a single resource instance, a plan to update or replace that instance will trigger replacement.</li><li>If the reference is to a single attribute of a resource instance, any change to the attribute value will trigger replacement.</li></ul><p>For example, in the below scenario we want the Load Balancer and Instance Template resources to be coupled with each other. If we remove the instance template we definitely wouldn&#39;t need the backend service anymore.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o1QtYi4vzUqgZ5UlQeIX7w.png" /><figcaption>replace_triggered_by meta-argument</figcaption></figure><p>I was prompted to write this article as I feel that the edge cases that these meta-arguments set out to address are becoming more and more common as larger enterprises adopt Terraform. Hope this article helps reduce the time it takes for you to discover that the edge case you are working with has been thought of.</p><p>To learn more about the lifecycle meta-arguments and their implementations, visit the official Hashicorp page below.</p><p><a href="https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle">https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a45b290bf013" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/when-to-allow-your-environment-to-drift-and-other-edge-cases-for-terraform-a45b290bf013">When to allow your environment to drift? and other edge cases for Terraform.</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>