<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Google Cloud for Startups - Medium]]></title>
        <description><![CDATA[Google Cloud for Startups - Medium]]></description>
        <link>https://medium.com/google-cloud-for-startups?source=rss----5bec3fa3633b---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Google Cloud for Startups - Medium</title>
            <link>https://medium.com/google-cloud-for-startups?source=rss----5bec3fa3633b---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 12:49:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/google-cloud-for-startups" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Four Steps for Startups to Master Visual Document Understanding with AI]]></title>
            <link>https://medium.com/google-cloud-for-startups/four-steps-for-startups-to-master-visual-document-understanding-with-ai-c25785722fb2?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/c25785722fb2</guid>
            <category><![CDATA[ocr]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[google-gemini]]></category>
            <dc:creator><![CDATA[Patrick Guha]]></dc:creator>
            <pubDate>Fri, 08 May 2026 18:40:13 GMT</pubDate>
            <atom:updated>2026-05-08T19:43:19.169Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ogu5yeBstizJpKEoWM1g5w.png" /></figure><p>AI agents are transforming the nature of work by automating complex workflows with speed, scale, and accuracy. For AI startups — whether you are building the next-generation AI assistant for legal professionals, automating digital mortgage and real estate closings, streamlining complex estate planning, or pioneering digital musculoskeletal health and physical therapy — processing complex, highly-formatted documents is at the core of your product.</p><p>Contracts, loan applications, and clinical records are rarely just plain text. They are densely packed with tables, signatures, charts, and highly specific spatial layouts. Traditionally, developers had to rely on brittle pipelines involving disparate optical character recognition (OCR) tools and text-only large language models (LLMs).</p><p>Today, native multimodal models have changed the game. Working closely with our DeepMind and Google Cloud AI teams, we’ve developed a simple four-step framework to help your startup build reliable, highly scalable, and economical document understanding systems using Gemini.</p><h3>Step #1: Skip the OCR pipeline and leverage dynamic tokenization</h3><p>For complex document processing, the golden standard is now direct inference through native multimodal models. Instead of preprocessing documents through a traditional OCR engine, pass the PDF pages directly to Gemini. Models like <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/gemini/3-1-pro">Gemini 3.1 Pro</a>, <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/gemini/3-flash">3 Flash</a>, and <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/gemini/3-1-flash-lite">3.1 Flash-Lite</a> natively understand document layouts, tables, and embedded images.</p><p><strong>Optimize your PDF format:</strong> For text-heavy workloads like master service agreements or estate planning documents, try to ingest native PDFs (where text is rendered as text) rather than flat scanned images. This makes the text machine-readable, which is far easier for the model to edit, search, and manipulate.</p><p><strong>Mastering Gemini 3 Tokenization:</strong> With the Gemini 3 models, document tokenization uses a variable sequence length, replacing the older Pan and Scan method for better quality and latency. You can now tightly control costs and performance by explicitly setting the media_resolution for your PDF inputs:</p><ol><li><strong>UNSPECIFIED (560 tokens/page):</strong> The default setting. The token count for this level varies significantly between Gemini 3 and earlier Gemini models.</li><li><strong>LOW (280 tokens/page):</strong> Best for simple text extraction or global summaries.</li><li><strong>MEDIUM (560 tokens/page):</strong> A balance between detail, cost, and latency. Ideal for standard document extraction.</li><li><strong>HIGH (1120 tokens/page):</strong> Higher token count, providing more detail for the model to work with, at the expense of increased latency and cost. Reserve this for fine-grained detail on complex layouts, tiny serial numbers on hardware, or dense financial tables.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SeEh5lh4p34oo9XQVTkRkw.png" /></figure><p><em>Pro-tip:</em> You can set these resolutions at the top level of your generationConfig, or override them for individual media parts if you have a mixed batch of high-res charts and low-res text pages.</p><p><strong>Example</strong>: Set media_resolution per individual media part:</p><pre>response = client.models.generate_content(<br>  model=&quot;gemini-3.1-pro-preview&quot;,<br>  contents=[<br>      types.Part(<br>          file_data=types.FileData(<br>              file_uri=&quot;gs://cloud-samples-data/generative-ai/image/a-man-and-a-dog.png&quot;,<br>             mime_type=&quot;image/jpeg&quot;,<br>          ),<br>          media_resolution=types.PartMediaResolution(<br>              level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_HIGH  # High resolution<br>          ),<br>      ),<br>      Part(<br>          file_data=types.FileData(<br>             file_uri=&quot;cloud-samples-data/generative-ai/video/behind_the_scenes_pixel.mp4&quot;,<br>            mime_type=&quot;video/mp4&quot;,<br>          ),<br>          media_resolution=types.PartMediaResolution(<br>              level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_LOW  # Low resolution<br>          ),<br>      ),<br>      &quot;When does the image appear in the video? What is the context?&quot;,<br>  ],<br>)<br>print(response.text)</pre><p><strong>Example</strong>: Or set media_resolution globally (via GenerateContentConfig)</p><pre>response = client.models.generate_content(<br>  model=&quot;gemini-3.1-flash-lite&quot;,<br>  contents=[<br>      types.Part(<br>          file_data=types.FileData(<br>              file_uri=&quot;gs://cloud-samples-data/generative-ai/pdf/1706.03762v7.pdf&quot;,<br>              mime_type=&quot;application/pdf&quot;,<br>          ),<br>      ),<br>      &quot;Please summarize the given document for a general audience.&quot;,<br>  ],<br>  config=types.GenerateContentConfig(<br>media_resolution=types.MediaResolution.MEDIA_RESOLUTION_MEDIUM  # Global setting<br>  ),<br>)<br>print(response.text)</pre><h3>Step #2: Prompt engineering for spatial and mixed-media context</h3><p>Many startups, particularly innovative legal case management platforms, deal with documents that mix dense text with embedded images (like exhibits in legal PDFs). When reasoning over these mixed-media documents, send the full PDF pages <em>as-is</em> to Gemini. Because Gemini processes the entire page visually, it preserves the spatial relationship between the text and the images.</p><p>To get the most out of the model, adhere to these structural prompt best practices:</p><ul><li><strong>Order matters:</strong> If your request contains a single PDF, always place the PDF file <em>before</em> the text prompt in your API request.</li></ul><p><strong>Example</strong>: Same example as above with a special callout to the ordering of the PDF Part object first then text the prompt/instruction in the Contents message</p><pre>text_part = &quot;&quot;&quot;<br>You are a professional document summarization specialist. Please summarize the given document for a general audience.&quot;&quot;&quot;<br><br>response = client.models.generate_content(<br>  model=&quot;gemini-3.1-flash-lite&quot;,<br>  contents=[<br>      types.Part(<br>          file_data=types.FileData(<br>              file_uri=&quot;gs://cloud-samples-data/generative-ai/pdf/1706.03762v7.pdf&quot;,<br>              mime_type=&quot;application/pdf&quot;,<br>          ),<br>      ),<br>      text_part,<br>  ],<br>  config=types.GenerateContentConfig(<br>media_resolution=types.MediaResolution.MEDIA_RESOLUTION_MEDIUM  # Global setting<br>  ),<br>)<br>print(response.text)</pre><ul><li><strong>Split massive files:</strong> If you are dealing with a long document (like a massive mortgage closing packet), consider splitting it into multiple smaller PDFs before processing it to improve retrieval accuracy and parsing speed. There are great tools out there powered by Gemini, such as <a href="https://developers.googleblog.com/build-a-smart-financial-assistant-with-llamaparse-and-gemini-31/">LlamaParse</a>, that can help you perform <a href="https://developers.llamaindex.ai/llamaparse/split/">splitting</a> and other tasks. It is also best practice to rotate pages to the correct orientation before uploading.</li></ul><p><strong>Example</strong>: Create a split job with category definitions with LlamaParse</p><pre>job = client.beta.split.create(<br>    document_input={<br>        &quot;type&quot;: &quot;file_id&quot;,<br>        &quot;value&quot;: file_id,<br>    },<br>    categories=[<br>        {<br>            &quot;name&quot;: &quot;essay&quot;,<br>            &quot;description&quot;: &quot;A philosophical or reflective piece of writing that presents personal viewpoints, arguments, or thoughts on a topic without strict formal structure&quot;,<br>        },<br>        {<br>            &quot;name&quot;: &quot;research_paper&quot;,<br>            &quot;description&quot;: &quot;A formal academic document presenting original research, methodology, experiments, results, and conclusions with citations and references&quot;,<br>        },<br>    ],<br>)<br><br>print(f&quot;✅ Split job created: {job.id}&quot;)<br>print(f&quot;   Status: {job.status}&quot;)</pre><ul><li><strong>Part isolation:</strong> For multi-document reasoning across a corpus, structure your requests so each document is a separate Part in the prompt with clear document identifiers.</li></ul><p><strong>Example</strong>: Interleaving document identifiers inside the prompt of the multi-part Content message with PDFs as unique Parts.</p><pre>response = client.models.generate_content(<br>    model=&quot;gemini-3.1-flash-lite&quot;,<br>    contents=[<br>        &quot;Document 1: Master Service Agreement&quot;,<br>        types.Part(<br>            file_data=types.FileData(<br>                file_uri=&quot;gs://&lt;SAMPLE-BUCKET&gt;/msa_doc.pdf&quot;,<br>                mime_type=&quot;application/pdf&quot;,<br>            ),<br>            # Apply medium resolution specifically to Document 1<br>            media_resolution=types.PartMediaResolution(<br>                level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_MEDIUM<br>            )<br>        ),<br>        &quot;Document 2: Statement of Work&quot;,<br>        types.Part(<br>            file_data=types.FileData(<br>                file_uri=&quot;gs://&lt;SAMPLE-BUCKET&gt;/sow_doc.pdf&quot;,<br>                mime_type=&quot;application/pdf&quot;,<br>            ),<br>            # Apply medium resolution specifically to Document 2<br>            media_resolution=types.PartMediaResolution(<br>                level=types.PartMediaResolutionLevel.MEDIA_RESOLUTION_MEDIUM<br>            )<br>        ),<br>        &quot;Analyze both documents. Compare the liability clauses in Document 1 with the deliverables outlined in Document 2.&quot;<br>    ]<br>)<br>print(response.text)</pre><h3>Step #3: Build dynamic routing and mitigate model limitations</h3><p>When building multi-agent systems (like those built on Google’s <a href="https://docs.cloud.google.com/agent-builder/agent-development-kit/overview">Agent Development Kit</a>), give your agent the ability to dynamically route tasks. We recommend setting up two distinct tools:</p><ul><li><strong>General RAG (Text-based retrieval):</strong> Standard vector search over extracted textual data.</li><li><strong>Multimodal RAG (Image-based retrieval):</strong> Sending relevant raw pages/images to Gemini for visual analysis.</li></ul><p>Instruct your agent’s system prompt to use the image retrieval tool when the user’s query involves visual content (e.g., charts, signatures, or scanned forms). Think of this as an on-demand visual analysis tool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pchvjB5yreVl_MPeRgG4gQ.jpeg" /></figure><p><strong>Designing for limitations:</strong> It is crucial to build your workflows with the model’s limitations in mind. While multimodal models are incredibly powerful, they are not perfect spatial calculators.</p><ul><li><strong>Approximations:</strong> The models are not precise at locating exact coordinates of text/objects and might only return approximated counts of objects on a page.</li><li><strong>Handwriting:</strong> Be aware that models may hallucinate when interpreting messy handwritten text in PDFs (a common challenge for startups processing intake clinical notes or physical therapy records). Design your agent to flag highly ambiguous handwriting for “human-in-the-loop” (HITL) review.</li><li><strong>Confidence Thresholding</strong>: Program your agent to return a “certainty score” alongside its extraction. For mission-critical data (like financial totals or medical IDs), set a threshold (e.g., 90%). If the model’s confidence is lower, route the task to a fallback OCR engine or a human-in-the-loop for verification.</li></ul><h3>Step #4: Optimize economics for massive scale</h3><p>At startup scale — processing tens of millions of pages per month — passing every single image to a full vision language model (VLM) during high-volume ingestion is economically unsustainable.</p><p>Here is our blueprint for massive, economical scale:</p><ul><li><strong>Adopt Multimodal Embeddings:</strong> Instead of sending every ingested image to Gemini for reasoning, send them to a multimodal embedding model such as <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/gemini/embedding-2">Gemini Embedding 2</a>. You pay a fraction of a cent to make the image searchable. You only invoke the more expensive Gemini model <em>after</em> a search finds a relevant image.</li><li><strong>Use Gemini Flash-Lite for the heavy lifting:</strong> For initial extraction passes and OCR-ing simple tables, use Gemini 3.1 Flash-Lite. It has the cheapest per-token cost and is incredibly fast.</li><li><strong>Leverage Context Caching:</strong> If your users have standard reference documents (like massive legal codebases) that your agent queries repeatedly, turn on <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/context-cache/context-cache-overview#explicit-caching">explicit context caching</a>. You pay to process the heavy document images once, and every subsequent query over the next hour is discounted by up to 90%.</li><li><strong>Routing Requests:</strong> For global applications, route your document processing requests to the nearest Google Cloud region by using the global endpoint. This minimizes the “time to first token” (TTFT) for your users and allows you to utilize regions with higher availability. You can also leverage lower pricing for non-urgent batch processing of documents by using <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/batch-prediction-gemini">batch inference</a> or <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/flex-paygo">Flex PayGo</a>.</li></ul><h3>Ready to start building visually intelligent agents? Let’s get started!</h3><ul><li>Explore the <a href="https://cloud.google.com/resources/content/building-ai-agents">Startups technical guide: AI agents</a> to dive deeper into agent architectures.</li><li>Try out Gemini 3.1 Pro, 3 Flash and, 3.1 Flash-Lite in <a href="https://aistudio.google.com/">Google AI Studio</a> to test multimodal document capabilities.</li><li>Review the full <a href="https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/document-understanding">Gemini Document Understanding best practices</a>.</li><li>Gemini models can be configured to take advantage of GCP’s <a href="https://docs.cloud.google.com/vertex-ai/generative-ai/docs/vertex-ai-zero-data-retention">enterprise-grade security and compliance</a>, so your team can spend more time building secure solutions.</li><li>No matter where you are with AI adoption, we are here to help. Contact our Startup team today to learn how you can get up to $350,000 USD in cloud credits with the <a href="https://cloud.google.com/startup?e=48754805&amp;hl=en">Google for Startups Cloud Program</a>.</li></ul><p>Special thank you to my co-author, <a href="https://www.linkedin.com/in/nanditha-e-9b21a7a1/">Nanditha Embar</a>!</p><p><strong><em>Disclaimer:</em></strong></p><p>The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of Google Cloud. Recommendations noted here should be tested in a non-production environment before production deployment. Users are responsible for assessing the cost and security implications of their own deployments.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c25785722fb2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/four-steps-for-startups-to-master-visual-document-understanding-with-ai-c25785722fb2">Four Steps for Startups to Master Visual Document Understanding with AI</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Startup’s Guide to GPU Access on Google Cloud — Part 4: Provisioning GPUs with DWS Flex-Start…]]></title>
            <link>https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-4-provisioning-gpus-with-dws-flex-start-307c82f6cb59?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/307c82f6cb59</guid>
            <category><![CDATA[google]]></category>
            <category><![CDATA[slurm]]></category>
            <category><![CDATA[ai-infrastructure]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ankita Deshmukh]]></dc:creator>
            <pubDate>Wed, 18 Mar 2026 18:47:36 GMT</pubDate>
            <atom:updated>2026-03-18T18:47:34.874Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>The Startup’s Guide to GPU Access on Google Cloud — Part 4: Provisioning GPUs with DWS Flex-Start on Slurm</strong></h3><p>Dynamic Workload Scheduler (DWS) continues to be the strategic advantage for startups needing high-demand GPU resources with enhanced availability.</p><p>In<a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-1-why-dws-flex-start-is-the-poc-power-up-eb322e6f1ae8"> Part 1</a> of this series, we introduced the core principles and financial implications of DWS Flex, highlighting how it tackles the GPU Availability Paradox. In<a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-2-deploying-gce-instances-using-dws-de1594724540"> Part 2</a>, we provided a hands-on guide for utilizing DWS Flex-start with standalone Compute Engine (GCE) instances and Managed Instance Groups (MIGs). Building on that foundation,<a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-3-provisioning-gpus-with-dws-flex-start-0f3105c8f317"> Part 3</a> introduced the sophisticated, cloud-native approach of integrating DWS Flex-start with Google Kubernetes Engine (GKE) for containerized workloads.</p><p>In this fourth and final part, we shift our focus to High-Performance Computing (HPC) environments, detailing how to provision Flex-start GPUs using <strong>Slurm</strong> on Google Cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*PkrgcwG0H47HyQNxrr8Ddg.png" /><figcaption>Image Generated by Gemini</figcaption></figure><h4>Why Slurm and the Cluster Toolkit?</h4><p>For engineers, data scientists, and academics already familiar with traditional HPC setups, Slurm is the dominant and often preferred workload manager. It is essential for orchestrating large-scale, highly parallel training jobs.</p><p>While setting up a custom, best-practices-compliant Slurm cluster on Google Cloud using traditional infrastructure-as-code (IaC) can be time-consuming, the Google Cloud Cluster Toolkit offers a fast-track solution for POCs.</p><p>The Cluster Toolkit provides abstracted and extensible blueprints for deploying complex HPC environments, pre-configured with Google Cloud best practices for networking, storage, and image selection. For smaller startup teams with minimal infrastructure support, the Cluster Toolkit is a great way to set up a managed Slurm environment.</p><p>We will use this tool to quickly and efficiently deploy a DWS Flex-start-enabled Slurm environment. If you are a Slurm expert, you are welcome to adapt this guide to your own custom deployment blueprint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vykxj7Cvo84GIuW-1FsUSA.png" /><figcaption>Example of a Cluster Toolkit Blueprint</figcaption></figure><h3>Implementation Guide: DWS Flex on Slurm</h3><h3>Prerequisites &amp; Setup</h3><h4>1. APIs, Tools and Permissions</h4><p>Before proceeding, ensure you have the following APIs enabled and tools installed:</p><p><strong>APIs to Enable:</strong></p><ul><li>Compute Engine API</li><li>Cloud Identity and Access Management (IAM) API</li><li>Cloud Storage API</li><li>Service Usage API</li><li>Cloud Resource Manager API</li></ul><p><strong>CLI Tools:</strong></p><ul><li><strong>Google Cloud SDK (gcloud CLI):</strong> Ensure you have the latest version installed and configured.</li><li><strong>terraform: </strong>The Cluster Toolkit relies on Terraform for provisioning infrastructure.</li><li><strong>Cluster Toolkit: </strong>Verify if <em>gcluster</em> is installed by running a command <em>./gcluster — version</em>. If not found, proceed to the installation in the next step.</li></ul><p><strong>Permissions:</strong></p><p>Ensure the service account used by Terraform (often the Compute Engine default service account) has the necessary roles to create and manage Compute resources, IAM roles, and Storage Buckets in your project.</p><h4>2. Installing and Setting up the Cluster Toolkit</h4><p>To begin, you must install and set up the Google Cloud HPC Cluster Toolkit. This is mandatory for leveraging the pre-configured blueprints in this guide.</p><p><strong>Recommended Action:</strong> Please follow the official<a href="https://cloud.google.com/cluster-toolkit/docs/setup/configure-environment"> Cluster Toolkit Installation and Setup Guide</a> to clone the repository and complete the initial setup steps before proceeding.</p><h4>3. GPU Quota and Permissions Check</h4><p>It is crucial to confirm you have sufficient GPU quota in your chosen region/zone before starting the setup.</p><ul><li><strong>Quota:</strong> Please refer back to<a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-2-deploying-gce-instances-using-dws-de1594724540"> Part 2</a> of this blog series for details on checking and requesting the necessary DWS Flex start-compatible GPU quota.</li></ul><h3>Step-by-Step Implementation</h3><p>Now that your prerequisites are met, we will focus on configuring and deploying your DWS Flex-enabled Slurm cluster using the Cluster Toolkit. This setup is optimized for high-performance computing (HPC) with A3 Mega/H100 GPUs.</p><h4>1. Configure Core Cloud Resources</h4><p>First, we will set up the essential supporting resources for the Slurm cluster’s data and state management, primarily a Cloud Storage bucket. This bucket is used by <strong>Terraform</strong> to store the state file.</p><ul><li><strong>Create a Dedicated Cloud Storage Bucket: </strong>The bucket must be created with uniform bucket-level access (UBLA) and versioning enabled for state integrity.</li></ul><pre># Create a GCS bucket for your project. Choose a globally unique name.<br>gcloud storage buckets create gs://BUCKET_NAME \<br>    --project=PROJECT_ID \<br>    --default-storage-class=STANDARD --location=REGION \<br>    --uniform-bucket-level-access<br><br># Example: gs://test-bucket<br># NOTE: Replace BUCKET_NAME, PROJECT_ID, and REGION with your actual values.</pre><ul><li><strong>Enable Bucket Versioning and Grant Permissions: </strong>It is highly recommended to enable versioning for the state bucket.</li></ul><pre># Enable versioning on the bucket<br>gcloud storage buckets update gs://BUCKET_NAME --versioning<br><br># Grant the Storage Admin role to your user account (replace USERNAME and PROJECT_ID)<br>gcloud projects add-iam-policy-binding PROJECT_ID \<br>    --member=user:USERNAME \<br>    --role=roles/storage.admin</pre><h4>2. Update Slurm Blueprint Files</h4><p>Since we are using the Cluster Toolkit, we need to modify two core files in the repository clone to enable DWS Flex and define our deployment parameters: the <strong>Deployment YAML</strong> (which sets variables) and the <strong>Blueprint YAML</strong> (which defines the infrastructure structure).</p><blockquote><strong>IMPORTANT NOTE TO AVOID RESOURCE CONFLICTS:</strong> To ensure a clean deployment and avoid conflicts with old resources, it is crucial to update the unique variables in these files. This is essential for preventing errors, especially if you have prior failed deployments.</blockquote><blockquote><strong>IMPORTANT NOTE ON NETWORK OPTIMIZATION:</strong>This configuration is optimized for single-node workloads or multiple independent single-node jobs. For optimizing communication in multi-node, high-bandwidth workloads (e.g., distributed training) on A3 Mega VMs,<a href="https://cloud.google.com/cluster-toolkit/docs/machine-learning/a3-mega-enable-gpudirect-tcpxo"> <strong>GPUDirect-TCPXO</strong></a> is required to be enabled on your Slurm cluster.</blockquote><ul><li><strong>Update the Deployment YAML (a3mega-slurm-deployment.yaml): </strong>Edit the file located at <em>examples/machine-learning/a3-megagpu-8g/a3mega-slurm-deployment.yaml. </em>This file sets the Terraform backend, project variables, and key Slurm cluster parameters. The values below are generic placeholders.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2XRAlsPWQcVg1qfPjBY9FA.png" /></figure><p>Example of Updated Variables Section (within <em>a3mega-slurm-deployment.yaml</em>):</p><pre>terraform_backend_defaults:<br>  type: gcs<br>  configuration:<br>    bucket: BUCKET_NAME # Example: test-bucket<br><br>vars:<br>  deployment_name: DEPLOYMENT_NAME # Example: slurm-dws-flex-poc<br>  project_id: PROJECT_ID<br>  region: REGION<br>  zone: ZONE<br>  # ... network and other generic variables<br>  disk_size_gb: DISK_SIZE_GB # Example: 200<br>  final_image_family: FINAL_IMAGE_FAMILY # Example: slurm-dws-flex-image<br>  slurm_cluster_name: SLURM_CLUSTER_NAME # Example: slurmdws<br>  a3mega_cluster_size: MAX_SLURM_NODES # Example: 16<br>  a3mega_dws_flex_enabled: true # DWS FLEX ENABLED<br>  max_run_duration: MAX_RUN_DURATION # Example: 7200<br>  # ... other variables</pre><ul><li><strong>Update the Blueprint YAML (a3mega-slurm-blueprint.yaml): </strong>Edit the file located at <em>examples/machine-learning/a3-megagpu-8g/a3mega-slurm-blueprint.yaml</em>. This file requires specific modifications in the compute node and partition definitions to fully enable and configure DWS Flex. The values reference the variables defined in the Deployment YAML (using <em>$(vars.VARIABLE_NAME)</em> syntax).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9k0j30KK7WNTABi-i2GXw.png" /></figure><p>Example of Updated Blueprint Section (within <em>a3mega-slurm-blueprint.yaml</em>):</p><pre>- id: a3mega_nodeset<br>    # ... other settings<br>    settings:<br>      # Key Change 1: Set static nodes to 0 to enable DWS Flex<br>      node_count_static: 0 <br>      # Set max dynamic nodes based on the deployment variable<br>      node_count_dynamic_max: $(vars.a3mega_cluster_size) <br>      disk_type: pd-ssd<br>      machine_type: MACHINE_TYPE # Example: a3-megagpu-8g<br>      # ... other settings<br>      dws_flex:<br>        # Key Change 2: Enable DWS Flex on the nodeset<br>        enabled: true<br>        # Key Change 3: Set hard limit for the node VM lifetime<br>        max_run_duration: $(vars.max_run_duration) <br>      # ... other settings<br>      <br>- id: a3mega_partition<br>    # ... other settings<br>    settings:<br>      # ... partition name and other settings<br>      # Key Change 4: Set Slurm suspend time equal to max run duration<br>      suspend_time: $(vars.max_run_duration) <br>      partition_conf:<br>        OverSubscribe: EXCLUSIVE<br>        ResumeTimeout: 900<br>        SuspendTimeout: 600</pre><p><strong>3. Provision the Slurm Cluster</strong></p><p>With the deployment files correctly configured, use the <em>gcluster deploy</em> command from the Cluster Toolkit to provision the infrastructure using <strong>Terraform</strong>.</p><pre># Execute the deployment command from the Cluster Toolkit directory<br>./gcluster deploy -d examples/machine-learning/a3-megagpu-8g/a3mega-slurm-deployment.yaml \<br>examples/machine-learning/a3-megagpu-8g/a3mega-slurm-blueprint.yaml \<br>--auto-approve</pre><blockquote><strong>NOTE:</strong> Building the custom compute node image is the most time-consuming part of this process, typically taking <strong>30–40 minutes</strong>. This step creates the controller, login, and debug nodes, as well as the custom image that your dynamic compute nodes will use.</blockquote><h3>Cluster Interaction and Job Management</h3><p>Once the Cluster Toolkit deployment command completes, your Slurm control plane (controller, login, and debug nodes) is running, and the DWS Flex-start-enabled compute node pool is configured but empty (set to <em>node_count_static: 0</em>). The next step is to interact with the scheduler to trigger dynamic provisioning.</p><h4>1. Access the Slurm Cluster</h4><p>SSH into the <strong>login node</strong> to interact with the Slurm scheduler and submit jobs. The login node is the gateway to your compute resources.</p><pre># Example command to SSH into the Slurm login node<br>gcloud compute ssh SLURM_CLUSTER_NAME-login --zone=ZONE</pre><h4>2. Verify Node Configuration</h4><p><strong><em>sinfo</em></strong> is a Slurm utility used to view information about Slurm partitions and check the current status of the compute nodes.</p><pre>sinfo</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*XEqKrXXI6gJi5ABrWiAbWA.png" /></figure><h4>3. Submitting Jobs and Triggering Dynamic Scaling</h4><p>When DWS Flex-start is enabled, any job submission command (<em>sbatch</em> or <em>salloc</em>) that requests resources Slurm does not currently have will automatically trigger the creation of new VMs. The job will simply wait until the requested resources are provisioned and ready.</p><p><strong>Option A: Non-Interactive Job Submission (<em>sbatch</em>)</strong></p><p>This is the standard and recommended way to submit batch workloads.</p><p><strong><em>sbatch</em></strong> submits a script for later execution, allowing the job to run in the background while you exit the shell. If no nodes are available, <em>sbatch</em> will trigger dynamic scaling to provision them.</p><p><strong>Process:</strong></p><ol><li>Create a job submission script (<em>test_job.sh</em>).</li><li>Run <em>sbatch test_job.sh</em>. Slurm returns a <strong>Job ID</strong>.</li><li>Slurm checks for available nodes. Finding none, it sets the job state to <strong>Pending (PD)</strong> or <strong>Configuring (CF)</strong>.</li><li>The Slurm controller secures nodes throuh DWS Flex-start provisioning mode.</li><li>The job waits until the VMs are booted and registered with Slurm, then transitions to the <strong><em>Running (R)</em></strong> state.</li></ol><p><strong>Option B: Interactive Allocation (<em>salloc</em>)</strong></p><p>This is useful for debugging, testing, or interactively running code on a provisioned compute node.</p><p><strong><em>salloc</em></strong> requests a resource allocation for a job in real-time, typically providing an interactive shell on a compute node once allocated.</p><pre># Request 2 nodes (-N 2) from the a3mega partition (-p a3mega) <br># for a 24-hour time limit (-t 1-00:00:00) with exclusive access (--exclusive)<br>salloc -p a3mega -N 2 -t 1-00:00:00 --exclusive</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4_nxhbCnwR-6mbTyTgZF0g.png" /></figure><h4>4. Run a Job (<em>sbatch</em>)</h4><p>Here is an example of the job script and submission.</p><p><strong>Example Job Submission Script</strong></p><p>Create a script file (e.g., <em>test_job.sh</em>) with your job parameters and commands:</p><pre>#!/bin/bash<br>#SBATCH --job-name=quick-test-job<br>#SBATCH --nodes=1<br>#SBATCH --time=00:03:00<br><br>echo &quot;Starting job on allocated node...&quot;<br><br># Example workload command<br>sleep 120 <br><br>echo &quot;Job finished successfully.&quot;</pre><p><strong>Submit the Job</strong></p><pre>sbatch test_job.sh</pre><blockquote>Slurm will return a Job ID, e.g., Submitted batch job 123</blockquote><h4>5. Check Job and GPU Status</h4><p>Use the following commands from the login node to manage your job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_b8Oclc0QmdKYF6ENCBJjQ.png" /></figure><h4>6. Manage and Destroy the Cluster</h4><p><strong>Cancel a Running Job (<em>scancel</em>)</strong></p><p>To stop a running job and allow the Slurm scheduler to trigger dynamic descaling/termination of the associated nodes, use <em>scancel</em>.</p><pre>scancel JOB_ID</pre><p><strong>Redeploy the Cluster</strong></p><p>To update the cluster’s configuration <strong>without rebuilding the custom compute node image</strong> (which saves significant time), use the <em>— only</em> flag. This is useful for minor configuration changes.</p><pre>./gcluster deploy -d \<br>examples/machine-learning/a3-megagpu-8g/a3mega-slurm-deployment.yaml \<br>examples/machine-learning/a3-megagpu-8g/a3mega-slurm-blueprint.yaml \<br>--only primary,cluster --auto-approve -w</pre><p><strong>Destroy the Cluster</strong></p><p>To avoid incurring charges for idle resources, <strong>always destroy your cluster</strong> when you are finished. Replace <strong><em>DEPLOYMENT_FOLDER</em></strong> with your specific path (e.g., <em>examples/machine-learning/a3-megagpu-8g</em>).</p><h3>Troubleshooting and Important Considerations</h3><p>Understanding compute node behavior is key to successful experimentation with Slurm with Flex-start nodes.</p><ul><li><strong>Node Down Error: Understanding VM Termination<br></strong>Depending on your blueprint configuration, you might observe compute nodes going down after a period of time. This can be attributed to two main settings:</li><li><strong><em>max_run_duration</em> Exceeded:</strong> This is a hard limit set in the blueprint file for DWS Flex-start provisioned GPUs. Once this duration is exceeded, the underlying VM will be returned to the resource pool irrespective of its current job state. If your longer job is terminated due to max_run_duration, Slurm’s scheduler will detect the incomplete job and automatically restart it on a newly provisioned VM. The job will maintain the same job ID and name and will be scheduled to run (from the beginning) on a new instance that is given the same node name. This is the intended behavior, ensuring that jobs interrupted by node de-provisioning are resumed to completion.</li><li><strong><em>suspend_time</em>:</strong> If a compute node remains idle for more than the configured SuspendTime (default 300 seconds), it will be placed into a power-save mode by Slurm’s <em>SuspendProgram</em>. While this is by design for cost optimization, during experimentation, you might prefer to keep nodes active. To prevent nodes from being suspended due to idle time before the DWS Flex-start <em>max_run_duration</em> limit, you can set <em>suspend_time</em> to -1 (never suspend) or equal to <em>max_run_duration</em> in your blueprint’s partition configuration.</li></ul><h3>Conclusion</h3><p>This series has proven that <strong>DWS Flex-start</strong> is the definitive “power-up” for AI startups on Google Cloud. By bridging the gap between cost-efficiency and high availability, it enables reliable, queue-managed access to the world’s most sought-after accelerators.</p><p>From simple standalone instances to complex Slurm and GKE clusters, DWS Flex-start ensures your infrastructure works as hard as your code. As you conclude this guide, you are now equipped to bypass the availability paradox and accelerate your journey from experimentation to global production.</p><h3>Additional Readings</h3><p>To get started with DWS Flex-start right away, refer to these official Google Cloud resources:</p><ul><li><a href="https://cloud.google.com/compute/docs/instances/about-flex-start-vms">About Flex-start VMs</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-vm#flex-start_1">DWS Flex-start on GCE</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-mig-with-a4-or-a3ultra">DWS Flex-start on Managed Instance Groups</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/gke-ai-hypercompute-custom#deploy-run-nccl-test">DWS Flex-start on GKE</a></li><li><a href="https://cloud.google.com/cluster-toolkit/docs/deploy/deploy-a3-mega-cluster">DWS Flex-start on Slurm</a></li><li><a href="https://docs.cloud.google.com/vertex-ai/docs/training/schedule-jobs-dws">DWS Flex-start on Vertex AI</a></li></ul><h3>Disclaimer</h3><p>The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of Google Cloud. The code snippets and CLI commands provided are for educational purposes and should be tested in a non-production environment before deployment. Users are responsible for assessing the cost and security implications of their own deployments.</p><h3>Partner with Google Cloud for Success</h3><p>Your success is our priority. We’re here to help you navigate the complexities of GPU availability, answer your questions, and find the best solution for your specific needs. Don’t hesitate to reach out to your <a href="https://cloud.google.com/startup">Google Cloud Startup Success</a> team for expert guidance and support.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=307c82f6cb59" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-4-provisioning-gpus-with-dws-flex-start-307c82f6cb59">The Startup’s Guide to GPU Access on Google Cloud — Part 4: Provisioning GPUs with DWS Flex-Start…</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Startup’s Guide to GPU Access on Google Cloud — Part 3: Provisioning GPUs with DWS Flex-Start…]]></title>
            <link>https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-3-provisioning-gpus-with-dws-flex-start-0f3105c8f317?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/0f3105c8f317</guid>
            <category><![CDATA[ai-infrastructure]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[google-kubernetes-engine]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[Ankita Deshmukh]]></dc:creator>
            <pubDate>Mon, 02 Mar 2026 21:17:53 GMT</pubDate>
            <atom:updated>2026-03-02T21:17:51.954Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>The Startup’s Guide to GPU Access on Google Cloud — Part 3: Provisioning GPUs with DWS Flex-Start on GKE</strong></h3><p>Welcome back to our series focused on helping startups efficiently spin up and utilize powerful GPUs using Dynamic Workload Scaling (DWS) Flex-Start. In<a href="https://medium.com/p/eb322e6f1ae8"> Part 1</a>, we introduced the general guidelines and core concepts of DWS Flex. In<a href="https://medium.com/p/de1594724540"> Part 2</a>, we covered the setup process using standalone Compute Engine (GCE) instances and Managed Instance Groups (MIGs).</p><p>Now, in this third part, we introduce one of the most sophisticated and native approaches on Google Cloud: integrating DWS Flex with Google Kubernetes Engine (GKE).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*v0IpmL4QijOHsNtWdCpJPA.png" /><figcaption>Image Generated by Gemini</figcaption></figure><h3>Why GKE?</h3><p>While standalone GCE instances are perfect for simple, one-off experiments, GKE is the superior choice for production or complex, multi-stage workflows. GKE abstracts away underlying infrastructure, offering features like automatic scaling, self-healing, and declarative resource management through Kubernetes manifests. When combined with DWS Flex, GKE enables startups to manage their containerized GPU workloads with robust scalability and high availability, effortlessly scaling from zero nodes to handle bursts of demand and back down again to save cost.</p><h3>Implementation Guide: DWS Flex on GKE</h3><p>We will now walk through the essential prerequisites and the step-by-step process to provision DWS Flex GPUs within a <strong>GKE Standard cluster</strong>.</p><blockquote><strong>Note on Hardware Selection:</strong> For this guide, we are focusing on the high-performance A3 Mega machine series, specifically the a3-megagpu-8g type, which features 8 x NVIDIA H100 80GB GPUs per node. If you are targeting a different accelerator, please ensure you adjust the MACHINE_TYPE, ACCELERATOR_TYPE, and GPU_COUNT environment variables below to match your chosen hardware and quota.</blockquote><p>To give you a better picture of what we’re building, here is the complete step-by-step lifecycle for getting these GPU workloads off the ground.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WN-bk6SA4nAA9I8nvFjyKw.png" /><figcaption>The implementational flowchart for deploying a GKE cluster with flex-start nodes (Image generated by Gemini)</figcaption></figure><h3>Prerequisites &amp; Setup</h3><h4>1. API and Tool Requirements</h4><p>You must ensure the following are enabled and installed:</p><p><strong>APIs to Enable:</strong></p><ul><li><strong>Kubernetes Engine API</strong> (Essential for GKE cluster management).</li><li><strong>Compute Engine API</strong> (Required for underlying VMs and GPUs).</li></ul><p><strong>CLI Tools:</strong></p><ul><li><strong>Google Cloud SDK (gcloud CLI):</strong> Ensure you have the latest version configured.</li><li><strong>kubectl:</strong> Ensure kubectl is installed to interact with your GKE cluster.</li></ul><p><strong>GKE Version:</strong></p><ul><li><strong>Standard Cluster:</strong> Version 1.28.3-gke.1098000 or later.</li><li><strong>Autopilot Cluster:</strong> Version 1.30.3-gke.1451000 or later.</li></ul><p><em>(For more details, read the GKE Release Notes.)</em></p><h4>2. GPU Quota Check</h4><p>It is critical to confirm you have sufficient GPU quota in your chosen region/zone before starting the setup.</p><p><strong>Note:</strong> For a detailed guide on checking and requesting quota, please refer back to<a href="https://medium.com/p/de1594724540"> Part 2</a> of this blog series.</p><h3><strong>Step 1: Set Environment Variables</strong></h3><p>Define the variables that will be used for your cluster, node pool, and workload configuration according to your requirements.</p><pre># UPDATE THESE VARIABLES!<br># Project and Location variables<br>export PROJECT_ID=&quot;$(gcloud config get-value project)&quot;<br>export CLUSTER_NAME=&quot;&lt;CLUSTER_NAME&gt;&quot;<br>export REGION=&quot;&lt;REGION&gt;&quot; <br>export COMPUTE_ZONE=&quot;&lt;ZONE&gt;&quot;<br># Hardware and Scaling variables<br>export NODEPOOL_NAME=&quot;&lt;NODEPOOL_NAME&gt;&quot;<br>export MACHINE_TYPE=&quot;&lt;MACHINE_TYPE&gt;&quot; #example: a3-megagpu-8g<br>export ACCELERATOR_TYPE=&quot;&lt;ACCELERATOR_TYPE&gt;&quot; #example: nvidia-h100-80gb<br>export GPU_COUNT=&quot;&lt;GPU_COUNT&gt;&quot; # example: 8 (8 GPUs per node)<br>export MAX_NODES=&quot;&lt;MAX_NODE&gt;&quot; # example: 2<br>export MAX_RUN_DURATION=&quot;&lt;MAX_DURATION&gt;&quot; # Max time for DWS Flex provisioning (e.g., min 10m, max 168h)</pre><h3>Step 2: Create the GKE Standard Cluster</h3><p>Create the GKE cluster:</p><pre>gcloud container clusters create &quot;${CLUSTER_NAME}&quot; \<br>    --zone &quot;${COMPUTE_ZONE}&quot; \<br>    --project=&quot;${PROJECT_ID}&quot; \<br>    --release-channel=REGULAR</pre><h3>Step 3: Get Cluster Credentials</h3><p>Configure kubectl to connect to your newly created cluster:</p><pre>gcloud container clusters get-credentials &quot;${CLUSTER_NAME}&quot; \<br>    --zone=&quot;${COMPUTE_ZONE}&quot; \<br>    --project=&quot;${PROJECT_ID}&quot;</pre><h3><strong>Step 4: Create the DWS Flex Node Pool</strong></h3><p>The node pool creation is where we enable the core DWS Flex functionality. By setting — <em>num-nodes=0</em> and enabling autoscaling, the node pool can scale from zero to the maximum configured node count based on demand.</p><p>The key flag here is — <em>flex-start</em>. This flag enables DWS Flex for the entire node pool, allowing nodes to be provisioned with enhanced availability guarantees compared to standard Preemptible VMs.</p><p>Mainly, there are two ways of leveraging DWS Flex-Start in your jobs, addressing different requirements for workload scheduling and resource guarantee:</p><h4><strong>Choosing Your DWS Flex Strategy</strong></h4><p>When configuring your GPU node pool, it is crucial to select the provisioning strategy that best matches your workload’s needs: <strong>Best-Effort</strong> or <strong>All-or-Nothing</strong>.</p><ul><li><strong>Best-Effort Provisioning (Option A)</strong> prioritizes the speed of start-up for single-node or smaller jobs by utilizing available capacity quickly.</li><li><strong>Queued Provisioning (Option B)</strong> ensures that large, interdependent workloads receive all required resources simultaneously, preventing resource starvation and guaranteeing a synchronized start for distributed training.</li></ul><p>The table below summarizes the technical and use-case differences between the two configuration options.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zTP8XJajP68gDCTHddu1Ug.png" /></figure><h4>Implementation Details for Node Pool Creation</h4><p>Once you have determined the correct strategy, you can create the corresponding node pool using the gcloud command, enabling the necessary flags as detailed below.</p><p><strong>Option A: DWS Flex Start (No Queued Provisioning)</strong></p><p>This node pool configuration utilizes DWS Flex for best-effort availability, relying solely on the — <em>flex-start</em> flag.</p><pre>gcloud container node-pools create &quot;${NODEPOOL_NAME}&quot; \<br>    --cluster=&quot;${CLUSTER_NAME}&quot; \<br>    --zone=&quot;${COMPUTE_ZONE}&quot; \<br>    --accelerator type=${ACCELERATOR_TYPE},count=${GPU_COUNT},gpu-driver-version=latest \<br>    --machine-type=&quot;${MACHINE_TYPE}&quot; \<br>    --flex-start \<br>    --max-run-duration=${MAX_RUN_DURATION} \<br>    --enable-autoscaling \<br>    --num-nodes=0 \<br>    --total-max-nodes &quot;${MAX_NODES}&quot; \<br>    --reservation-affinity=none \<br>    --no-enable-autorepair \<br>    --enable-gvnic \<br>    --project=&quot;${PROJECT_ID}&quot;</pre><p><strong>Option B: DWS Flex-Start with Queued Provisioning</strong></p><p>This configuration enables the “All-or-Nothing” resource guarantee by adding the critical —<em> enable-queued-provisioning</em> flag alongside — <em>flex-start</em>.</p><h3>Step 5: Verify DWS Flex Start Configuration (Optional)</h3><p>You can confirm that <em>flexStart</em> is enabled on your node pool:</p><pre>gcloud container node-pools describe &quot;${NODEPOOL_NAME}&quot; \<br>    --cluster=&quot;${CLUSTER_NAME}&quot; \<br>    --zone=&quot;${COMPUTE_ZONE}&quot; \<br>    --format=&quot;get(config.flexStart)&quot;</pre><blockquote><em>Expected output:</em> True</blockquote><p>You can also check the flex-start enablement from UI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*1xulkihqT6UFpsXI8-wo7A.gif" /><figcaption>Verify flex-start enablement in Google Cloud Console</figcaption></figure><h3><strong>Step 6: Deploying Workloads</strong></h3><p>The final step is submitting your GPU workload. Regardless of the DWS Flex option you chose, you must ensure your Kubernetes manifest correctly targets the specialized node pool using <strong>Taints, Labels, and Annotations</strong>.</p><h4>Deploying a Single-Node Workload (Targeting Option A)</h4><p>This sample single Pod deployment will trigger the GKE cluster autoscaler to provision a DWS Flex node from zero, as needed.</p><p>The YAML uses specific Kubernetes fields to ensure the Pod is scheduled on the correct node pool:</p><p><strong>nodeSelector</strong>: This is used to target nodes with the specific attributes required by the workload.</p><ul><li><em>cloud.google.com/gke-flex-start: “true”</em>: Crucially, this directs the Pod to the node pool where DWS Flex has been enabled.</li><li><em>cloud.google.com/gke-accelerator: nvidia-h100–80gb</em>: This ensures the Pod only lands on nodes with the specified GPU type.</li></ul><p><strong>resources.limits &amp; requests</strong>: The values of <em>“8”</em> for <em>nvidia.com/gpu</em> indicate that the Pod requires all 8 GPUs available on a single node (matching the <em>a3-megagpu-8g</em> machine type used in the node pool).</p><p><strong>tolerations</strong>: This allows the Pod to be scheduled onto GPU nodes, which are typically tainted to prevent general workloads from consuming expensive resources.</p><p><strong>Create a file named single-node-deploy.yaml:</strong></p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: h100-pod-a<br>spec:<br>  nodeSelector:<br>    cloud.google.com/gke-flex-start: &quot;true&quot; # Targets DWS Flex enabled nodes (Option A)<br>    cloud.google.com/gke-accelerator: nvidia-h100-80gb # Ensures it goes to H100 GPU nodes<br>  containers:<br>  - name: cuda-container<br>    image: nvidia/cuda:12.9.1-base-ubuntu22.04 # A CUDA-enabled image<br>    command: [&quot;sleep&quot;, &quot;3600&quot;] # Keeps the container running for 1 hour<br>    resources:<br>      limits:<br>        nvidia.com/gpu: &quot;8&quot; # Requests 8 GPU, matching the node configuration<br>      requests:<br>        nvidia.com/gpu: &quot;8&quot;<br>  tolerations: # Allows scheduling on tainted GPU nodes<br>  - key: nvidia.com/gpu<br>    operator: Exists<br>    effect: NoSchedule</pre><p><strong>Apply the Pod:</strong></p><pre>kubectl apply -f single-node-deploy.yaml</pre><blockquote><strong>Note:</strong> Once applied, the GKE cluster autoscaler will detect the pending Pod and provision a new DWS Flex-enabled node to satisfy the request.</blockquote><h4><strong>Deploying a Multi-Node Workload (Targeting Option B)</strong></h4><p>To successfully target the <strong>Queued Provisioning</strong> node pool, the following fields are essential for the Job manifest:</p><ul><li><strong>Label (kueue.x-k8s.io/queue-name)</strong>: This label must be set to <em>dws-local-queue</em>. This is a mandatory requirement that signals the workload should be handled by the DWS provisioning queue.</li><li><strong>Annotation (provreq.kueue.x-k8s.io/maxRunDurationSeconds)</strong>: This annotation specifies the maximum amount of time the workload is allowed to run once it is scheduled. It is a key parameter for DWS Flex Queued Provisioning. We will set this to <strong>86400 seconds (24 hours)</strong>, though you can adjust this as needed.</li><li><strong>parallelism and completions</strong>: Setting these to 2 indicates a request for two separate Pods, each requiring 8 GPUs, totaling a requirement of 16 GPUs across two nodes. This effectively triggers the multi-node, “All-or-Nothing” requirement.</li><li><strong>Required Toleration (cloud.google.com/gke-queued)</strong>: This toleration allows the pod to be scheduled onto nodes that are tainted with the <em>cloud.google.com/gke-queued:NoSchedule</em> key/effect, which is automatically applied to nodes in the Queued Provisioning node pool.</li></ul><p><strong>Create a file named multi-node-deploy-qp.yaml:</strong></p><pre>apiVersion: batch/v1<br>kind: Job<br>metadata:<br>  name: queued-provisioning-job<br>  namespace: default<br>  labels:<br>    # REQUIRED: Targets the DWS provisioning queue<br>    kueue.x-k8s.io/queue-name: dws-local-queue <br>  annotations:<br>    # REQUIRED: Maximum duration the job can run after starting<br>    provreq.kueue.x-k8s.io/maxRunDurationSeconds: &quot;86400s&quot; # 24 hours<br>spec:<br>  parallelism: 2 # Requests 2 Pods (2 nodes in this case)<br>  completions: 2<br>  suspend: false<br>  template:<br>    spec:<br>      nodeSelector:<br>        cloud.google.com/gke-accelerator: nvidia-h100-80gb <br>      tolerations:<br>      - key: &quot;nvidia.com/gpu&quot;<br>        operator: &quot;Exists&quot;<br>        effect: &quot;NoSchedule&quot;<br>      - key: &quot;cloud.google.com/gke-queued&quot; # Targets queued provisioned nodepool (Option B)<br>        operator: &quot;Equal&quot;<br>        value: &quot;true&quot;<br>        effect: &quot;NoSchedule&quot;<br>      containers:<br>      - name: dummy-job<br>        image: gcr.io/k8s-staging-perf-tests/sleep:v0.0.3<br>        args: [&quot;3600s&quot;] <br>        resources:<br>          requests:<br>            nvidia.com/gpu: 8<br>          limits:<br>            nvidia.com/gpu: 8<br>      restartPolicy: Never</pre><p><strong>Apply the Job:</strong></p><pre>kubectl apply -f multi-node-deploy-qp.yaml</pre><blockquote><strong>Result:</strong> When this Job is applied, GKE will first check the availability of all 16 required GPUs (across 2 nodes). If they are not immediately available, the Job will remain in a pending/suspended state until the DWS Flex provisioning mechanism successfully acquires all necessary resources, ensuring your multi-node job starts completely and simultaneously.</blockquote><h3>Best Practices and Troubleshooting for DWS Flex-Start</h3><h4>Preventing Autoscaler Scale-Down for Flex-Start Nodes</h4><p>By default, the GKE Cluster Autoscaler is designed for cost efficiency and will automatically remove (scale down) nodes that are not running any Pods. For DWS Flex nodes, this typically occurs within 1 to 10 minutes of inactivity, depending on your autoscaler configuration.</p><p>If your workflow requires Flex-Start nodes to remain available for a longer period (up to their maximum run duration) and you accept the associated cost, you can prevent this automatic scale-down using two standard Kubernetes methods:</p><ul><li><strong>Node Annotation (Granular Control): </strong>You can add a specific annotation directly to the node to tell the Cluster Autoscaler to ignore it:</li></ul><p><em>“cluster-autoscaler.kubernetes.io/scale-down-disabled”: “true”</em></p><ul><li><strong>DaemonSet with Pod Annotation (Easier Implementation): </strong>A simpler method for cluster-wide control is to run a small DaemonSet on the nodes and apply the following annotation to the Pod specification:</li></ul><p><em>“cluster-autoscaler.kubernetes.io/safe-to-evict”: “false”</em></p><p>This signals that a critical Pod is running on the node, preventing the autoscaler from scaling down the underlying node until the Pod is removed.</p><h4>Note on Compact Placement</h4><p>When requesting large GPU resources, users often require compact placement, meaning the nodes are physically located close to one another to minimize network latency (e.g., for multi-node distributed workload).</p><p>It is important to understand that DWS Flex offers best-effort compact placement but does not provide a hard guarantee.</p><h3>What’s Next?</h3><p>We have successfully demonstrated how to integrate DWS Flex with Google Kubernetes Engine, moving your AI workloads from simple VM provisioning to a robust, scalable, and production-ready orchestration environment. By leveraging GKE’s cluster autoscaler with DWS Flex-Start provisioning mode, you can achieve powerful scale-to-zero GPU efficiency while maintaining excellent availability for high-demand resources.</p><p>In the next of this series, we will continue exploring sophisticated orchestration methods by diving into the hands-on, step-by-step guide for making use of DWS Flex on Slurm, a powerful workload manager often used in high-performance computing (HPC) environments.</p><h3>Additional Readings</h3><p>To get started with DWS Flex right away, refer to these official Google Cloud resources:</p><ul><li><a href="https://cloud.google.com/compute/docs/instances/about-flex-start-vms">About Flex-start VMs</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-vm#flex-start_1">DWS Flex-start on GCE</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-mig-with-a4-or-a3ultra">DWS Flex-start on Managed Instance Groups</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/gke-ai-hypercompute-custom#deploy-run-nccl-test">DWS Flex-start on GKE</a></li><li><a href="https://cloud.google.com/cluster-toolkit/docs/deploy/deploy-a3-mega-cluster">DWS Flex-start on Slurm</a></li><li><a href="https://docs.cloud.google.com/vertex-ai/docs/training/schedule-jobs-dws">DWS Flex-start on Vertex AI</a></li></ul><h3>Disclaimer</h3><p>The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of Google Cloud. The code snippets and CLI commands provided are for educational purposes and should be tested in a non-production environment before deployment. Users are responsible for assessing the cost and security implications of their own deployments.</p><h3>Partner with Google Cloud for Success</h3><p>Your success is our priority. We’re here to help you navigate the complexities of GPU availability, answer your questions, and find the best solution for your specific needs. Don’t hesitate to reach out to your <a href="https://cloud.google.com/startup">Google Cloud Startup Success</a> team for expert guidance and support.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f3105c8f317" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-3-provisioning-gpus-with-dws-flex-start-0f3105c8f317">The Startup’s Guide to GPU Access on Google Cloud — Part 3: Provisioning GPUs with DWS Flex-Start…</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You’re Probably Using the Wrong Medical AI Model; Here’s How to Fix It]]></title>
            <link>https://medium.com/google-cloud-for-startups/youre-probably-using-the-wrong-medical-ai-model-here-s-how-to-fix-it-314418c78007?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/314418c78007</guid>
            <category><![CDATA[medgemma]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[google]]></category>
            <dc:creator><![CDATA[Ada Amanna]]></dc:creator>
            <pubDate>Fri, 30 Jan 2026 22:32:44 GMT</pubDate>
            <atom:updated>2026-01-30T22:32:25.579Z</atom:updated>
            <content:encoded><![CDATA[<p>You’re building a medical imaging application and staring at two powerful AI models from Google: MedGemma and MedSigLip. Both handle medical data… Both promise to revolutionize healthcare AI… So… which one do you choose?</p><p>Make the wrong call, and you could waste weeks of development time building with the wrong tool. Make the right one, and you’ll create something truly transformative for patient care. This guide provides a comprehensive comparison of MedGemma and MedSigLip, offering a clear roadmap on when to leverage each model to its full potential.</p><h3>TL;DR: Quick Decision Guide</h3><p><strong>Use MedSigLip for:</strong> Image classification, semantic search, feature extraction, and zero-shot analysis</p><p><strong>Use MedGemma for:</strong> Report generation, conversational AI, clinical decision support, and multimodal text generation</p><p><strong>Still Can’t decide?</strong> Keep reading…</p><h3>MedSigLip: Advancing Medical Image Interpretation</h3><p>MedSigLip is a specialized vision-language model derived from the Sigmoid Loss for Language Image Pre-training (SigLIP) architecture. It has been meticulously trained on a vast and diverse dataset of de-identified medical imagery, including chest X-rays, dermatology images, and radiological scans, paired with their corresponding textual descriptions. This training enables MedSigLip to create a shared embedding space where medical images and related text are semantically aligned. Think of it as teaching the model to “understand” images and text in the same conceptual language. <strong>The primary function of MedSigLip is to excel at tasks that require a deep understanding of visual medical data without the need for generating descriptive text.</strong> It’s an analyzer and classifier first.</p><h3>Key Capabilities of MedSigLip</h3><p><strong>Data-Efficient Medical Image Classification: </strong>MedSigLip can serve as a powerful feature extractor. By training a simple classifier on top of its embeddings, developers can achieve state-of-the-art results with significantly less labeled data compared to training a model from scratch. This is a game-changer for rare diseases or specialized imaging where labeled data is scarce.</p><p><strong>Zero-Shot Classification: </strong>leveraging its rich embedding space, the model can classify medical images using textual prompts alone, even for categories it has not been explicitly trained on. This is invaluable for rapid prototyping and applications involving rare conditions where training data simply doesn’t exist at scale.</p><p><strong>Semantic Image Retrieval: </strong>MedSigLip’s text encoder allows for highly effective content-based image retrieval. Clinicians and researchers can use natural language queries to search vast databases of medical images and find visually and clinically relevant results. Imagine typing “chest X-ray with pneumothorax” and getting precisely what you need from thousands of scans.</p><p><strong>Fine-Tuning for Specialization: </strong>While powerful out-of-the-box, MedSigLip can be fine-tuned on specific datasets to further enhance its performance on niche tasks or adapt it to new imaging modalities. Whether you’re working with ultrasounds, MRIs, or specialized pathology slides, you can customize it for your specific domain.</p><p><strong>Pro Tip:</strong> Start with MedSigLip’s zero-shot capabilities for rapid prototyping. Only invest in fine-tuning once you’ve validated your use case and gathered sufficient domain-specific data.</p><figure><img alt="Medsiglip Architecture Diagram" src="https://cdn-images-1.medium.com/max/512/1*r8vq_Zcgs3SdFWbVnsJA9Q.png" /><figcaption><strong>Figure 1: The MedSigLip Workflow.</strong> Images are processed into embeddings and stored in a Vector DB, enabling “search by image” capabilities.</figcaption></figure><h3>MedGemma: Multimodal Comprehension and Generation</h3><p>Now let’s shift gears to its counterpart. MedGemma represents a significant leap forward in multimodal AI for medicine. Built upon the robust Gemma 3 architecture, MedGemma is a family of models designed for both the comprehension and generation of medical information across text and images. It is available in several configurations, including a 4B parameter multimodal model and larger 27B parameter text-only and multimodal versions, allowing developers to choose the optimal balance of performance and computational efficiency. <strong>MedGemma is engineered for complex, interactive applications that require a sophisticated understanding of context and the ability to generate coherent, clinically relevant text.</strong> This is your go-to when you need an AI that can think, reason, and communicate.</p><h3>Key Capabilities of MedGemma</h3><p><strong>Multimodal Medical Dialogue: </strong>MedGemma can engage in natural language conversations about medical images. This allows for applications where a user can ask detailed questions about a scan or image and receive insightful, text-based answers. Think of it as having a knowledgeable colleague who can discuss what they’re seeing in an image.</p><p><strong>Automated Report Generation: </strong>One of its standout use cases is the generation of preliminary medical image reports. By analyzing an image, MedGemma can produce a structured, descriptive summary, helping to streamline radiologist workflows. This doesn’t replace the radiologist, it gives them a head start with a draft they can review and refine.</p><p><strong>Advanced Clinical Reasoning: </strong>The larger MedGemma models are adept at complex clinical reasoning. They can be utilized in applications for patient triaging, differential diagnosis support, and summarizing extensive electronic health records. This is where the model’s deep understanding of medical context truly shines.</p><p><strong>Medical Text Understanding: </strong>Beyond its multimodal capabilities, MedGemma excels at a wide array of text-based tasks, including clinical knowledge extraction, patient interview simulations, and decision support systems. It’s not just about images, it’s about understanding the complete clinical picture.</p><figure><img alt="MedGemma Architecture Diagram" src="https://cdn-images-1.medium.com/max/512/1*yY1SLhdoj0oQj5puAS5Lpg.png" /><figcaption><strong>Figure 2: The MedGemma Workflow.</strong> A user interacts directly with the model, providing an image and a question to receive a generated clinical summary.</figcaption></figure><h3>MedGemma vs. MedSigLip: At a Glance</h3><p>To clarify the distinction between these two models, here’s a side-by-side comparison of their core attributes and ideal applications:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iJ1rA6EdQK4GjzcSY2arkg.png" /><figcaption><strong>MedGemma vs. MedSigLip At a Glance</strong></figcaption></figure><h3>When to Use Which: Making the Right Choice</h3><p>The decision to use MedGemma or MedSigLip depends entirely on the specific requirements of your application. Here’s your decision framework:</p><h3>Choose MedSigLip when:</h3><p>Your primary goal is to <strong>classify, search, or analyze medical images based on their content</strong>. If your application does not require the generation of new text and instead relies on understanding the semantic content of images, MedSigLip provides a highly efficient and powerful solution.</p><p><strong>Real-world example:</strong> You’re building a hospital’s radiology archive system that allows doctors to search for “similar cases” by uploading an image or describing what they’re looking for. MedSigLip powers the semantic search engine that finds visually and clinically similar scans from thousands of historical cases.</p><p>It is the ideal choice for building systems for:</p><ul><li>Image retrieval and semantic search</li><li>Data-efficient classification with limited labeled data</li><li>Visual similarity tasks</li><li>Feature extraction for downstream ML tasks</li></ul><h3>Choose MedGemma when:</h3><p>Your application requires <strong>interactive, generative, or multimodal capabilities</strong>. If you need to generate medical reports, answer questions about images, summarize clinical notes, or build conversational AI tools for healthcare, MedGemma is the superior choice.</p><p><strong>Real-world example:</strong> You’re developing an AI assistant that helps emergency room doctors by analyzing incoming X-rays and generating preliminary assessment reports. The system can also answer follow-up questions like “What are the differential diagnoses for this opacity?” or “Should we order additional imaging?” MedGemma handles both the report generation and interactive dialogue.</p><p>Its ability to process and generate both text and image data makes it the more versatile and powerful option for complex, end-to-end clinical workflows.</p><h3>What About Using Both?</h3><p>Here’s something developers often overlook: these models aren’t mutually exclusive. In a sophisticated healthcare AI system, you might use both:</p><ul><li><strong>MedSigLip</strong> for the initial image classification and retrieval layer</li><li><strong>MedGemma</strong> for generating detailed reports and providing interactive explanations</li></ul><p>For example, MedSigLip could quickly categorize and retrieve relevant prior cases, then MedGemma could analyze those cases and generate a comparative summary for the clinician.</p><h3>Conclusion</h3><p>Both MedGemma and MedSigLip are at the forefront of AI in medicine, offering developers unprecedented tools to tackle some of healthcare’s most pressing challenges. By understanding their distinct architectures and intended purposes, you can make informed decisions, select the right model for the job, and accelerate the development of innovative AI applications that can improve patient outcomes and streamline clinical practice.</p><p><strong>Still unsure which model to choose?</strong> Start with MedSigLip for its simplicity and efficiency. Once your use case evolves to require text generation or more complex multimodal interactions, you can graduate to MedGemma or implement both in complementary roles. The future of medical AI isn’t about choosing one model over another, it’s about understanding when and how to leverage each tool’s unique strengths to build better healthcare solutions.</p><p><strong>Ready to start building?</strong> Check out Google Cloud’s <a href="https://developers.google.com/health-ai-developer-foundations"><strong>Health AI Developer Foundations</strong></a> resources and documentation to get hands-on with both models.</p><p><em>Tags: #AI #Healthcare #MachineLearning #GoogleCloud #MedicalAI #HealthTech #ArtificialIntelligence</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=314418c78007" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/youre-probably-using-the-wrong-medical-ai-model-here-s-how-to-fix-it-314418c78007">You’re Probably Using the Wrong Medical AI Model; Here’s How to Fix It</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Startup’s Guide to GPU Access on Google Cloud — Part 2: Deploying GCE Instances using DWS…]]></title>
            <link>https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-2-deploying-gce-instances-using-dws-de1594724540?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/de1594724540</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[startupş]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[ai-infrastructure]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Ankita Deshmukh]]></dc:creator>
            <pubDate>Thu, 04 Dec 2025 21:44:36 GMT</pubDate>
            <atom:updated>2025-12-04T21:44:34.783Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>The Startup’s Guide to GPU Access on Google Cloud — Part 2: Deploying GCE Instances using DWS Flex-Start</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wtu86SFO6L28fd5N5aQvqA.png" /><figcaption>Image Generated by Gemini</figcaption></figure><p>Dynamic Workload Scheduler (DWS) is a resource management and job scheduling platform designed for AI Hypercomputer on Google Cloud. The DWS Flex-start provisioning mode offers a reliable path for startups to procure powerful GPU accelerators without the high cost or long-term commitment of traditional methods. In <a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-1-why-dws-flex-start-is-the-poc-power-up-eb322e6f1ae8">Part 1</a> of this guide, we established why DWS Flex is the POC Power-Up on Google Cloud. Now that you understand the “why,” it’s time to put your hands on the keyboard. This Part 2 is a complete, hands-on tutorial. We will walk through the prerequisites and provide the exact Google Cloud CLI commands to provision GPUs through DWS Flex-start mode. While provisioning is also available through the Cloud Console UI, we focus on the CLI as it is highly useful for integrating DWS Flex into automated workflows and deployment pipelines. We will demonstrate deployment for both a standalone Compute Engine (GCE) instance and a scalable Managed Instance Group (MIG).</p><h3>Prerequisites &amp; Setup</h3><p>Before you can start provisioning, ensure your environment is correctly configured.</p><h4><strong>IAM &amp; Permissions</strong></h4><p>To create the necessary Compute Engine resources (VMs, Instance Templates, MIGs), you must have the <strong>Compute Instance Admin (v1)</strong> (roles/compute.instanceAdmin.v1) IAM role on your project.</p><p><strong>2. Tools</strong></p><p>You will need a command-line environment with the Google Cloud CLI installed and initialized. Choose one of the following:</p><ul><li><strong>Cloud Shell:</strong> An integrated shell environment accessible directly from the Google Cloud Console.</li><li><strong>Cloud Workstation:</strong> Offers a fully customizable and more flexible development environment.</li><li><strong>Local Shell:</strong> Ensure you have the gcloud CLI installed and initialized.</li></ul><p><strong>3. Preemptible GPU Quota Requirements</strong></p><p>DWS Flex consumes your preemptible quotas for the requested machine types. It is critical to check and request sufficient quota in your target region before attempting to provision resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I9VNtql7YL8bHIhvJCNmyA.png" /><figcaption>Preemptible GPU Quota Requirements</figcaption></figure><blockquote><strong><em>Note:</em></strong><em> Your GCP project is likely to have some default preemptible quota in most regions but it is important to check your quota in the target region before running the provisioning commands. If your required quota is higher than the current limit, you must create a quota increase request from the console.</em></blockquote><p><strong>4. Networking for Scalability</strong></p><p>For a demanding, multi-node AI workload, you need high-speed, low-latency communication between VMs. This is achieved using Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE). RDMA allows GPUs on different VMs to exchange data directly, bypassing the CPU and OS, which is critical for distributed training frameworks that rely on collective communication like NCCL.</p><p><strong>Scalability Checkpoint</strong></p><ul><li><strong>Single Node:</strong> If you are running a single-node workload for short-term experimentation, you can use the default network and skip the custom VPC and RDMA configuration steps.</li><li><strong>Future Scale-Up:</strong> Even for single-node work, if you anticipate scaling up to multiple nodes in the future, it is highly recommended to configure the custom VPC network and enable RDMA now. This prevents complex networking rebuilds later.</li></ul><p><strong>Network Setup Script</strong></p><p>For multi-node workloads, create a shell script (<em>create-vpc-rdma.sh</em>) to provision the required VPC networks, subnets, and firewall rules for gVNIC and RDMA (RoCEv2) interfaces.</p><ol><li><strong>Create a file</strong> named <em>create-vpc-rdma.sh</em>.</li><li><strong>Copy and paste</strong> the script below into the file.</li><li><strong>Update the variables</strong> at the top (e.g., <em>GVNIC_NAME_PREFIX</em>, <em>REGION</em>, <em>ZONE</em>, <em>IP_RANGE</em>, etc.).</li><li><strong>Run the script</strong>: <em>chmod +x create-vpc-rdma.sh &amp;&amp; ./create-vpc-rdma.sh</em></li></ol><pre># Define variables - UPDATE THESE!<br>GVNIC_NAME_PREFIX=&quot;a3ultra-gvnic&quot; # e.g., &#39;a4high-gvnic&#39; for A4 VMs<br>RDMA_NAME_PREFIX=&quot;a3ultra-rdma&quot;   # e.g., &#39;a4high-rdma&#39; for A4 VMs<br>REGION=&lt;REGION&gt;                   # check availability<br>ZONE=&lt;ZONE&gt;                       # check availability<br>IP_RANGE=&lt;YOUR_IP/32&gt;             # SSH source IP range (&#39;0.0.0.0/0&#39; for all)<br><br># Create regular VPC networks and subnets for the gVNICs<br>echo &quot;Creating gVNIC networks and subnets...&quot;<br>for N in $(seq 0 1); do<br>  gcloud compute networks create $GVNIC_NAME_PREFIX-net-$N \<br>    --subnet-mode=custom \<br>    --mtu=8896<br><br>  gcloud compute networks subnets create $GVNIC_NAME_PREFIX-sub-$N \<br>    --network=$GVNIC_NAME_PREFIX-net-$N \<br>    --region=$REGION \<br>    --range=10.$N.0.0/16<br><br>  gcloud compute firewall-rules create $GVNIC_NAME_PREFIX-internal-$N \<br>    --network=$GVNIC_NAME_PREFIX-net-$N \<br>    --action=ALLOW \<br>    --rules=tcp:0-65535,udp:0-65535,icmp \<br>    --source-ranges=10.0.0.0/8<br>done<br><br># Create SSH firewall rules (assumes external IP on vNIC 0)<br>echo &quot;Creating SSH firewall rules...&quot;<br>gcloud compute firewall-rules create $GVNIC_NAME_PREFIX-ssh \<br>  --network=$GVNIC_NAME_PREFIX-net-0 \<br>  --action=ALLOW \<br>  --rules=tcp:22 \<br>  --source-ranges=$IP_RANGE<br><br>gcloud compute firewall-rules create $GVNIC_NAME_PREFIX-allow-ping-net-0 \<br>  --network=$GVNIC_NAME_PREFIX-net-0 \<br>  --action=ALLOW \<br>  --rules=icmp \<br>  --source-ranges=$IP_RANGE<br><br># List and ensure network profiles exist in the machine type&#39;s zone<br>echo &quot;Checking for network profiles in $ZONE...&quot;<br>gcloud compute network-profiles list --filter &quot;location.name=$ZONE&quot;<br><br># Create network for CX-7 (MRDMA - RoCEv2)<br>echo &quot;Creating MRDMA (RoCEv2) network...&quot;<br>gcloud compute networks create $RDMA_NAME_PREFIX-mrdma \<br>  --network-profile=$ZONE-vpc-roce \<br>  --subnet-mode custom \<br>  --mtu=8896<br><br># Create subnets for MRDMA<br>echo &quot;Creating MRDMA subnets...&quot;<br>for N in $(seq 0 7); do<br>  gcloud compute networks subnets create $RDMA_NAME_PREFIX-mrdma-sub-$N \<br>    --network=$RDMA_NAME_PREFIX-mrdma \<br>    --region=$REGION \<br>    --range=10.$((N+2)).0.0/16 # offset to avoid overlap with gVNICs<br>done</pre><p>As the <strong>multi-node networking setup</strong> is optional, depending on your scaling needs, you can choose to run this script. If you do, remember to uncomment the corresponding networking configurations in the subsequent provisioning commands (referred to as <strong>Optional Network Setup</strong>). If your workload remains single-node, you may keep these configurations commented out.</p><h3><strong>Strategic Decision: Standalone VM vs. MIG</strong></h3><p>Before running the provisioning commands, it is important to select the right “Consumption Mode” for your specific workload. While both methods utilize DWS Flex capacity, they serve different stages of the development lifecycle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y_t-7vPiHdrlB973jK9XRw.png" /><figcaption>Standalone VM vs MIGs</figcaption></figure><p><strong>Recommendation:</strong> If you are just getting started or running a quick test, choose <strong>Mode A — Standalone GCE Instance</strong>. If you are setting up a repeatable training environment or anticipate scaling, choose <strong>Mode B — Managed Instance Groups</strong>.</p><h3>Hands-On Guide: Provisioning DWS Flex VMs</h3><h4>Consumption Mode A: Standalone GCE Instance</h4><p>For rapid prototyping where you simply need to spin up a machine and get to work, the Standalone GCE approach is the most direct path.</p><p>This method involves a single step: executing the <em>gcloud compute instances create</em> command with the <em>— provisioning-model=FLEX_START</em> flag. Run the following command to request your instance immediately:</p><pre>gcloud compute instances create VM_NAME \ # provide unique VM name<br>    --machine-type=MACHINE_TYPE \ # e.g. a3-ultragpu-8g<br>    --image=IMAGE \ # e.g. ubuntu-accelerator-2204-amd64-with-nvidia-570-v20250624<br>    --image-project=IMAGE_PROJECT \ # e.g. ubuntu-os-accelerator-images<br>    --zone=ZONE \ # e.g. us-central1-b<br>    --boot-disk-type=hyperdisk-balanced \<br>    --boot-disk-size=DISK_SIZE \ # the size of the boot disk in GB<br>    --reservation-affinity=none \ # FLEX_START does not require a reservation<br>    --provisioning-model=FLEX_START \ # Without the FLEX_START flag, the VMs will be charged as per the on-demand rates.<br>    --request-valid-for-duration=REQUEST_VALID_FOR_DURATION \ # the duration that the request to create the VM is valid for - `30m` for 30 minutes<br>    --max-run-duration=MAX_RUN_DURATION \ # the duration you want the requested VMs to run e.g. 1d2h3m4s for one day, two hours, three minutes, and four seconds<br>    --instance-termination-action=DELETE \<br>    --maintenance-policy=TERMINATE \<br>    --scopes=cloud-platform \<br>    [ # (Optional Network Setup)<br>    --network-interface=nic-type=GVNIC,network=GVNIC_NAME_PREFIX-net-0,subnet=GVNIC_NAME_PREFIX-sub-0 \<br>    --network-interface=nic-type=GVNIC,network=GVNIC_NAME_PREFIX-net-1,subnet=GVNIC_NAME_PREFIX-sub-1,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-0,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-1,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-2,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-3,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-4,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-5,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-6,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-7,no-address]</pre><h4>Consumption Mode B: Managed Instance Group (MIG)</h4><p>MIGs are recommended for multi-VM management and easier scaling. This approach requires a few more steps to define a template, but it pays off with greater flexibility for production workloads.</p><p><strong>Step 1: Create an Instance Template</strong></p><p>This template defines the configuration for all VMs in your group.</p><pre>gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME \ # unique template name<br>    --machine-type=MACHINE_TYPE \ # e.g. a3-ultragpu-8g<br>    --image=IMAGE \ # e.g. ubuntu-accelerator-2204-amd64-with-nvidia-570-v20250624<br>    --image-project=IMAGE_PROJECT \ # e.g. ubuntu-os-accelerator-images<br>    --boot-disk-type=hyperdisk-balanced \<br>    --boot-disk-size=DISK_SIZE \ # the size of the boot disk in GB<br>    --reservation-affinity=none \ # FLEX_START does not require a reservation<br>    --instance-termination-action=DELETE \<br>    --maintenance-policy=TERMINATE \<br>    --max-run-duration=RUN_DURATION \ # the duration you want the requested VMs to run e.g. 1d2h3m4s for one day, two hours, three minutes, and four seconds or 20h for 20 hours<br>    --provisioning-model=FLEX_START \ # Without the FLEX_START flag, the VMs will be charged as per the on-demand rates<br>    --scopes=cloud-platform \<br>    [ # (Optional Network Setup)<br>    --network-interface=nic-type=GVNIC,network=GVNIC_NAME_PREFIX-net-0,subnet=GVNIC_NAME_PREFIX-sub-0 \<br>    --network-interface=nic-type=GVNIC,network=GVNIC_NAME_PREFIX-net-1,subnet=GVNIC_NAME_PREFIX-sub-1,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-0,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-1,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-2,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-3,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-4,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-5,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-6,no-address \<br>    --network-interface=nic-type=MRDMA,network=RDMA_NAME_PREFIX-mrdma,subnet=RDMA_NAME_PREFIX-mrdma-sub-7,no-address]</pre><p>You can also achieve similar results through the UI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*Za0vD54Cj6tkmbnPdS14rA.gif" /><figcaption>Create an Instance Template in Google Cloud Console</figcaption></figure><p><strong>Step 2: Create a Zonal MIG</strong></p><p>Crucially, start with a size of 0 for DWS Flex MIGs. This ensures no VMs are created until you explicitly request capacity in the next step.</p><pre>gcloud compute instance-groups managed create MIG_NAME \<br>    --template=INSTANCE_TEMPLATE_URL \<br>    --size=0 \<br>    --default-action-on-vm-failure=do-nothing \<br>    --zone=ZONE</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*tTC-OJdpLoszYloPhDfGQg.gif" /><figcaption>Create Managed Instance Group in Google Cloud Console</figcaption></figure><p><strong>Step 3: Create a Resize Request in the Zonal MIG</strong></p><p>This single command triggers the DWS Flex provisioning system, queuing a request for <em>COUNT</em> number of instances for the specified duration defined in the instance template.</p><pre>gcloud compute instance-groups managed resize-requests create MIG_NAME \<br>    --resize-request=RESIZE_REQUEST_NAME \<br>    --resize-by=COUNT \<br>    --zone=ZONE</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*cqdoWuFuTXAsk6zZ3hOkgA.gif" /><figcaption>Create MIG Resize Request in Google Cloud Console</figcaption></figure><p><strong>Step 4: Monitor the Status of the Instances in Your MIGs</strong></p><p>After submitting the resize request, the VMs may take some time to provision, especially for high-demand resources like H100, H200 or B200 GPUs. Use this command to track their status:</p><pre>gcloud compute instance-groups managed list-instances ${INSTANCE_GROUP_NAME} --zone=${ZONE}</pre><blockquote><em>Important Note on Capacity: </em>On running this command, you may temporarily see a LAST_ERROR such as <em>ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: Waiting for resources. Currently there are not enough resources available to fulfill the request.</em> This is expected and indicates that the DWS Flex Start system is actively queuing your request and waiting for the full capacity to become available. The VM’s STATUS will change to RUNNING once the provisioning is successful.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*jrULbtofiVbvBlHO-hNJqA.gif" /><figcaption>Monitor VM Status in Google Cloud Console</figcaption></figure><p><strong>Step 5: Access the Provisioned VM and Verify Configuration</strong></p><p>Once the VM’s status changes to RUNNING, the instance is fully available. You can grab its generated name from the list-instances output and connect to it using <em>SSH</em>.</p><pre># Replace VM_NAME with the actual instance name from the list-instances output<br>gcloud compute ssh VM_NAME --zone=ZONE</pre><p><strong>Step 6: Verify GPU Drivers and Readiness</strong></p><p>After SSH’ing into the VM instance, the final step is to verify that the NVIDIA GPU drivers are correctly installed and the accelerators are detected.</p><pre>nvidia-smi</pre><p>This command should output information about the detected GPUs, their status, driver version, and CUDA version, confirming your GPUs are ready to run your AI workloads.</p><h3>What’s Next?</h3><p>You have successfully learned how to provision DWS Flex GPUs using GCE and MIGs. In the next parts of this series, we will elevate this by exploring more sophisticated orchestration mechanisms, focusing on how to integrate DWS Flex with <strong>Google Kubernetes Engine (GKE)</strong> and <strong>Slurm</strong> to handle complex, distributed AI workflows at scale.</p><h3>Additional Readings</h3><p>To get started with DWS Flex right away, refer to these official Google Cloud resources:</p><ul><li><a href="https://cloud.google.com/compute/docs/instances/about-flex-start-vms">About Flex-start VMs</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-vm#flex-start_1">DWS Flex-start on GCE</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-mig-with-a4-or-a3ultra">DWS Flex-start on Managed Instance Groups</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/gke-ai-hypercompute-custom#deploy-run-nccl-test">DWS Flex-start on GKE</a></li><li><a href="https://cloud.google.com/cluster-toolkit/docs/deploy/deploy-a3-mega-cluster">DWS Flex-start on Slurm</a></li><li><a href="https://docs.cloud.google.com/vertex-ai/docs/training/schedule-jobs-dws">DWS Flex-start on Vertex AI</a></li></ul><h3>Disclaimer</h3><p>The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of Google Cloud. The code snippets and CLI commands provided are for educational purposes and should be tested in a non-production environment before deployment. Users are responsible for assessing the cost and security implications of their own deployments.</p><h3>Partner with Google Cloud for Success</h3><p>Your success is our priority. We’re here to help you navigate the complexities of GPU availability, answer your questions, and find the best solution for your specific needs. Don’t hesitate to reach out to your <a href="https://cloud.google.com/startup">Google Cloud Startup Success</a> team for expert guidance and support.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=de1594724540" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-2-deploying-gce-instances-using-dws-de1594724540">The Startup’s Guide to GPU Access on Google Cloud — Part 2: Deploying GCE Instances using DWS…</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Startup’s Guide to GPU Access on Google Cloud — Part 1: Why DWS Flex-Start is the POC Power-Up]]></title>
            <link>https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-1-why-dws-flex-start-is-the-poc-power-up-eb322e6f1ae8?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/eb322e6f1ae8</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[ai-infrastructure]]></category>
            <dc:creator><![CDATA[Ankita Deshmukh]]></dc:creator>
            <pubDate>Tue, 18 Nov 2025 14:52:35 GMT</pubDate>
            <atom:updated>2025-11-18T14:52:34.675Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Startup’s Guide to GPU Access on Google Cloud — Part 1: Why DWS Flex-Start is the POC Power-Up</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/923/1*ilX1mvVAFmXCsNfvmZuONQ.png" /><figcaption>Image Generated by Gemini</figcaption></figure><p>As an AI Infrastructure Solutions Architect at Google Cloud, I spend a lot of time helping innovative startups get their AI workloads off the ground. A recurring challenge, particularly for those just beginning their Proof-of-Concept (POC) phase, is the costly and complex dance of securing high-demand compute resources like GPUs.</p><p>Startups operate under tight constraints: limited budget, lean engineering teams, and a critical need to validate their models quickly and cost-effectively without long-term commitments.</p><p>This is where the Dynamic Workload Scheduler (DWS) Flex-Start mode on Google Cloud shines. It’s the perfect bridge between the uncertainty of available capacity and the demand for reliable, short-term compute.</p><h3>Understanding Accelerator Provisioning Models on Google Cloud</h3><p>Before diving deeper into DWS Flex, it’s helpful to understand the various ways you can provision GPUs on Google Cloud. Each model offers different characteristics regarding availability, duration, and catering to diverse workload needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z2okHwpGICdzzjmSwdvGDQ.png" /><figcaption>Provisioning Models on Google Cloud</figcaption></figure><p>Having reviewed the landscape of consumption models, we can see that each has their own pros and cons. For startups and teams running urgent, short-duration projects, this forces a difficult choice. This problem is what we term: <strong><em>The GPU Availability Paradox.</em></strong></p><h3>The GPU Availability Paradox for POCs</h3><p>For startups and teams running short-term POCs that require powerful accelerators (from a few hours to a few days), the traditional provisioning models present significant trade-offs, as highlighted below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*560lUFt5ODP3Co5zhsoz4g.png" /><figcaption>Limitations of Traditional GPU Provisioning Methods for Rapid POCs</figcaption></figure><p>Most startups need a solution that is cheaper than On-Demand, more flexible than reservations and also significantly more reliable than Spot.</p><h3>Introducing DWS Flex: Guaranteed Compute for the Short Haul</h3><p>DWS Flex-start mode is purpose-built to address this exact gap. It provides a unique, queue-based, and non-preemptible way to consume high-demand resources like Cloud GPUs &amp; TPUs. As a fully supported offering, DWS Flex is now Generally Available (GA) on Google Cloud.</p><h4>1. The Critical Non-Preemption Guarantee</h4><p>The defining feature of DWS Flex is its reliability. Once your request is fulfilled and your instances are provisioned, they are guaranteed to run uninterrupted until one of two things happens:</p><ul><li><strong>The workload finishes:</strong> The user manually terminates the instance, or the system may terminate the instances if autoscaling is enabled.</li><li><strong>The maximum duration is reached:</strong> The system automatically terminates the resource.</li></ul><p>Unlike Spot instances, your workload will not be randomly preempted by Google Cloud, making it safe for critical, stateful training and fine-tuning jobs.</p><h4>2. Designed for Short-Term Agility</h4><p>DWS Flex is ideal for short-lived, experimentation-focused workloads:</p><ul><li><strong>Minimum Duration:</strong> 10 minutes</li><li><strong>Maximum Duration:</strong> 7 days</li></ul><h4>3. Pay-for-What-You-Use Cost Efficiency</h4><p>A key cost-saving advantage is the flexibility to terminate early.</p><ul><li>If you commit to a 24-hour run but your model finishes training in 20 hours, you can immediately terminate the resources and only pay for the 20 hours of actual usage.</li><li>This pay-as-you-go model, combined with significant discounts compared to standard On-Demand pricing, makes DWS Flex-start a compelling cost-optimization strategy for budget-conscious startups.</li></ul><h3>How to Use DWS Flex: 5 Paths to Your Accelerated Workload</h3><p>Now that we understand why DWS Flex-start is the ideal choice for reliable, short-term capacity, the next question is: How do I actually use it?</p><p>DWS Flex-start is a provisioning mode integrated across the core Google Cloud compute services. This means you can integrate it into your existing workflow using one of these primary methods:</p><h4>1. Compute Engine (GCE) Instance (Standalone VMs)</h4><ul><li><strong>What it is:</strong> A standalone Flex-start VM is a single Compute Engine instance created directly via the API or CLI, specifying the <em>FLEX_START</em> provisioning model, a maximum run duration, and the desired GPU accelerator. This acts as a direct, non-queued request for a single node.</li><li><strong>Why GCE Instance:</strong> This method is ideal for ad-hoc experimentation, manual fine-tuning, or when you are running a single-node job and do not require any orchestration or auto-scaling. It eliminates the need to configure an Instance Group or a cluster, allowing the fastest path to a running VM.</li></ul><h4>2. Managed Instance Groups (MIGs)</h4><ul><li><strong>What it is:</strong> A Managed Instance Group allows you to create and manage a group of one or more identical VMs. For DWS Flex, you define an Instance Template specifying the <em>FLEX_START</em> provisioning model and then submit a Resize Request to the MIG, asking for your full capacity all at once.</li><li><strong>Why Managed Instance Group:</strong> This is the closest to simply saying, “I need N machines right now.” It requires the least amount of orchestration overhead compared to GKE or Slurm. It’s the lowest-friction option for multi-node training, offering a straightforward way to manage a group of Flex-start VMs.</li></ul><h4>3. Google Kubernetes Engine (GKE)</h4><ul><li><strong>What it is:</strong> GKE is Google Cloud’s managed Kubernetes service. You use DWS Flex by enabling it on a GKE Node Pool and configuring your batch jobs (e.g., a standard Kubernetes Job object or a Ray Job) to target that pool. This method can be leveraged with both Standard and Autopilot clusters.</li><li><strong>Why GKE:</strong> GKE is the foundation for an MLOps/Infra platform. If your team is already familiar with Kubernetes, this is the most logical path. It allows for advanced use cases like seamless model serving/inference alongside training on the same platform, multi-tenancy, and leveraging powerful cloud-native scheduling tools like Kueue. It offers the most elasticity and future-proofing for a growing organization.</li></ul><h4>4. Slurm (HPC/AI Clusters)</h4><ul><li><strong>What it is:</strong> Slurm is a powerful, open-source workload manager used to schedule and manage compute jobs across a cluster of nodes. Google Cloud provides tools to deploy a managed Slurm environment. Users can submit standard batch jobs that leverage the DWS Flex-start provisioning model for the required compute nodes.</li><li><strong>Why Slurm:</strong> Slurm is the preferred choice for distributed model training if your team comes from a strong academic or HPC background and prioritizes a deterministic scheduler with which they are already familiar. It provides a great fine-grained control over resource allocation and placement for highly optimized training workloads.</li></ul><h4><strong>5. Vertex AI</strong></h4><ul><li><strong>What it is:</strong> When you submit a Custom Training Job on Vertex AI, you can specify the <em>scheduling.strategy</em> API field as <em>FLEX_START</em>. Vertex AI then uses the Dynamic Workload Scheduler to queue your job until all required Cloud accelerators (like GPUs or TPUs) become available. You can also leverage DWS Flex-start for model deployment when setting up online inference endpoints for short-duration model serving.</li><li><strong>Why Vertex AI:</strong> This is the highest-level, most integrated, and simplest path for data scientists and ML engineers who prioritize their MLOps workflow. You focus entirely on your training code and container image, as Vertex AI handles all the cluster provisioning, autoscaling, node health checks, and DWS queue management. By setting a maxRunDuration on your training job, compute resources are automatically terminated, ensuring you pay only for active usage.</li></ul><h3>Coming Up Next: Your Technical DWS Flex Quickstart</h3><p>In this first post, we have established the financial and technical imperative for using DWS Flex-start mode for your AI POCs.</p><p>In the next parts of this series, we will dive into the hands-on technical guidance, including:</p><ul><li><strong>Prerequisites:</strong> Required IAM Permissions and Quota configurations for your GCP project.</li><li><strong>Step-by-Step Commands</strong> for requesting DWS Flex resources via GCE, MIGs, GKE, and Slurm.</li><li><strong>Code Artifacts:</strong> Templates for instance creation and job submission.</li></ul><p>Stay tuned to turn this conceptual knowledge into practical infrastructure!</p><h3>Additional Readings</h3><p>To dive deeper into the technical details and best practices for DWS Flex-start, check out the following resources:</p><ul><li><a href="https://cloud.google.com/compute/docs/instances/about-flex-start-vms">About Flex-start VMs</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-vm#flex-start_1">DWS Flex-start on GCE</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/create-mig-with-a4-or-a3ultra">DWS Flex-start on Managed Instance Groups</a></li><li><a href="https://cloud.google.com/ai-hypercomputer/docs/create/gke-ai-hypercompute-custom#deploy-run-nccl-test">DWS Flex-start on GKE</a></li><li><a href="https://cloud.google.com/cluster-toolkit/docs/deploy/deploy-a3-mega-cluster">DWS Flex-start on Slurm</a></li><li><a href="https://docs.cloud.google.com/vertex-ai/docs/training/schedule-jobs-dws">DWS Flex-start on Vertex AI</a></li></ul><h3>Partner with Google Cloud for Success</h3><p>Your success is our priority. We’re here to help you navigate the complexities of GPU availability, answer your questions, and find the best solution for your specific needs. Don’t hesitate to reach out to your <a href="https://cloud.google.com/startup">Google Cloud Startup Success</a> team for expert guidance and support.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb322e6f1ae8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/the-startups-guide-to-gpu-access-on-google-cloud-part-1-why-dws-flex-start-is-the-poc-power-up-eb322e6f1ae8">The Startup’s Guide to GPU Access on Google Cloud — Part 1: Why DWS Flex-Start is the POC Power-Up</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Integrating MCP: A New Standard for AI in Startups]]></title>
            <link>https://medium.com/google-cloud-for-startups/integrating-mcp-a-new-standard-for-ai-in-startups-4eccbb44b54e?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/4eccbb44b54e</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[google-adk]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[google-ai]]></category>
            <dc:creator><![CDATA[Vibha Kurpad]]></dc:creator>
            <pubDate>Thu, 13 Nov 2025 18:30:35 GMT</pubDate>
            <atom:updated>2025-11-13T18:30:33.789Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KnW-E26CJn_Z8-0SNK9h8Q.jpeg" /></figure><h3>The “Why”: Understanding the Motivation and Purpose of MCP</h3><h3>A stroll through history: Our interactions with LLMs</h3><p>The simplest interaction with an LLM sounds like this. You have a prompt, you give it to your LLM, and the LLM generates a response. This flow works great for use cases like summarizing some input, extracting particular information from a document/image, or analyzing some input.</p><p>But what if the LLM needs access to real-time information on the web to answer your prompts, or you need to give your LLM access to some emails to parse through the email body to summarize a conversation? We know that LLMs are extremely capable models, but they need access to tools to carry out these kinds of use cases.</p><p>By tools, we mean the ability to carry out tasks like web search, access to memory, databases, documents, etc. Tools are APIs that the model can use to interact with the outside world, thereby overcoming its inherent limitations, mainly:</p><ul><li>Static Knowledge: Their knowledge was frozen when the training data was collected. Hence, LLMs are not up to date on recent events.</li><li>No real-world interaction: LLM cannot check your calendar, documents, or book an appointment on your behalf.</li></ul><p>Hence, we started giving tool calls to our model. This enabled LLMs to perform a wide range of tasks they were previously limited from.</p><p>Now after providing a prompt, the LLM would decide which tool to call (it could be a tool to initiate a web search for the current weather in a city, it could be a tool call to fetch the price of a current stock or it could be a tool call to fetch a particular document or a row of information from your favourite RDBMS) and then it would give you a response.</p><p>This changed how we interacted with LLMs and led to another operational problem. Each tool was written differently, and there was no standard. Now developers are spending more time writing custom integration code and updating the same across multiple applications, creating bottlenecks.</p><h3>Enter MCP! (And why you should care as a Startup)</h3><p>Model Context Protocol standardised tool calling for large language models. It allowed anyone to create an MCP server exposing their tool to a large language model in a standard way.</p><p>We understand that for any startup, the critical priorities are minimizing time-to-market and establishing a unique value proposition. Model-Context Protocol (MCP) directly accelerates these goals by providing a framework to build intelligent, reliable AI solutions with speed and efficiency.</p><p>For example:</p><ul><li>A <strong>fintech startup</strong> can empower its chatbot with an MCP tool to instantly check a user’s account balance.</li><li>An <strong>e-commerce startup</strong> can build an AI assistant that uses tools to check real-time inventory across warehouses and automatically initiate the shipping process.</li><li>A <strong>legal tech startup</strong> can leverage tools to search specific, up-to-date legal databases, ensuring the precedents provided to lawyers are accurate and verifiable.</li></ul><p>A helpful analogy for MCP is to think of it as a “USB-C port” for AI applications. Just as USB-C provides a standardized way to connect your devices to various peripherals and accessories, MCP provides a standardized way to connect AI models to different data sources and tools. Now developers do not have to worry about:</p><ul><li>Integration Bottlenecks: Managing the integration of various tools with their LLMs.</li><li>Security and Access Control: MCP provides controlled, permission-based access through standardized servers with built-in safety boundaries.</li><li>Improved Developer Experience: MCP standardized patterns for authentication, transport, and error handling across all integrations.</li></ul><h3>MCP Fundamentals: Concepts and Architecture</h3><h4>Core MCP Concepts and Terminology</h4><p>MCP follows a client-server architecture where an MCP host (AI application) establishes connections with one or more MCP servers.</p><ul><li><strong>MCP Host:</strong> The AI application that manages one or more MCP Clients.</li><li><strong>MCP Client:</strong> A component that maintains a connection to an MCP server and obtains context from an MCP server for the MCP host to use</li><li><strong>MCP Server:</strong> A program that provides context to MCP clients.</li><li><strong>Resources:</strong> Resources are how you expose data from files, APIs, databases, or any other source to your LLMs.</li><li><strong>Tools:</strong> Tools enable LLMs to perform actions through server-implemented functions.</li><li><strong>Prompts:</strong> Prompts are reusable templates that help LLMs interact with your server effectively.</li></ul><h4>Data Layers and Transport Layer</h4><p>MCP consists of 2 layers, the data layer and the transport layer.</p><p>The data layer defines a JSON-RPC based protocol to facilitate all client-server communications. Anything to do with lifecycle management, prompts, resources and tools are all taken care of by the data layer.</p><p>The transport layer defines the communication mechanisms and channels that enable data exchange between clients and servers, including transport-specific connection establishment, message framing, and authorization.</p><p>Conceptually the data layer is the inner layer, while the transport layer is the outer layer.</p><h3>MCP + ADK + A2A: How it all works together</h3><p>You may have encountered these terms while reading up on the latest of what is happening in the Agentic world. At first glance, it is easy to assume that these are competing technologies; however, they each serve their own purpose and can be used together to achieve multi-agent architectures.</p><p><strong>What is ADK: </strong>ADK or Agent Development Kit is a model-agnostic and deployment-agnostic open framework meant to simplify AI agent development and deployment.</p><p><strong>What is A2A:</strong> A2A is a protocol that enables various AI agents to discover, communicate, and collaborate with each other regardless of where and how the agent was developed.</p><p>Let’s walk through an example of how these 3 components complement each other and come together.</p><p>I’m working with an AI assistant that needs to order pizza and schedule a dinner event on my calendar.</p><p>So I tell my main AI assistant, “Order a large cheese pizza from ‘Pizza Place’ for 8 PM and schedule a pizza night event on my calendar”.</p><p>First, developers would use ADK to build 2 special-purpose AI Agents.</p><ol><li>Pizza ordering agent: This agent knows how to connect to the Pizza Place’s API, place an order, and handle payment.</li><li>Calendar agent: That knows how to connect to your favourite calendar app and create an event.</li></ol><p>Next, your main AI assistant, essentially acting as an orchestrator, will begin interacting with the previously mentioned agents.</p><p>The main assistant does not know how to order pizza or create a calendar event. It then uses the Model Context Protocol to send standardized requests to these special agents.</p><p>The Pizza Agent receives the MCP command, places the order, and once successful, sends a confirmation message back, like [orderStatus(confirmationId=’XYZ123&#39;, status=’success’)].</p><p>Upon receiving the success message, the main assistant uses MCP again to send a new command to the Calendar Agent: [createEvent(title=’Pizza Night’, time=’7 PM’, details=’Order #XYZ123&#39;)].</p><p>The Calendar Agent creates the event and returns a final confirmation.</p><p>Your main assistant says, “Your pizza is ordered for 8 PM, and I’ve added ‘Pizza Night’ to your calendar.”</p><p>In this scenario:</p><ul><li><strong>ADK</strong> built the specialised agents: the Pizza ordering agent and the Calendar agent</li><li><strong>MCP</strong> was the clear, simple language the main AI assistant used to give instructions.</li></ul><p><strong>A2A</strong> was the entire workflow of the main AI assistant, delegating tasks to the right experts and ensuring the job got done.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tIh1zkistm4zQ58Wrg9rcg.jpeg" /></figure><p>Think of it as a cohesive team: ADK creates the individual players, MCP defines the rules they operate by, and A2A facilitates their collaboration.</p><h3>MCP Use Case: Creating an MCP server for Math operations</h3><p><strong>Prerequisites and initial set up:</strong></p><p>Python 3.10+</p><p>Uv: for package management (Uv is a fast and powerful package and project manager, installation steps <a href="https://docs.astral.sh/uv/getting-started/installation/">here</a>)</p><ul><li>Enable the Artifact Registry, Cloud Run Admin API, and Cloud Build APIs</li><li>Make sure you have the appropriate <a href="https://cloud.google.com/run/docs/reference/iam/roles#additional-configuration">permissions to deploy services</a>, and the <a href="https://cloud.google.com/iam/docs/roles-permissions/run#run.admin">Cloud Run Admin</a> (roles/run.admin) and <a href="https://cloud.google.com/iam/docs/roles-permissions/iam#iam.serviceAccountUser">Service Account User</a> (roles/iam.serviceAccountUser) roles granted to your account.</li><li>Grant the <a href="https://cloud.google.com/run/docs/securing/managing-access#invoker">Cloud Run Invoker (roles/run.invoker)</a> role to your account. This role allows the remote MCP server to access the Cloud Run service.</li></ul><p><strong>Getting started:</strong></p><p>Create a folder called mcp-on-cloudrun to store code for deployment</p><pre>mkdir mcp-on-cloudrun<br>cd mcp-on-cloudrun</pre><p>Next, we’ll use Uv to create a project:</p><pre>uv init --name &quot;mcp-on-cloudrun&quot; --description &quot;Example of deploying a MCP server on Cloud Run&quot; --bare --python 3.10</pre><p>After running the above command, you should see the following pyproject.toml</p><pre>[project]<br>name = &quot;mcp-on-cloudrun&quot;<br>version = &quot;0.1.0&quot;<br>description = &quot;Example of deploying a MCP server on Cloud Run&quot;<br>requires-python = &quot;&gt;=3.10&quot;<br>dependencies = []</pre><p>Create the following additional new files:</p><ul><li>server.py for the MCP server source code</li><li>test_server.py to test the remote server</li><li>A Dockerfile for deploying to Cloud Run</li></ul><pre>touch server.py test_server.py Dockerfile</pre><p>Your project directory should contain the following structure:</p><pre>├── mcp-on-cloudrun<br>│   ├── pyproject.toml<br>│   ├── server.py<br>│   ├── test_server.py<br>│   └── Dockerfile</pre><p>Now that we have our file structure taken care of, let’s configure our Google Cloud credentials and set our project:</p><pre>gcloud auth login<br>export PROJECT_ID=&lt;your-project-id&gt;<br>gcloud config set project $PROJECT_ID</pre><h4>Creating the server</h4><p>To provide valuable context for improving the use of LLMs with MCP, set up a math MCP server with <a href="https://gofastmcp.com/getting-started/welcome">FastMCP</a>. FastMCP provides a quick way to build MCP servers and clients with Python.</p><ol><li>Run the following command to add FastMCP as a dependency in the pyproject.toml file:</li></ol><pre>uv add fastmcp==2.8.0 --no-sync</pre><p>2. Add the following math MCP server source code in the server.py file:</p><pre>import asyncio<br>import logging<br>import os<br><br>from fastmcp import FastMCP <br><br>logger = logging.getLogger(__name__)<br>logging.basicConfig(format=&quot;[%(levelname)s]: %(message)s&quot;, level=logging.INFO)<br><br>mcp = FastMCP(&quot;MCP Server on Cloud Run&quot;)<br><br>@mcp.tool()<br>def add(a: int, b: int) -&gt; int:<br>    &quot;&quot;&quot;Use this to add two numbers together.<br><br>    Args:<br>        a: The first number.<br>        b: The second number.<br><br>    Returns:<br>        The sum of the two numbers.<br>    &quot;&quot;&quot;<br>    logger.info(f&quot;&gt;&gt;&gt; 🛠️ Tool: &#39;add&#39; called with numbers &#39;{a}&#39; and &#39;{b}&#39;&quot;)<br>    return a + b<br><br>@mcp.tool()<br>def subtract(a: int, b: int) -&gt; int:<br>    &quot;&quot;&quot;Use this to subtract two numbers.<br><br>    Args:<br>        a: The first number.<br>        b: The second number.<br><br>    Returns:<br>        The difference of the two numbers.<br>    &quot;&quot;&quot;<br>    logger.info(f&quot;&gt;&gt;&gt; 🛠️ Tool: &#39;subtract&#39; called with numbers &#39;{a}&#39; and &#39;{b}&#39;&quot;)<br>    return a - b<br><br>if __name__ == &quot;__main__&quot;:<br>    logger.info(f&quot;🚀 MCP server started on port {os.getenv(&#39;PORT&#39;, 8080)}&quot;)<br>    # Could also use &#39;sse&#39; transport, host=&quot;0.0.0.0&quot; required for Cloud Run.<br>    asyncio.run(<br>        mcp.run_async(<br>            transport=&quot;streamable-http&quot;,<br>            host=&quot;0.0.0.0&quot;,<br>            port=os.getenv(&quot;PORT&quot;, 8080),<br>        )<br>    )</pre><p>3. Include the following code in the Dockerfile to use the uv tool for running the server.py file:</p><pre># Use the official Python image<br>FROM python:3.13-slim<br><br># Install uv<br>COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/<br><br># Install the project into /app<br>COPY . /app<br>WORKDIR /app<br><br># Allow statements and log messages to immediately appear in the logs<br>ENV PYTHONUNBUFFERED=1<br><br># Install dependencies<br>RUN uv sync<br><br>EXPOSE $PORT<br><br># Run the FastMCP server<br>CMD [&quot;uv&quot;, &quot;run&quot;, &quot;server.py&quot;]</pre><h4>Deploy to Cloud Run</h4><p>You can deploy directly from source, or by using a container image, but for the purpose of this example, we’ll deploy using a container image.</p><ol><li>Create an <a href="https://cloud.google.com/artifact-registry/docs">Artifact Registry</a> repository to store the container image.</li></ol><pre>gcloud artifacts repositories create remote-mcp-servers \<br>--repository-format=docker \<br>--location=us-central1 \<br>--description=&quot;Repository for remote MCP servers&quot; \<br>--project=$PROJECT_ID</pre><p>2. Build the container image and push it to Artifact Registry with <a href="https://cloud.google.com/build">Cloud Build</a>.</p><pre>gcloud builds submit --region=us-central1 --tag us-central1-docker.pkg.dev/<br>$PROJECT_ID<br>/remote-mcp-servers/mcp-server:latest</pre><p>3. Deploy our MCP server container image to Cloud Run.</p><pre>gcloud run deploy mcp-server \<br>--image us-central1-docker.pkg.dev/<br>$PROJECT_ID<br>/remote-mcp-servers/mcp-server:latest \<br>--region=us-central1 \<br>--no-allow-unauthenticated</pre><p>Once you have completed, if your service has successfully deployed you will see a message like the following:</p><pre>Service [mcp-server] revision [mcp-server-12345-abc] has been deployed and is serving 100 percent of traffic.</pre><h4>Authenticate MCP client</h4><p>Since we specified — no-allow-unauthenticated to require authentication, any MCP client connecting to our remote MCP server will need to authenticate.</p><p>The official docs for <a href="https://cloud.google.com/run/docs/host-mcp-servers#authenticate_mcp_clients">Host MCP servers on Cloud Run</a> provides more information on this topic depending on where you are running your MCP client.</p><p>For this example, we will run the <a href="https://cloud.google.com/sdk/gcloud/reference/run/services/proxy">Cloud Run proxy</a> to create an authenticated tunnel to our remote MCP server on our local machines.</p><ol><li>Grant the <a href="https://cloud.google.com/run/docs/securing/managing-access#invoker">Cloud Run Invoker</a> (roles/run.invoker) role to the service account. This Identity and Access Management policy binding makes sure that a strong security mechanism is used to authenticate your local MCP client.</li><li>Run the <a href="https://cloud.google.com/sdk/gcloud/reference/run/services/proxy">Cloud Run proxy</a> to create an authenticated tunnel to the remote MCP server on your local machine:</li></ol><pre>gcloud run services proxy mcp-server --region=us-central1</pre><p>3. If the Cloud Run proxy is not yet installed, this command prompts you to download the proxy. Follow the prompts to download and install the proxy.</p><p>Cloud Run authenticates all traffic to <a href="http://127.0.0.1:8080/">http://127.0.0.1:8080</a> and forwards requests to the remote MCP server.</p><h4>Test the remote MCP server</h4><p>You test and connect to the remote MCP server by using the FastMCP client and accessing the URL <a href="http://127.0.0.1:8080/mcp">http://127.0.0.1:8080/mcp</a>.</p><p>Note: The /mcp at the end as we are using the Streamable HTTP transport</p><p>Add the following code to the empty test_server.py file:</p><pre>import asyncio<br><br>from fastmcp import Client<br><br>async def test_server():<br>    # Test the MCP server using streamable-http transport.<br>    # Use &quot;/sse&quot; endpoint if using sse transport.<br>    async with Client(&quot;http://localhost:8080/mcp&quot;) as client:<br>        # List available tools<br>        tools = await client.list_tools()<br>        for tool in tools:<br>            print(f&quot;&gt;&gt;&gt; 🛠️  Tool found: {tool.name}&quot;)<br>        # Call add tool<br>        print(&quot;&gt;&gt;&gt; 🪛  Calling add tool for 1 + 2&quot;)<br>        result = await client.call_tool(&quot;add&quot;, {&quot;a&quot;: 1, &quot;b&quot;: 2})<br>        print(f&quot;&lt;&lt;&lt; ✅ Result: {result[0].text}&quot;)<br>        # Call subtract tool<br>        print(&quot;&gt;&gt;&gt; 🪛  Calling subtract tool for 10 - 3&quot;)<br>        result = await client.call_tool(&quot;subtract&quot;, {&quot;a&quot;: 10, &quot;b&quot;: 3})<br>        print(f&quot;&lt;&lt;&lt; ✅ Result: {result[0].text}&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(test_server())</pre><p><strong>NOTE:</strong> Make sure you have the Cloud Run proxy running before running the test server.</p><p>In a <strong>new terminal</strong> run:</p><pre>uv run test_server.py</pre><p>You should see the following output:</p><pre> 🛠️ Tool found: add<br> 🛠️ Tool found: subtract<br> 🪛 Calling add tool for 1 + 2<br> ✅ Result: 3<br> 🪛 Calling subtract tool for 10 - 3<br> ✅ Result: 7</pre><p>You successfully deployed a remote MCP server to Cloud Run and tested it using the FastMCP client.</p><h4>Clean up</h4><p>To avoid additional charges to your Google Cloud account, delete all the resources you deployed with this tutorial.</p><h3>Continue Reading</h3><ul><li><a href="https://cloud.google.com/run/docs/host-mcp-servers">Host MCP servers on Cloud Run</a></li><li><a href="https://github.com/GoogleCloudPlatform/cloud-run-mcp">MCP server for helping deploy applications to Cloud Run</a></li></ul><h3>Partner with Google Cloud for Success</h3><p>Your success is our priority. We’re here to help you navigate the complexities of GPU availability, answer your questions, and find the best solution for your specific needs. Don’t hesitate to reach out to your <a href="https://cloud.google.com/startup">Google Cloud Startup Success</a> team for expert guidance and support.</p><h3>Disclaimer</h3><p>The article is not a recommended practice for building and deploying production ready applications, only a fun way to explore the Google Cloud Platform and learn. Any opinions in this article are my own and not those of Google.</p><p>Google Inc makes no guarantee or warranty concerning the accuracy of said information and shall not be responsible for any loss or damage of whatever nature resulting from the use of, or reliance upon it. Google Inc does not guarantee that the use of any information contained herein will not infringe upon patent, trademark, copyright, or rights of third parties.</p><p>The entire demo can be done with the free trial credit ($300 in free credits for new customers still available as of September 16th 2022) and most services should not exceed the free quotas. However, it is your responsibility to follow the clean up instructions to stop all services to avoid incurring charges to your account.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4eccbb44b54e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/integrating-mcp-a-new-standard-for-ai-in-startups-4eccbb44b54e">Integrating MCP: A New Standard for AI in Startups</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Insights from Sound: A Hands-On Guide to HeAR for Healthcare Developers]]></title>
            <link>https://medium.com/google-cloud-for-startups/insights-from-sound-a-hands-on-guide-to-hear-for-healthcare-developers-e118a93759bd?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/e118a93759bd</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[healthai]]></category>
            <category><![CDATA[google-ai-model]]></category>
            <category><![CDATA[hear-model]]></category>
            <category><![CDATA[healthcare]]></category>
            <dc:creator><![CDATA[Ada Amanna]]></dc:creator>
            <pubDate>Wed, 30 Jul 2025 17:36:11 GMT</pubDate>
            <atom:updated>2025-07-30T17:36:11.531Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bt6z5VyJ5_m0PFYb" /><figcaption>Created with Google <a href="https://deepmind.google/models/veo/">Veo3</a></figcaption></figure><p>Our journey through Google Cloud’s groundbreaking domain-specific Models continues! <a href="https://medium.com/google-cloud-for-startups/unlocking-therapeutic-innovation-with-txgemma-a-hands-on-guide-for-healthcare-developer-4400e411f5f8?source=user_profile_page---------0-------------83cc658a04b6----------------------">In our last post</a>, we explored TxGemma, diving into how specialized AI is revolutionizing therapeutic development. Now, we turn our attention to another frontier: <strong>Sound as a health signal.</strong> Google’s <strong>Health Acoustic Representations (HeAR) model</strong>. For startups in the HCLS space, HeAR is particularly valuable because it represents a <strong>widely accessible model</strong> that enables rapid prototyping and scaling of innovative solutions without the immense upfront investment typically required for building highly specialized AI from scratch. This capability is especially critical in today’s healthcare landscape, where remote monitoring and telemedicine have become essential. HeAR empowers developers to create applications that can provide preliminary health assessments through something as simple as a smartphone recording, potentially democratizing access to respiratory health screening globally. By leveraging Google Cloud’s robust platform, startups can accelerate their R&amp;D cycles, maximize the impact of their limited resources, and bring novel insights from health audio data to market faster than ever before.</p><p>These capabilities are especially impactful in areas like Pulmonology and respiratory health, where the sensitivity and complexity of audio signals necessitates a specialized approach. Traditional machine learning models often struggle to capture the nuanced acoustic features that indicate early signs of respiratory conditions. HeAR is built to extract meaningful, high-quality representations from diverse health audio signals, offering targeted solutions for diagnostics, monitoring, and research where the slightest deviation in sound could be the key to early detection of critical diseases like tuberculosis, COVID-19, and chronic obstructive pulmonary disease (COPD).</p><h3>HeAR at a Glance: Background And Functionality</h3><p><strong>Background</strong>: HeAR is a Masked Autoencoder, a self-supervised learning model that utilizes a transformer-based architecture optimized for audio processing, to capture both temporal and spectral features crucial for health-related audio analysis. This means the model learns by predicting masked audio segments, allowing it to capture rich contextual information without requiring manually labeled data during the initial pre-training phase. The model is trained on a massive YouTube Non-semantic (YT-NS) dataset of over 300 million two-second audio clips, summing up to 174,000 hours of audio. This extensive training enables HeAR to understand the subtle acoustic patterns that correlate with various health conditions, particularly respiratory ailments.</p><p><strong>Input vs Output Expectation:</strong> HeAR’s core function is to generate <strong>high-quality audio embeddings, </strong>not predictions or independent diagnosis. These embeddings, which are the compact, numerical representations of the audio clips, captures their essential characteristics and can then be used as input for various downstream machine learning tasks. Hence, for every two-second long 16kHz input clip, HeAR outputs a rich 512-long embedding vector that captures health-relevant acoustic features, allowing for a more nuanced downstream analysis. The generated embeddings further enable efficient AI model training for health acoustics with less data and compute compared to random or generic dataset initializations. This makes HeAR a powerful tool for rapid prototyping in health acoustics anomaly detection and sound classification (e.g., coughs, heart murmurs).</p><h3>Potential Use Cases for HCLS Startups</h3><p>Non-semantic audio data hold crucial diagnostics information that can be wielded for the following use cases.</p><p><strong>1. Respiratory Condition Classification:</strong> HeAR enables data-efficient classification of respiratory conditions through acoustic biomarker analysis, making it ideal for applications such as:</p><ul><li><strong>COVID-19 Detection</strong>: Identifying subtle changes in cough patterns and breathing associated with COVID-19 infection</li><li><strong>Tuberculosis Screening</strong>: Detecting characteristic cough patterns that may indicate TB, particularly valuable in resource-limited settings and telemedicine platforms</li><li><strong>COPD Assessment</strong>: Evaluating chronic obstructive pulmonary disease through cough and breath sound analysis</li><li><strong>Asthma Monitoring</strong>: Analyzing breathing patterns to assess asthma severity and predict potential exacerbations</li></ul><p><strong>Broader Applications:</strong></p><ul><li><strong>Telemedicine Platforms</strong>: Integrate HeAR into telehealth applications to provide objective, AI-powered preliminary assessments of respiratory health during virtual consultations</li><li><strong>Mobile Health Applications</strong>: Develop consumer-facing apps that can assess respiratory health using smartphone microphones, providing users with insights into their respiratory wellness</li></ul><p><strong>2. Health Acoustic Identification &amp; Monitoring: </strong>HeAR is adept at identifying and classifying different types of health acoustic events, enabling comprehensive respiratory monitoring solutions. For example:</p><ul><li><strong>Cough Detection and Analysis</strong>: Identifying various types of coughs and their characteristics</li><li><strong>Wheeze Recognition</strong>: Detecting wheezing patterns associated with respiratory distress</li><li><strong>Snore Analysis</strong>: Classifying sleep-related breathing patterns and potential sleep disorders</li></ul><p><strong>Broader Applications:</strong></p><ul><li><strong>Remote Patient Monitoring</strong>: Create comprehensive monitoring solutions for patients with chronic respiratory conditions, enabling continuous health assessment between clinical visits</li><li><strong>Clinical Decision Support Systems</strong>: Build tools that assist healthcare professionals by providing acoustic analysis alongside traditional diagnostic methods</li></ul><p><strong>3. Disease Severity Assessment and Progression Tracking: </strong>HeAR can classify the severity of respiratory diseases based on acoustic features and track changes over time:</p><ul><li><strong>Longitudinal Analysis</strong>: Comparing audio patterns across multiple time points to detect health changes</li><li><strong>Treatment Response Monitoring</strong>: Tracking how respiratory sounds change in response to treatment interventions</li><li><strong>Early Warning Systems</strong>: Identifying subtle acoustic changes that precede clinical symptom onset</li></ul><p><strong>Broader Applications:</strong></p><ul><li><strong>Pharmaceutical Research and Development</strong>: Leverage HeAR for clinical trials and drug development, providing objective acoustic biomarkers to assess treatment efficacy</li><li><strong>Population Health Monitoring</strong>: Large-scale screening for respiratory conditions in communities and real-time assessment of respiratory health trends</li></ul><p><strong>4. Quantitative Acoustic Measurements: </strong>Beyond classification tasks, HeAR can perform regression analysis to measure quantities proportional to acoustic intensity:</p><ul><li><strong>Spirometry Analysis</strong>: Measuring lung function parameters through acoustic analysis</li><li><strong>Urine Flow Assessment</strong>: Quantifying flow rates based on acoustic patterns</li><li><strong>Respiratory Volume Estimation</strong>: Assessing breathing capacity and patterns</li></ul><p><strong>Broader Applications:</strong></p><ul><li><strong>Digital Biomarker Platforms</strong>: Develop solutions that provide quantitative health metrics through non-invasive acoustic analysis</li><li><strong>Clinical Trial Support</strong>: Provide objective acoustic endpoints for respiratory medication trials and treatment efficacy studies</li></ul><h3>Accessing and Using HeAR: A Hands-On Guide</h3><p>HeAR is designed for flexible deployment across various computational environments, making it accessible for both research exploration and production-scale applications. The model’s versatility shines through its multiple access routes, each optimized for different use cases and computational requirements. Let’s explore these access routes with practical implementation examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SkGCDhLfK09kzARD" /><figcaption>Figure 1: End-to-End Pipeline for Processing Raw Audio with HeAR on Google Cloud to Generate Embeddings for Downstream Machine Learning.</figcaption></figure><h3>Local Execution</h3><p>For developers who want to run HeAR locally on their own hardware, you can download the model from <a href="https://huggingface.co/google/hear">Hugging Face</a>. This approach is ideal for preliminary exploration and prototyping where you need direct control over the model execution environment. In the code snippet below, we will explore a local HeAR implementation to extract embeddings from respiratory audio samples.</p><p><strong>NB</strong>: In addition to embedding extraction, you can also follow these guides to explore <a href="https://github.com/Google-Health/hear/blob/master/notebooks/hear_event_detector_demo.ipynb">sound detection</a> and <a href="https://github.com/Google-Health/hear/blob/master/notebooks/train_data_efficient_classifier.ipynb">classification</a> with HeAR.</p><p><strong>Disclaimer:</strong> <em>The following code snippets are for educational purposes and not a recommended practice for building and deploying production ready applications, only a fun way to explore the Google Cloud Platform and learn. Any opinions in this article are my own and not those of Google. Additionally, this entire demo can be done with free trial credit ($300 in free credits for new customers as of July, 2025) and most services should not exceed the free quotas. However, it is your responsibility to clean up services when done to avoid incurring charges to your account.</em></p><pre>#Local Execution: Generating Audio Embeddings with HeAR via pyorch<br>#Note: you will need a Hugging Face account to obtain an access token <br><br># Install required packages<br>! git clone https://github.com/Google-Health/hear.git<br>! pip install --upgrade --quiet transformers==4.50.3<br><br># Import necessary libraries<br>from huggingface_hub.utils import HfFolder<br>from IPython.display import Audio, display<br>from transformers import AutoModel<br>from scipy.io import wavfile<br>import matplotlib.pyplot as plt<br>import numpy as np<br>import importlib<br>import torch<br><br>#Authenticate with HF if token secret is unavailable, otherwise skip<br><br>if HfFolder.get_token() is None:<br>    from huggingface_hub import notebook_login<br>    notebook_login()<br><br>#download sample audio recording<br>!wget -nc https://upload.wikimedia.org/wikipedia/commons/b/be/Woman_coughing_three_times.wav<br><br># Load audio file<br>with open(&#39;Woman_coughing_three_times.wav&#39;, &#39;rb&#39;) as f:<br>  original_sampling_rate, audio_array = wavfile.read(f)<br><br>#Initialize audio preprocessing constants <br>SAMPLE_RATE = 16000  <br>CLIP_DURATION = 2  <br>CLIP_LENGTH = SAMPLE_RATE * CLIP_DURATION      <br><br>audio_utils = importlib.import_module(<br>    &quot;hear.python.data_processing.audio_utils&quot;)<br>resample_audio_and_convert_to_mono = audio_utils.resample_audio_and_convert_to_mono<br><br>audio_array = resample_audio_and_convert_to_mono(<br>  audio_array=audio_array, <br>  sampling_rate=original_sampling_rate,<br>  new_sampling_rate=SAMPLE_RATE,<br>)<br>display(Audio(audio_array, rate=SAMPLE_RATE))<br><br># Load HeAR model and processor from Hugging Face<br>model_name = &quot;google/hear-pytorch&quot;<br>processor = AutoProcessor.from_pretrained(model_name)<br>model = AutoModel.from_pretrained(<br>    model_name,<br>    torch_dtype=torch.float16,  # Use half-precision for efficiency<br>    device_map=&quot;auto&quot;  # Automatically use available GPUs or fallback to CPU<br>)<br><br>#Compute embeddings<br>preprocess_audio = audio_utils.preprocess_audio<br><br>START = 0 #cough index<br># Add batch dimension<br>input_tensor = np.expand_dims(audio_array[START: START + CLIP_LENGTH], axis=0)<br><br># Call inference<br>infer = lambda audio_array: loaded_model.forward(<br>    preprocess_audio(audio_array), return_dict=True, output_hidden_states=True)<br>output = infer(torch.Tensor(input_tensor))<br><br># Extract the embedding vector<br>embedding_vector = np.asarray(output.pooler_output.detach()).flatten()<br>print(&quot;Size of embedding vector:&quot;, len(embedding_vector))<br><br># Plot the embedding vector<br>plt.figure(figsize=(12, 4))<br>plt.plot(embedding_vector)<br>plt.title(&#39;Embedding Vector&#39;)<br>plt.xlabel(&#39;Index&#39;)<br>plt.ylabel(&#39;Value&#39;)<br>plt.grid(True)<br>plt.show()</pre><h3>Vertex AI Model Garden Deployment</h3><p>For production-grade deployment, you can conveniently deploy HeAR to a managed endpoint on Vertex AI’s “<strong>Model Garden</strong>” with one-click deployment. This route allows you to quickly create a scalable endpoint for both batch and real-time prediction requests, while providing high availability and managed infrastructure with <a href="https://cloud.google.com/vertex-ai/sla">SLA</a> guarantees. Let’s explore the different paths of Vertex deployment.</p><p>NB: To deploy on Vertex AI, you’ll need a <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects">Google Cloud Project</a> set up and the following <a href="https://cloud.google.com/vertex-ai/docs/general/access-control#predefined-roles">IAM roles</a>: Vertex AI User (roles/aiplatform.user) on a basic level to use resources on vertex, and/or Vertex AI Administrator (roles/aiplatform.admin) for full access to all Vertex resources.</p><h3>Vertex AI One-Click Deployment</h3><ol><li>Open <a href="https://console.cloud.google.com/"><strong>Google cloud console</strong></a><strong>,</strong> navigate to Vertex AI platform.</li><li>In the sidebar click on <strong>Model Garden</strong>.</li><li>Within the Model Garden, search and select <strong>HeAR</strong> model</li><li>Under <strong>Deployment Option</strong>s, select <strong>One-click deployment</strong> Vertex AI</li><li>Fill out the deployment configuration:</li></ol><ul><li><strong>Resource ID</strong>: Unique identifier for the deployment workload (google/hear)</li><li><strong>Endpoint name</strong>: Preferred name for your HeAR deployment</li><li><strong>Deployment setting</strong>: Choose between Basic and Advanced configurations</li><li><strong>Region</strong>: Select your preferred deployment region (consider latency to your end users)</li><li><strong>Machine spec</strong>: Choose relevant machine type (e.g., n1-standard-4)</li></ul><p>6. Advanced Configuration (if selected):</p><ul><li><strong>Availability Policies</strong>: Configure auto-scaling and availability zones</li><li><strong>VM Provisioning</strong>: Choose between Standard (guaranteed) vs Spot (cost-effective)</li><li><strong>Endpoint Access</strong>: Set to Public or Private based on your security requirements</li></ul><p>7. Click <strong>Deploy</strong> and wait for the endpoint to be provisioned (typically takes 5–15 minutes)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MjgpiFZFUjs1qE-k" /><figcaption>Figure 2: One-click Deployment on VertexAI console</figcaption></figure><p><strong>Vertex AI Deployment (gcloud CLI or Python)</strong></p><p>You can also deploy HeAR programmatically using the gcloud CLI or Python with the Vertex AI SDK. This is particularly useful for automating deployments or integrating them into MLOps pipelines.</p><pre>#install the Vertex AI SDK<br>pip install --upgrade google-cloud-aiplatform<br>gcloud auth application-default login<br><br>#Deploy the model<br>import vertexai<br>from vertexai.preview import model_garden<br><br>vertexai.init(project=&quot;white-vortex-449419-n7&quot;, location=&quot;us-central1&quot;)<br><br>model = model_garden.OpenModel(&quot;google/hear@hear&quot;)<br>endpoint = model.deploy(<br>  accept_eula=True,<br>  machine_type=&quot;n1-standard-4&quot;,<br>  serving_container_image_uri=&quot;us-docker.pkg.dev/deeplearning-platform-release/vertex-model-garden/health-ai-hear.cpu.1-0.ubuntu2004.py312.tf218:20250311-1800-rc0&quot;,<br>  endpoint_display_name=&quot;google_hear-mg-one-click-deploy&quot;,<br>  model_display_name=&quot;google_hear-1748972084590&quot;,<br>  use_dedicated_endpoint=True,<br>)</pre><pre>#authenticate google account<br>gcloud auth login<br><br>#deploy model<br>gcloud beta ai model-garden models deploy \<br>--model=&quot;google/hear@hear&quot; \<br>--region=&quot;us-central1&quot; \<br>--project=&quot;white-vortex-449419-n7&quot; \<br>--accept-eula \<br>--machine-type=&quot;n1-standard-4&quot; \<br>--container-image-uri=&quot;us-docker.pkg.dev/deeplearning-platform-release/vertex-model-garden/health-ai-hear.cpu.1-0.ubuntu2004.py312.tf218:20250311-1800-rc0&quot; \<br>--use-dedicated-endpoint \<br>--endpoint-display-name=&quot;google_hear-mg-one-click-deploy&quot;</pre><h3>Batch Processing with Vertex AI</h3><p>For large-scale data processing, HeAR can be deployed as a Vertex AI batch prediction job. This approach is cost-effective for processing thousands of audio files without the need for real-time responses. You can create a HeAR focused batch prediction job to run on a defined set of audio clips (each constrained to two seconds). Here, the prediction request is not sent to a deployed endpoint from the model garden, rather they are sent directly to an uploaded model to generate associated embeddings.</p><p>You can use the Google Cloud console to request a batch prediction on HeAR following the steps below.</p><h3>Requirements</h3><ul><li>Prebuilt HeAR model, uploaded to the Vertex AI Model Registry</li><li>Compute Engine Service Account to run the batch prediction Jobs</li><li>Service Account User Role (roles/iam.serviceAccountUser) assigned to the project or the compute engine above</li><li>Format each input Audio file using <a href="https://jsonlines.org/">JSON Lines</a> and store in a Google cloud storage bucket</li></ul><h3>Batch processing steps</h3><h3>Batch processing steps</h3><ol><li>Open <a href="https://console.cloud.google.com/">Google cloud console</a>, navigate to Vertex AI platform.</li><li>In the sidebar click and navigate to the <strong>Batch predictions</strong> page.</li><li>Click “create” to initiate a new <strong>batch prediction job</strong></li><li>For <strong>your batch prediction job</strong>, complete the following steps:</li></ol><ul><li>Enter a name for your batch prediction.</li><li>For <strong>Model name</strong>, select the name of the pre-built Hear model from the registry.</li><li>For <strong>Select source, </strong>select the source that applies to your input data:</li></ul><p>a. Select File on (<strong>Cloud Storage JSONL)</strong></p><p>b. Specify the <strong>Source path</strong> field</p><ul><li>For Batch <strong>Prediction Output configuration</strong></li></ul><p>a. Select <strong>JSONL</strong> as the output format</p><p>b. In the <strong>Destination path</strong> field, specify the Cloud Storage bucket for storing the batch prediction output.</p><ul><li>Click <strong>Create</strong></li></ul><p><strong>Research Endpoint</strong></p><p>If you are still unsure of the model’s relevance to your use case, initial testing and evaluation, Google provides a research endpoint that allows you to experiment with HeAR without deploying your own infrastructure. This is perfect for preliminary Proof of concepts and model evaluation without the pressure to commit. You can access the research endpoint following these steps below</p><h3>Accessing Research Endpoint</h3><ol><li>First, determine applicable <a href="https://developers.google.com/health-ai-developer-foundations/model-serving/research-endpoints#access_options">access option</a> (individual vs commercial access)</li><li>Fill out the <a href="https://docs.google.com/forms/d/e/1FAIpQLSc0Riwp31Vq033_Prklyfp_rSp3OqdgC7xHMzoB98-sDSw5KA/viewform">Access Request Form</a> for HeAR model</li><li>If approved, you will receive further instructions on how to access the associated research endpoint.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nE3UzHEvvqy7Fm1npWpRFw.png" /></figure><p>In summary, each access route offers unique advantages depending on your specific use case, scale requirements, and technical constraints. The local execution route provides maximum control and privacy, while Vertex AI deployments offer enterprise-grade scalability and reliability. Choose the approach that best aligns with your startup’s current needs and growth trajectory.</p><h3>Conclusion and Future Directions</h3><p>HeAR is a foundational model, and its capabilities will continue to expand, opening up exciting new avenues for research and application in health audio AI:</p><ul><li><strong>Enhanced Specificity:</strong> Further fine-tuning or specialization for very niche medical audio tasks (e.g., specific vocal biomarkers for neurological disorders).</li><li><strong>Expanded Input Length</strong>: Moving beyond the current 2-second input constraint to accommodating longer audio inputs.</li><li><strong>Multi-modal Integration:</strong> Combining HeAR’s audio embeddings with other data modalities (e.g., patient text records from TxGemma-Chat, imaging data from CXR Foundation) to build more comprehensive diagnostic and prognostic models. Imagine correlating lung sounds with chest X-ray findings and patient history for a more precise diagnosis.</li><li><strong>Real-time Edge Deployment:</strong> Optimizing HeAR for ultra-low latency inference on edge devices for continuous, real-time monitoring in clinical or home settings.</li></ul><p>HeAR marks a significant advancement in leveraging audio data for healthcare applications. By providing universal health audio representations, it simplifies complex audio analysis, empowers developers to build sophisticated solutions with less effort, and opens doors to innovative diagnostic and monitoring tools. Its pre-trained knowledge allows for highly effective transfer learning, democratizing access to powerful audio AI capabilities. We’ve explored its technical underpinnings, walked through how to access and use it locally and on Google Cloud, and highlighted its diverse and impactful use cases from clinical diagnostics to public health surveillance.</p><p>This is just the next step in our series on Google’s HCLS Domain Specific Models. We encourage you to explore HeAR further using the provided resources and imagine how it can transform your healthcare solutions. In our upcoming blog post, we’ll dive into Google’s latest collection of open models, MedGemma. Stay tuned!</p><h3>Resources</h3><ul><li><a href="https://developers.google.com/health-ai-developer-foundations/hear">HeAR Documentation</a></li><li><a href="https://developers.google.com/health-ai-developer-foundations">Google Health AI Developer Foundations</a></li><li><a href="https://cloud.google.com/vertex-ai/pricing">Vertex AI Pricing</a></li><li><a href="https://cloud.google.com/vertex-ai/docs/predictions/get-batch-predictions#input_data_requirements">Batch Prediction Configuration</a>:</li><li><a href="https://developers.google.com/health-ai-developer-foundations/quick-model-guide">Research Endpoints for HAI-DEF Models</a></li><li><a href="https://developers.google.com/health-ai-developer-foundations/model-serving/research-endpoints#access_options">Research Endpoint Access Options</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e118a93759bd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/insights-from-sound-a-hands-on-guide-to-hear-for-healthcare-developers-e118a93759bd">Insights from Sound: A Hands-On Guide to HeAR for Healthcare Developers</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlock Production-Ready AI: Build Your Agent with Vertex AI Agent Engine & ADK]]></title>
            <link>https://medium.com/google-cloud-for-startups/unlock-production-ready-ai-build-your-agent-with-vertex-ai-agent-engine-adk-4ba31dfceb11?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/4ba31dfceb11</guid>
            <category><![CDATA[vertex-ai]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[adk]]></category>
            <dc:creator><![CDATA[Maurice]]></dc:creator>
            <pubDate>Fri, 27 Jun 2025 17:05:42 GMT</pubDate>
            <atom:updated>2025-07-14T15:01:22.498Z</atom:updated>
            <content:encoded><![CDATA[<p>Are you struggling to take your innovative AI agent prototypes from your development environment to a scalable, reliable production system? Do concerns about managing infrastructure, ensuring uptime, and handling complex data integrations keep you up at night? You’re not alone. Many developers face these hurdles when trying to manage their AI Agents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tQN8n87MBS9ikuoEEPUykg.jpeg" /></figure><h3>Introduction</h3><p>This article aims to guide you through building and deploying production-ready AI agents using <strong>Vertex AI Agent Engine</strong>, Google Cloud’s fully managed service designed to simplify this very challenge. We’ll demonstrate how you can leverage the <strong>Agent Development Kit (ADK)</strong> to rapidly prototype and deploy robust agent-based solutions.</p><p>Agent development is the first crucial step before deploying on Vertex AI Agent Engine. While you can build custom agents, utilizing pre-built, framework-specific templates is often the simplest path. These templates streamline common development tasks by managing object serialization and separating initialization from prompt response logic. Here are some of the available framework-specific templates:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*eCRV5beBys9gT07O-1VdSA.png" /></figure><p>For use cases not covered by these templates, you have the option to<a href="https://cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/develop/custom"> develop a custom agent</a>.</p><p><strong>Disclaimer:</strong> This article utilizes the <strong>Agent Development Kit (ADK)</strong> to construct an agent. Our demonstration focuses on an agent designed to retrieve vehicle complaint data from an external API.</p><p><strong>Important Considerations:</strong></p><ul><li>This article is intended for exploratory learning within Google Cloud Platform and is not a recommended guide for building and deploying production-ready applications.</li><li>The opinions expressed are solely my own and do not reflect those of Google.</li><li>Google Inc. disclaims all warranties regarding the accuracy of this information and will not be liable for any losses or damages incurred from its use or reliance.</li><li>Google Inc. does not guarantee that the use of this information will not infringe upon third-party intellectual property rights (patent, trademark, copyright).</li></ul><p><strong>Cost and Usage:</strong></p><ul><li>The entire demonstration can be completed using the free trial credit ($300 for new customers, available as of September 16th, 2022).</li><li>Most services should not exceed free quotas.</li><li>However, it is your responsibility to follow the cleanup instructions to stop all services and avoid incurring charges to your account.</li></ul><h3>Use Cases for AI Agents</h3><p>AI agents built with Vertex AI Agent Engine can tackle a wide variety of tasks, such as:</p><ul><li><strong>Customer Service Automation:</strong> Handling FAQs, guiding users through processes, and escalating complex issues.</li><li><strong>Data Retrieval &amp; Summarization:</strong> Fetching information from various sources (databases, APIs) and providing concise summaries.</li><li><strong>Task Orchestration:</strong> Automating multi-step workflows by interacting with different tools and services.</li><li><strong>Personalized Assistants:</strong> Providing tailored information or recommendations based on user context.</li></ul><h3>Skill-Set Prerequisites</h3><ul><li>Basic understanding of Python programming.</li><li>Familiarity with Google Cloud Platform (GCP) concepts.</li><li>Experience with Large Language Models (LLMs) is helpful but not strictly required to follow along.</li></ul><h3>Step-by-Step Walkthrough: Building a Car Complaint Agent with ADK</h3><p>Let’s dive into building an agent that can fetch car complaint data from the National Highway Traffic Safety Administration (NHTSA) API.</p><h4>Prerequisites: Setting Up Your Google Cloud Environment</h4><p>Before you begin, ensure your Google Cloud environment is properly set up:</p><ol><li><strong>Have a Google Cloud Project:</strong> Select or create a Google Cloud project and ensure billing is enabled for it.</li><li><strong>Enable Necessary APIs:</strong> Enable the Vertex AI, Cloud Storage, Cloud Logging, Cloud Monitoring, and Cloud Trace APIs within your project.</li><li><strong>Obtain Required IAM Roles:</strong> Your Google Cloud user account needs appropriate permissions, specifically the Vertex AI User (roles/aiplatform.user) and Storage Admin (roles/storage.admin) roles on your project.</li><li><strong>Set up Service Agent Permissions:</strong> Vertex AI Agent Engine agents use the AI Platform Reasoning Engine Service Agent service account. This account has default permissions granted by the Vertex AI Reasoning Engine Service Agent role. If your agent needs to access other Google Cloud services, grant additional permissions to this service agent (principal: service-PROJECT_NUMBER@gcp-sa-aiplatform-re.iam.gserviceaccount.com) in the IAM page (ensure “include Google-provided role grants” is checked).</li><li><strong>Create a Cloud Storage Bucket:</strong> Set up a Cloud Storage bucket in your project (e.g., gs://agentassets). This bucket is used by Vertex AI Agent Engine to stage your agent’s code and artifacts during deployment. Ensure your user account has Storage Admin access to this bucket.</li></ol><p>For detailed instructions, please refer to the official <a href="https://cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/set-up">Vertex AI Agent Engine documentation</a>.</p><h4>Architecture Diagram</h4><p>Here’s a reference architecture from the official docs of how are agent will look:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/0*q-qsS0DjWab6_53n" /></figure><h4>Deployment/Configuration of GCP Services &amp; Code</h4><p>We’ll be working within a Colab environment, but the principles apply broadly.</p><ol><li><strong>Initial Setup and Authentication (Colab/Python Environment)</strong></li></ol><pre># Install required packages<br>!pip install google-cloud-aiplatform[adk,agent_engines] requests</pre><pre>from google.colab import auth<br>auth.authenticate_user(project_id=&quot;[project_ID]&quot;) # Replace with your project ID</pre><p>Initialize Vertex AI, specifying your project, location, and staging bucket.</p><pre>import vertexai<br>from vertexai import agent_engines<br>from google.adk.agents import Agent<br>from vertexai.preview.reasoning_engines import AdkApp<br><br>vertexai.init(<br>    project=&quot;[project_ID]&quot;,<br>    location=&quot;[Location]&quot;,<br>    staging_bucket=&quot;[Your-bucket]&quot;,<br>)</pre><p><strong>2. Defining a Tool: The get_complaints Function</strong></p><pre>import requests<br><br># --- Tool Definition ---<br>def get_complaints(make: str, model: str, model_year: str) -&gt; dict:<br>    &quot;&quot;&quot;<br>    Retrieves complaints about the vehicle&#39;s make, model, and model year from NHTSA.<br><br>    Args:<br>        make (str): The manufacturer (make) of the vehicle (e.g., Toyota).<br>        model (str): The specific model of the vehicle (e.g., Camry).<br>        model_year (str): The model year of the vehicle (e.g., 2022).<br><br>    Returns:<br>        dict: The JSON response directly from the NHTSA API containing complaints data,<br>              or an error dictionary if the request fails or returns no results.<br>    &quot;&quot;&quot;<br>    # Use f-string correctly to insert variables into the URL<br>    api_url = f&quot;https://api.nhtsa.gov/complaints/complaintsByVehicle?make={make}&amp;model={model}&amp;modelYear={model_year}&quot;<br>    print(f&quot;--- Tool Call: get_complaints(make=&#39;{make}&#39;, model=&#39;{model}&#39;, model_year=&#39;{model_year}&#39;) ---&quot;)<br>    print(f&quot;--- Calling NHTSA API URL: {api_url} ---&quot;)<br><br>    try:<br>        # Make the GET request - remove the incorrect &#39;params&#39; argument<br>        response = requests.get(api_url, timeout=15) # Add a timeout<br><br>        # Check for HTTP errors (e.g., 404 Not Found, 500 Server Error)<br>        response.raise_for_status()<br><br>        result_data = response.json()<br><br>        # Check if the API returned results successfully<br>        if result_data and result_data.get(&quot;count&quot;, 0) &gt; 0:<br>             print(f&quot;--- NHTSA API Success: Found {result_data.get(&#39;count&#39;)} complaints. ---&quot;)<br>             # Return the full JSON response if successful and contains data<br>             return result_data<br>        else:<br>            print(f&quot;--- NHTSA API Info: No complaints found for the specified vehicle. ---&quot;)<br>            return {&quot;status&quot;: &quot;success&quot;, &quot;message&quot;: &quot;No complaints found for the specified vehicle.&quot;}<br><br>    except requests.exceptions.HTTPError as http_err:<br>        print(f&quot;--- NHTSA API HTTP Error: {http_err} ---&quot;)<br>        return {&quot;status&quot;: &quot;error&quot;, &quot;error_message&quot;: f&quot;HTTP error accessing NHTSA API: {http_err}&quot;}<br>    except requests.exceptions.ConnectionError as conn_err:<br>        print(f&quot;--- NHTSA API Connection Error: {conn_err} ---&quot;)<br>        return {&quot;status&quot;: &quot;error&quot;, &quot;error_message&quot;: f&quot;Could not connect to NHTSA API: {conn_err}&quot;}<br>    except requests.exceptions.Timeout as timeout_err:<br>        print(f&quot;--- NHTSA API Timeout Error: {timeout_err} ---&quot;)<br>        return {&quot;status&quot;: &quot;error&quot;, &quot;error_message&quot;: f&quot;Request to NHTSA API timed out: {timeout_err}&quot;}<br>    except requests.exceptions.RequestException as req_err:<br>        print(f&quot;--- NHTSA API Request Error: {req_err} ---&quot;)<br>        return {&quot;status&quot;: &quot;error&quot;, &quot;error_message&quot;: f&quot;Error during NHTSA API request: {req_err}&quot;}<br>    except ValueError as json_err: # Handles JSON decoding errors<br>        print(f&quot;--- NHTSA API JSON Decode Error: {json_err} ---&quot;)<br>        return {&quot;status&quot;: &quot;error&quot;, &quot;error_message&quot;: f&quot;Error decoding response from NHTSA API: {json_err}&quot;}3. Creating the ADK Agent</pre><p><strong>3. Creating the ADK Agent</strong></p><pre># --- Agent Definition ---<br>root_agent = Agent(<br>   model=&#39;gemini-2.5-flash&#39;,<br>   name=&#39;car_search_agent&#39;,<br>   description=&quot;Agent that finds vehicle complaints using the US National Highway Traffic Safety Administration (NHTSA) API.&quot;,<br>   instruction=&quot;You are a helpful agent. When asked about car complaints, retrieve the information using the get_complaints tool. In your response, state the total number of complaints. Then, summarize the top 3-5 most common types of complaints. If possible, provide a brief, anonymized example summary from one or two of the complaints to illustrate the issues.&quot;,<br>   tools=[get_complaints],<br><br>)</pre><p><strong>4. Local Testing with AdkApp</strong></p><p>Before deploying, it’s wise to test your agent locally. AdkApp allows you to do this.</p><pre>from vertexai.preview import reasoning_engines<br><br>app = reasoning_engines.AdkApp(<br>    agent=root_agent,<br>    enable_tracing=True,<br>)<br><br>session = app.create_session(user_id=&quot;u_123&quot;)<br>session</pre><pre>#Query the Agent <br><br>for event in app.stream_query(<br>    user_id=&quot;u_123&quot;,<br>    session_id=session.id,<br>    message=&quot;What can you tell me about the complaints for the 2021 Honda Accord&quot;,<br>):<br>  print(event)</pre><p><strong>Results/Demo (Simulated Output from stream_query)</strong></p><p>When you run the stream_query code, you’ll see a series of events. The key parts will include:</p><ol><li>The agent deciding to use the get_complaints tool.</li><li>The tool call execution (our print statements inside get_complaints will show up).</li><li>The agent processes the tool’s output and generates a final response.</li></ol><p>A summarized final response might look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pqErvmUeahW5aSkU" /></figure><p><strong>5. Deploying to Vertex AI Agent Engine</strong></p><p>Once you’re satisfied with local testing, you can deploy your ADK agent to Vertex AI Agent Engine. This makes it a scalable, managed endpoint.</p><pre># Deploy the agent to Vertex AI Agent Engine<br>remote_app = agent_engines.create(<br>    agent_engine=root_agent,<br>    requirements=[<br>        &quot;google-cloud-aiplatform[adk,agent_engines]&quot;<br>    ]<br>)</pre><p>This agent_engines.create() call packages your ADK agent, its tools, and dependencies, and deploys it as a Reasoning Engine on Vertex AI. You’ll get a resource name and a link to manage it in the Google Cloud Console.</p><p>Once deployed, you can interact with your production agent using the Vertex AI SDK or REST APIs, similar to how you might query other Vertex AI models or endpoints.</p><h3>Key Benefits of Using Agent Engine</h3><ul><li><strong>Fully Managed Service:</strong> No infrastructure to manage. Google handles scaling, reliability, and maintenance.</li><li><strong>Scalability:</strong> Automatically scales to handle varying loads.</li><li><strong>Monitoring &amp; Logging:</strong> Integrates with Cloud Logging and Cloud Monitoring for operational insights.</li><li><strong>Simplified Deployment:</strong> Streamlines the process of taking agents from development to production.</li><li><strong>Integration with Google Cloud:</strong> Leverages the broader Vertex AI and Google Cloud ecosystem.</li></ul><h3>Outro</h3><p><strong>Key Takeaways<br></strong>In this article, we’ve walked through the process of building a production-ready AI agent using Vertex AI Agent Engine with the Agent Development Kit (ADK). We covered:</p><ul><li>Setting up your GCP environment.</li><li>Defining a custom tool to interact with an external API (NHTSA).</li><li>Creating an ADK agent that uses this tool.</li><li>Testing the agent locally.</li><li>Deploying the agent to Vertex AI Agent Engine for scalable, managed use.</li></ul><p>Vertex AI Agent Engine empowers developers to focus on building intelligent agent logic, while Google Cloud handles the operational complexities of running them in production.</p><p>For examples and sample agents, check this <a href="https://github.com/google/adk-samples">GitHub</a> repository.</p><p><strong>Call to Action/Next Steps<br></strong>Ready to build your own production-grade AI agents?</p><ol><li>Explore the<a href="https://cloud.google.com/vertex-ai/generative-ai/docs/agent-engine"> official Vertex AI Agent Engine documentation</a>.</li><li>Sign Up to attend our <a href="https://rsvp.withgoogle.com/events/google-cloud-workshop-marathon-2025_41441d">Google Cloud Workshop Marathon</a></li><li>Experiment with the Agent Development Kit (ADK) and other supported frameworks.</li><li>Start deploying your prototypes and bring your AI innovations to life!</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ba31dfceb11" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/unlock-production-ready-ai-build-your-agent-with-vertex-ai-agent-engine-adk-4ba31dfceb11">Unlock Production-Ready AI: Build Your Agent with Vertex AI Agent Engine &amp; ADK</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking Therapeutic Innovation with TxGemma: A Hands-On Guide for Healthcare Developers]]></title>
            <link>https://medium.com/google-cloud-for-startups/unlocking-therapeutic-innovation-with-txgemma-a-hands-on-guide-for-healthcare-developer-4400e411f5f8?source=rss----5bec3fa3633b---4</link>
            <guid isPermaLink="false">https://medium.com/p/4400e411f5f8</guid>
            <category><![CDATA[healthai]]></category>
            <category><![CDATA[txgemma]]></category>
            <category><![CDATA[hcl]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[google-ai]]></category>
            <dc:creator><![CDATA[Ada Amanna]]></dc:creator>
            <pubDate>Tue, 13 May 2025 17:14:26 GMT</pubDate>
            <atom:updated>2025-05-13T19:02:15.998Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x34phz9oCY0HQrGvkjM4pA.jpeg" /><figcaption>Created with Google <a href="https://labs.google/fx/tools/image-fx">Imagen</a></figcaption></figure><h3>Introduction</h3><p>The healthcare and life sciences (HCLS) industries are undergoing a significant transformation, driven by advancements in Artificial Intelligence. With incredible foresight, Google is expanding the depth of its foundational models to accelerate research, improve patient outcomes, and streamline complex processes. Health AI Developer Foundation (HAI-DEF), Google’s initiative to <a href="https://developers.google.com/health-ai-developer-foundations">provide AI assets in the HCLS space for developers</a>, is at the forefront of this movement, providing powerful foundational models tailored to the unique context, complex data types, and stringent requirements of the HCLS field. Unlike general-purpose AI, these models possess specialized knowledge crucial for navigating the intricacies of biological and chemical data, as well as the clinical landscape. For HCLS Startups in particular, these domain-specific models offer a tremendous advantage, enabling rapid prototyping, efficient resource utilization, and accelerated innovation in a highly competitive landscape.</p><p>Due to a huge interest in fine tuning <a href="https://cloud.google.com/vertex-ai/generative-ai/docs/models">Gemini models</a> for therapeutic insights, we commenced this blog series to explore models provided through <a href="https://developers.google.com/health-ai-developer-foundations">HAI-DEF</a> and dive into key exploratory steps, demonstrating their practical applications for healthcare developers and researchers. We’ll start with <a href="https://developers.google.com/health-ai-developer-foundations/txgemma">TxGemma</a>, a collection of cutting-edge AI models specifically designed to address critical needs in therapeutic development. While general-purpose large language models (LLMs) have shown impressive capabilities, the complexity and nuances of biological and chemical data necessitate models with specialized knowledge and architectures. TxGemma is built to navigate this intricate landscape, offering targeted solutions for drug discovery and development, allowing startups to achieve breakthroughs and scale their therapeutic research with agility.</p><h3>TxGemma at a Glance</h3><p>TxGemma is a collection of efficient, large language models trained specifically for the therapeutics domain. Built upon the powerful and lightweight Gemma 2 family of open models, TxGemma is engineered to excel in key areas crucial for therapeutic development:</p><ul><li><strong>Therapeutic Property Prediction:</strong> Accurately predicting a wide range of properties for molecules, proteins, and other biological entities.</li><li><strong>Interactive Reasoning:</strong> Engaging in natural language discussions (chat capability) about therapeutic concepts and model predictions.</li><li><strong>Explainability:</strong> Providing insights and mechanistic reasoning behind its predictions, moving beyond a black-box approach.</li><li><strong>Agentic Capability</strong>: Orchestrating complex workflows and integrating external tools through Agentic-Tx.</li></ul><p>What truly sets TxGemma apart from general-purpose LLMs is this specialized focus and grounding in HCLS concepts. While a general model might struggle with the nuances of chemical structures or biological pathways, TxGemma is trained to understand and process this complex information effectively. This allows it to serve as a powerful tool for researchers and developers working to bring new therapies to market.</p><p>The TxGemma suite is available in several parameter sizes, offering flexibility for different computational environments and application requirements. The choice of model size can depend on the specific use case and computational resources available:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MasjkwtNlhSntRYGydXHHg.png" /></figure><p>TxGemma models are fine-tuned on the <a href="https://tdcommons.ai/">Therapeutics Data Commons (TDC)</a>, a curated collection of instruction-tuning datasets covering 66 tasks across the drug discovery and development pipeline. This extensive dataset, with over 15 million data points across different biomedical entities, provides the foundation for TxGemma’s domain-specific expertise. This domain-specific fine-tuning allows TxGemma to synthesize information from diverse sources across the therapeutic development pipeline. Now let’s dive into a couple of popular approaches you can use to try on its incredible abilities.</p><h3>Setting Up TxGemma: A Step-by-Step Guide</h3><p>As a lightweight model suite, TxGemma is designed for flexible execution across various computational environments and hardware. This highlights its accessibility-by-design nature and ease of use, making it incredibly valuable even for resource-constrained applications. Let’s explore these common access routes for TxGemma using tangible case studies.</p><h3>Local Execution</h3><p>For developers who want to run TxGemma locally on their own hardware, you can download your desired variant from <a href="https://huggingface.co/collections/google/health-ai-developer-foundations-hai-def-6744dc060bc19b6cf631bb0f">Hugging face</a>; this is ideal for preliminary, lightweight exploration and well-suited for tasks like predicting specific drug properties for individual molecules.</p><p>Let’s demonstrate how to use TxGemma-Predict (using the 2B parameter version for local execution stability) to predict the solubility of a molecule.</p><p><em>Disclaimer: The article is not a recommended practice for building and deploying production ready applications, only a fun way to explore the Google Cloud Platform and learn. Any opinions in this article are my own and not those of Google.</em></p><pre>#Exploring Predictive Capability with TxGemma-Predict<br><br># Install required packages<br>!pip install -q torch transformers huggingface_hub<br><br># Import libraries<br>import torch<br>from transformers import AutoTokenizer, AutoModelForCausalLM<br><br># Load TxGemma model and tokenizer (2B parameter version for local execution)<br>model_name = &quot;google/txgemma-2b-predict&quot;<br>tokenizer = AutoTokenizer.from_pretrained(model_name)<br>model = AutoModelForCausalLM.from_pretrained(<br>    model_name,<br>    torch_dtype=torch.float16,  # Use half-precision for efficiency<br>    device_map=&quot;auto&quot;  # Automatically use available GPUs or fallback to CPU<br>)<br><br># Create a therapeutic prompt<br>prompt = f&quot;&quot;&quot;<br>Instructions: Predict the solubility class for the following molecule based on its SMILES representation.<br><br>Context: Water solubility affects drug absorption and bioavailability.<br>Question: Given a drug SMILES string, predict the solubility of the molucule.<br><br>Drug SMILES: CC(=O)Oc1ccccc1C(=O)O<br><br>Response:<br>&quot;&quot;&quot;<br><br># Tokenize and generate<br>inputs = tokenizer(prompt, return_tensors=&quot;pt&quot;).to(model.device)<br>outputs = model.generate(<br>    inputs[&quot;input_ids&quot;],<br>    max_new_tokens=512,<br>    temperature=0.3,<br>    do_sample=True<br>)<br>response = tokenizer.decode(outputs[0], skip_special_tokens=True)<br><br>print(f&quot;Prediction Prompt:\n{prompt}&quot;)<br>print(f&quot;\nTxGemma&#39;s Response:\n{response}&quot;)<br></pre><p>This example demonstrates the basic steps to load a TxGemma-Predict model locally and use it for a practical drug property prediction task. You can adapt the prompt and the input SMILES string for other predictive tasks. You can also further fine-tune your chosen variant for your specific task following the steps showcased in the <a href="https://github.com/google-gemini/gemma-cookbook/blob/main/TxGemma/%5BTxGemma%5DFinetune_with_Hugging_Face.ipynb">Official Github Notebook</a>.</p><h3>Vertex Deployment</h3><p>For more complex applications requiring low-latency responses, scalability, and managed infrastructure, you can deploy any of the TxGemma variants (2B, 9B, or 27B parameters, including conversational variants) on the Vertex AI platform. Vertex provisions multiple deployment options to accommodate different customization needs.</p><p>NB: To deploy on Vertex AI, you’ll need a <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects">Google Cloud Project</a> set up and the following <a href="https://cloud.google.com/vertex-ai/docs/general/access-control#predefined-roles">IAM roles</a>: Vertex AI User (roles/aiplatform.user) on a basic level to use resources on vertex, and/or Vertex AI Administrator (roles/aiplatform.admin) for full access to all Vertex resources.</p><p><strong>Vertex AI One Click Deployment</strong></p><p>The most convenient way to deploy any TxGemma variant to a managed endpoint on the Vertex Platform is through its “one-click deploy” option. This allows you to quickly create an endpoint to which you can send both batch and real-time prediction requests. You can easily deploy your preferred variant following these steps below.</p><ol><li>On Google cloud console, navigate to Vertex AI platform.</li><li>On the sidebar click on “Model Garden”.</li><li>Within the Model Garden, search “TxGemma,” and click on it.</li><li>Under “Deployment Options”, select “One-click deployment, Vertex AI”</li><li>On the sidebar, fill out these information:</li></ol><ul><li><strong>Resource ID</strong>: Unique identifier for the deployment workload</li><li><strong>Model name:</strong> your desired Model Variant</li><li><strong>Deployment setting:</strong> Choose between Basic and Advanced configurations.</li><li><strong>Region</strong>: select your desired deployment region</li><li><strong>Machine spec: </strong>Choose appropriate <a href="https://cloud.google.com/vertex-ai/pricing">GPU-accelerated machine type</a></li><li>For Advanced deployment setting, set Availability Policies and vm provisioning model (Standard vs <a href="https://cloud.google.com/compute/docs/instances/spot#:~:text=In%20contrast%20to%20standard%20VMs,the%20Virtual%20machine%20instances%20documentation.">Spot</a>)</li><li>Set Endpoint Access (Public vs Private)</li><li>Then click “<strong>Deploy</strong>”</li></ul><p><strong>Vertex AI Deployment (gcloud CLI or Python)</strong></p><p>You can also deploy TxGemma programmatically using the gcloud CLI or Python with the Vertex AI SDK. This is particularly useful for automating deployments or integrating them into MLOps pipelines.</p><pre>#Install Vertex AI SDK<br>pip install --upgrade google-cloud-aiplatform<br>gcloud auth application-default login<br><br>#Deploy the model (sample execution commmand)<br>import vertexai <br>from vertexai.preview import model_garden<br><br>#initialize project and location<br>vertexai.init(project=&quot;white-vortex-449419-n7&quot;, location=&quot;us-west1&quot;)<br><br>#specify model parameters and deploy<br>model = model_garden.OpenModel(&quot;google/txgemma@txgemma-27b-chat&quot;)<br>endpoint = model.deploy(<br>  accept_eula=True,<br>  machine_type=&quot;a3-highgpu-2g&quot;,<br>  accelerator_type=&quot;NVIDIA_H100_80GB&quot;,<br>  accelerator_count=2,  serving_container_image_uri=&quot;us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:20250114_0916_RC00_maas&quot;,<br>  endpoint_display_name=&quot;google_txgemma-27b-chat-mg-one-click-deploy&quot;,<br>  model_display_name=&quot;google_txgemma-27b-chat-1745431158561&quot;,<br>)</pre><pre>#Authenticate your Google Account<br>gcloud auth login<br><br>#Specify model Parameters and deploy<br>gcloud beta ai model-garden models deploy \<br>--model=&quot;google/txgemma@txgemma-27b-chat&quot; \<br>--region=&quot;us-west1&quot; \<br>--project=&quot;white-vortex-449419-n7&quot; \<br>--accept-eula \<br>--machine-type=&quot;a3-highgpu-2g&quot; \<br>--accelerator-type=&quot;NVIDIA_H100_80GB&quot; \<br>--container-image-uri=&quot;us-docker.pkg.dev/vertex-ai/vertex-vision-model-garden-dockers/pytorch-vllm-serve:20250114_0916_RC00_maas&quot; \<br>--endpoint-display-name=&quot;google_txgemma-27b-chat-mg-one-click-deploy&quot;</pre><p>Deployment can take a few minutes to finalize. Afterward, you can find your endpoint listed on the <a href="https://console.cloud.google.com/vertex-ai/online-prediction?project=white-vortex-449419-n7">Online prediction page</a>. With that, you can send both batch and real-time prediction requests to your endpoint. Note that there are <a href="https://cloud.google.com/endpoints/pricing">costs</a> associated with running endpoints on Vertex AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RZ1Qz4rhWDKWRMID" /></figure><p><strong>Custom Deployment on Google Kubernetes Engine</strong></p><p>For workloads requiring extensive control, custom configurations, or integration with existing Kubernetes-based infrastructure, you can deploy TxGemma as a containerized AI/ML workload on a GKE cluster. This method is suitable for complex application requirements and also supports online and batch inference requests.</p><p>Requirements:</p><p>* A Google Cloud <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects"><strong>Project</strong></a> already set up.</p><p>* A Hugging Face account to obtain an access token for model access within your GKE cluster.</p><p>Steps for deployment via the Google Cloud console:</p><ol><li>On Google cloud console, navigate to Vertex AI platform.</li><li>On the sidebar click on Model Garden.</li><li>Within the Model Garden, search “TxGemma,” and click on it.</li><li>Under “Deployment Options”, select “Custom Deployment — Google Kubernetes”</li><li>On the sidebar, set resource ID and click “Configure in Kubernetes Engine” to begin configuration.</li><li>In the GKE console, you may be prompted to enable necessary APIs. Click “enable.”</li><li>GKE will be pre-selected as the deployment environment, click “create Autopilot cluster” to create an <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/creating-an-autopilot-cluster?_gl=1*1y94z81*_ga*MTE5OTg0NzAzNC4xNzI5MTAwOTk1*_ga_WH2QY8WWF5*MTc0NTQzMjUxMy4zNS4xLjE3NDU0MzI5ODguNjAuMC4w">Autopilot Cluster</a> preconfigured for best practices. This may take a few minutes.</li><li>Once created, repeat step (iv) to spin up the customization window.</li><li>Enter configuration information including the huggingface access token, and click “Deploy”.</li></ol><p>Once your model is deployed to the GKE cluster, you have fine-grained control over scaling, resource allocation, and integration within your Kubernetes environment to serve both real-time and Batch workloads. You would then interact with the model running on your GKE cluster via the service endpoints you configure, similar to the Vertex AI endpoint interaction but managed within your Kubernetes environment. Note that there are <a href="https://cloud.google.com/kubernetes-engine/pricing">costs</a> associated with running GKE clusters.</p><h3>Optimizing TxGemma Performance</h3><p>Achieving optimal performance with TxGemma involves understanding how to effectively communicate your task through prompt engineering and leveraging techniques like few-shot learning.</p><ul><li><strong>Prompt Engineering Strategies:</strong> TxGemma is trained on instruction-tuning datasets formatted with clear instructions, context, questions, and expected answers. For optimal results, structure your prompts to align with this format whenever possible. Be clear and specific in your instructions and questions. Providing relevant context helps the model understand the domain and the specific problem you want to solve.</li><li><strong>Few-Shot Learning:</strong> Including a few examples of input-output pairs directly in your prompt can significantly improve TxGemma’s performance, especially for tasks where the desired output format or style is nuanced. This allows the model to learn from in-context examples without requiring additional fine-tuning.</li><li><strong>Performance Considerations and Tuning:</strong> When deploying TxGemma on Vertex AI, selecting the appropriate machine type and accelerator is crucial for performance, particularly for the larger 27B model. Utilizing GPUs (like NVIDIA H100) is highly recommended for faster inference. For high-throughput applications, consider batching your prediction requests to improve efficiency and reduce latency.</li></ul><h3>Conclusion and Future Directions</h3><p>TxGemma represents a significant step forward in applying AI to therapeutic development, it opens up numerous avenues for future research exploration as it shows incredible promise for improving explainability and interpretability of complex predictions. Its generalist capabilities, conversational variants, and agentic features set it apart, making it a valuable tool for researchers and developers in the HCLS field.</p><p>We’ve discussed TxGemma’s technical foundation, and multiple exploration paths to try out its immense capability. Although, incredibly capable, this is the least equipped it will ever be to tackle your questions. It will keep getting better as we continue to refine its features. In our next blog post, we’ll delve into <strong>HeAR</strong>, another ground-breaking model in the Google HCLS suite, and explore how it is transforming the way we interact with audio healthcare data, especially in the aftermath of global respiratory pandemics like COVID-19.</p><h3>Additional Documentations</h3><p><a href="https://arxiv.org/abs/2504.06196">TxGemma: Efficient and Agentic LLMs for Therapeutics</a></p><p><a href="https://developers.google.com/health-ai-developer-foundations">HCLS Domain-specific Foundational Models</a></p><p><a href="https://cloud.google.com/kubernetes-engine/docs/how-to/creating-an-autopilot-cluster?_gl=1*1y94z81*_ga*MTE5OTg0NzAzNC4xNzI5MTAwOTk1*_ga_WH2QY8WWF5*MTc0NTQzMjUxMy4zNS4xLjE3NDU0MzI5ODguNjAuMC4w">Creating an Autopilot Cluster</a></p><p><a href="https://cloud.google.com/kubernetes-engine/pricing">GKE Autopilot Pricing</a></p><p><a href="https://cloud.google.com/vertex-ai/pricing">Vertex AI pricing</a></p><p><a href="https://tdcommons.ai/">Therapeutics Data Commons (TDC)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4400e411f5f8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud-for-startups/unlocking-therapeutic-innovation-with-txgemma-a-hands-on-guide-for-healthcare-developer-4400e411f5f8">Unlocking Therapeutic Innovation with TxGemma: A Hands-On Guide for Healthcare Developers</a> was originally published in <a href="https://medium.com/google-cloud-for-startups">Google Cloud for Startups</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>