<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Palakmundra on Medium]]></title>
        <description><![CDATA[Stories by Palakmundra on Medium]]></description>
        <link>https://medium.com/@palakmundra95?source=rss-9265b470b1c5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZuBB5aMkbSe138OV6lSy1g.jpeg</url>
            <title>Stories by Palakmundra on Medium</title>
            <link>https://medium.com/@palakmundra95?source=rss-9265b470b1c5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:43:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@palakmundra95/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Grounding Gemini: One-Time BigQuery Ingestion with Terraform]]></title>
            <link>https://medium.com/google-cloud/grounding-gemini-one-time-bigquery-ingestion-with-terraform-913f1ea49f02?source=rss-9265b470b1c5------2</link>
            <guid isPermaLink="false">https://medium.com/p/913f1ea49f02</guid>
            <category><![CDATA[google-gemini]]></category>
            <category><![CDATA[bigquery]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[gemini-enterprise]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <dc:creator><![CDATA[Palakmundra]]></dc:creator>
            <pubDate>Wed, 01 Jul 2026 04:25:37 GMT</pubDate>
            <atom:updated>2026-07-01T04:25:37.273Z</atom:updated>
            <content:encoded><![CDATA[<p>This guide covers the Terraform configuration for <a href="https://docs.cloud.google.com/gemini/enterprise/docs/connectors/connect-bigquery">One-Time Ingestion</a> to ground a Gemini Enterprise Agent in BigQuery data.</p><p>For details regarding <strong>Periodic Ingestion (Automated Sync)</strong>, please refer to the guide available <a href="https://medium.com/@palakmundra95/bd9d159ee879">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*Na_02b0czNTOfogVMoRZ9w.png" /></figure><h3>When to Use One-Time Ingestion</h3><p>One-Time Ingestion is ideal when:</p><ul><li>Your source data in BigQuery is static or rarely changes.</li><li>You want to control exactly when data is refreshed (on-demand) rather than relying on an automated schedule.</li><li>You are testing the integration and want the data to be available immediately without waiting for the 1-hour initial sync delay associated with managed BigQuery Data Connectors.</li></ul><h3>Prerequisites</h3><p>Before getting started, make sure you have:</p><p><strong>1. Google Cloud Project &amp; APIs</strong></p><ul><li>You must have an active Google Cloud project with the following APIs enabled:</li><li><strong>Discovery Engine API</strong> (discoveryengine.googleapis.com): Powers the core search indexing, data store capabilities, and conversational infrastructure required for Gemini Enterprise.</li><li><strong>BigQuery API</strong> (bigquery.googleapis.com): Enables Discovery Engine connectors to query, read, and sync tabular data from your BigQuery datasets.</li></ul><p><strong>2. Licensing &amp; Tooling</strong></p><ul><li><strong>Gemini Enterprise Subscription</strong>: Your Google Cloud project or organization must have an active, purchased <a href="https://docs.cloud.google.com/gemini/enterprise/docs/licenses"><strong>Gemini Enterprise</strong> <strong>subscription tier.</strong></a></li><li>Your Google Cloud Organization ID</li><li>Terraform v1.3+ installed</li></ul><p><strong>3. IAM Permissions</strong></p><p>Your identity (or the service account running Terraform) will need:</p><p><em>Project level:</em></p><ul><li>Discovery Engine Admin (roles/discoveryengine.admin)</li><li>Project IAM Admin (roles/resourcemanager.projectIamAdmin)</li><li>BigQuery Data Viewer (roles/bigquery.dataViewer) access to the target BigQuery dataset and table.</li></ul><p><em>Cross-Project BigQuery Ingestion (If Applicable):</em></p><ul><li>If you intend to import data from a source Google Cloud project that is different from the project hosting your Gemini Enterprise data store, you must configure cross-project permissions.</li><li>Locate the Discovery Engine service account inside the gemini enterprise data store project:</li></ul><pre>service-PROJECT_NUMBER@gcp-sa-discoveryengine.iam.gserviceaccount.com</pre><ul><li>Grant this service account the following IAM roles in the external project containing your BigQuery data source (source google cloud project):<br>BigQuery Job User (roles/bigquery.jobUser)<br>BigQuery Data Editor (roles/bigquery.dataEditor)</li></ul><h3>Terraform Configuration</h3><p>We divide the configuration into three files: <strong>variables.tf</strong>, <strong>provider.tf</strong>, and <strong>main.tf</strong>.</p><h4>variables.tf</h4><p>To keep our configuration reusable, we define variables for project-specific details.</p><pre>variable &quot;project_id&quot; {<br> type        = string<br> description = &quot;The Google Cloud Project ID.&quot;<br>}<br><br>variable &quot;location&quot; {<br> type        = string<br> default     = &quot;global&quot;<br> description = &quot;The region for the Discovery Engine (e.g., global, us).&quot;<br>}<br><br>variable &quot;datastore_id&quot; {<br> type        = string<br> description = &quot;The ID of the Discovery Engine Data Store.&quot;<br>}<br><br>variable &quot;bq_dataset_id&quot; {<br> type        = string<br> description = &quot;The BigQuery dataset ID.&quot;<br>}<br><br>variable &quot;bq_table_id&quot; {<br> type        = string<br> description = &quot;The BigQuery table ID.&quot;<br>}<br><br>variable &quot;engine_id&quot; {<br> type        = string<br> description = &quot;The ID of the Gemini Search Engine.&quot;<br>}</pre><h4>provider.tf</h4><p>We configure the Google providers. Note that some features of Gemini Enterprise often require the `google-beta` provider.</p><pre>terraform {<br>  required_version = &quot;&gt;= 1.0&quot;<br>  required_providers {<br>    google-beta = {<br>      source  = &quot;hashicorp/google-beta&quot;<br>      version = &quot;&gt;= 5.0.0&quot;<br>    }<br>  }<br>}<br><br>provider &quot;google-beta&quot; {<br>  project               = var.project_id<br>  region                = var.region<br>  user_project_override = true<br>  billing_project       = var.project_id<br>}</pre><h3>main.tf</h3><p>Here, we declare our search infrastructure, trigger the immediate data load, and layer the Gemini Enterprise configuration over it. Since this flow is for One-Time Ingestion, we use a standalone `datastore` resource rather than a continuous connector resource. To populate it, we leverage a Terraform provisioner to trigger the ingestion API immediately during deployment. This ensures your search store is populated and ready for queries as soon as the Terraform apply finishes.</p><p><em>Step 1: Create the Data Store &amp; Trigger Ingestion</em></p><ul><li>We create a standalone google_discovery_engine_data_store.</li><li>To trigger the one-time import, we use a terraform_data resource with a local-exec provisioner. This provisioner executes a curl command to call the Discovery Engine API immediately upon deployment, importing your BigQuery data.</li></ul><p><strong>Note</strong>: This is a one-time snapshot, it will only re-trigger if you change the target BigQuery table or dataset in your Terraform configuration, not when the data inside the table changes.</p><p><em>Step 2: Create the Gemini Search Agent</em></p><ul><li>Next, we create the google_discovery_engine_search_engine.</li><li>We set the subscription tier to SUBSCRIPTION_TIER_SEARCH_AND_ASSISTANT to enable advanced Gemini features.</li><li>We link it to the standalone Data Store created in Step 1.</li><li>Crucially, we use depends_on to ensure the search engine is only provisioned after the initial data import has been successfully triggered.</li></ul><p><em>Step 3: Secure the Agent</em></p><p>Finally, we configure the Access Control (ACL) to use Google Workspace (GSUITE) for authentication.</p><pre># Access token datasource for the API call<br>data &quot;google_client_config&quot; &quot;default&quot; {}<br><br># 1. Standalone Data Store<br>resource &quot;google_discovery_engine_data_store&quot; &quot;bq_data_store&quot; {<br> provider              = google-beta<br> project               = var.project_id<br> location              = var.location<br> data_store_id         = var.datastore_id<br> display_name          = &quot;BigQuery One-Time Data Store&quot;<br> industry_vertical     = &quot;GENERIC&quot;<br> content_config        = &quot;NO_CONTENT&quot;<br> solution_types        = [&quot;SOLUTION_TYPE_SEARCH&quot;]<br><br> lifecycle {<br>   ignore_changes = [document_processing_config]<br> }<br>}<br><br># 2. API Trigger for immediate import<br>resource &quot;terraform_data&quot; &quot;trigger_import&quot; {<br> triggers_replace = {<br>   table_id          = var.bq_table_id<br>   dataset_id        = var.bq_dataset_id<br>   datastore_id      = google_discovery_engine_data_store.bq_data_store.data_store_id<br>   datastore_created = google_discovery_engine_data_store.bq_data_store.create_time<br> }<br><br> provisioner &quot;local-exec&quot; {<br>   command = &lt;&lt;EOT<br>     curl -f -X POST \<br>       -H &quot;Authorization: Bearer ${data.google_client_config.default.access_token}&quot; \<br>       -H &quot;Content-Type: application/json&quot; \<br>       -H &quot;X-Goog-User-Project: ${var.project_id}&quot; \<br>       &quot;https://discoveryengine.googleapis.com/v1beta/projects/${var.project_id}/locations/${var.region}/collections/default_collection/dataStores/${google_discovery_engine_data_store.bq_data_store.data_store_id}/branches/default_branch/documents:import&quot; \<br>       -d &#39;{<br>         &quot;bigquerySource&quot;: {<br>           &quot;projectId&quot;: &quot;${var.project_id}&quot;,<br>           &quot;datasetId&quot;: &quot;${var.bq_dataset_id}&quot;,<br>           &quot;tableId&quot;: &quot;${var.bq_table_id}&quot;,<br>           &quot;dataSchema&quot;: &quot;custom&quot;<br>         },<br>         &quot;reconciliationMode&quot;: &quot;INCREMENTAL&quot;,<br>         &quot;autoGenerateIds&quot;: true<br>       }&#39;<br>   EOT<br> }<br>}<br><br># 3. Create the Gemini Search Agent<br>resource &quot;google_discovery_engine_search_engine&quot; &quot;bq_search_agent&quot; {<br> provider      = google-beta<br> project       = var.project_id<br> engine_id     = var.engine_id<br> collection_id = &quot;default_collection&quot;<br> location      = var.region<br> display_name  = &quot;Gemini BigQuery Search Agent One-Time&quot;<br> app_type      = &quot;APP_TYPE_INTRANET&quot;<br>  # Reference the standalone data store directly<br> data_store_ids = [google_discovery_engine_data_store.bq_data_store.data_store_id]<br>  search_engine_config {<br>   search_tier                = &quot;SEARCH_TIER_ENTERPRISE&quot;<br>   search_add_ons             = [&quot;SEARCH_ADD_ON_LLM&quot;]<br>   required_subscription_tier = &quot;SUBSCRIPTION_TIER_SEARCH_AND_ASSISTANT&quot;<br> }<br><br> # Ensure the import starts before the search engine points to the store<br> depends_on = [<br>   terraform_data.trigger_import<br> ]<br><br> lifecycle {<br>   replace_triggered_by = [<br>     google_discovery_engine_data_store.bq_data_store<br>   ]<br> }<br>}<br><br># 4. Secure the Agent (ACL Config)<br>resource &quot;google_discovery_engine_acl_config&quot; &quot;identity_provider&quot; {<br> provider = google-beta<br> project  = var.project_id<br> location = var.region<br> idp_config {<br>   idp_type = &quot;GSUITE&quot;<br> }<br>}</pre><h3>Deployment and Verification</h3><h4>Deploying the configuration:</h4><ol><li>Initialize the working directory to pull down the required Google Beta binaries:</li></ol><pre>terraform init</pre><p>2. Execute a dry-run verification preview to validate your HCL block syntax against your live GCP environment:</p><pre>terraform plan</pre><p>Once your plan step cleanly compiles and outputs `Plan: 3 to add, 0 to change, 0 to destroy`, your code logic is completely pristine.</p><p>3. Apply the configuration to provision the backend engines:</p><pre>terraform apply</pre><h4>Verify Ingestion</h4><p>The import is triggered immediately during terraform apply. You can monitor the import job status in the Google Cloud Console under<strong> Agent Builder &gt; Data Stores &gt; [Your Data Store] &gt; Activity</strong></p><h4>Test the Agent</h4><p>Once the import completes, go to <strong>Gemini Enterprise &gt; Apps</strong>, select your app, and use the preview link to test the search interface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sCCU9GTWvAOXw5z9Aexc8Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VdnbaK2A6yr0su__Y6RZmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U6RUjnyl05-z-ZY8lWZu0Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fmzv3G2nMdgd_BjdRRC1cw.png" /></figure><h3>Key Considerations and Final Thoughts</h3><ul><li>This flow is designed for <strong>static data</strong>. Terraform triggers the import once during deployment. It <strong>will not</strong> automatically sync new rows or updates added to your BigQuery table later.</li><li>If your BigQuery data changes and you need to force a refresh of the search index, you can tell Terraform to re-run the import command by replacing the trigger resource:</li></ul><pre>terraform apply -replace=&quot;terraform_data.trigger_import&quot;</pre><ul><li>Use this one-time flow for archives, documentation, or static datasets. If your data changes frequently (e.g., daily updates), you should use the Periodic Ingestion flow instead, which handles automatic syncing on a schedule.</li></ul><p>Happy building! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=913f1ea49f02" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/grounding-gemini-one-time-bigquery-ingestion-with-terraform-913f1ea49f02">Grounding Gemini: One-Time BigQuery Ingestion with Terraform</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gemini Enterprise: Building a Periodic BigQuery Search Agent with Terraform]]></title>
            <link>https://medium.com/google-cloud/gemini-enterprise-building-a-periodic-bigquery-search-agent-with-terraform-bd9d159ee879?source=rss-9265b470b1c5------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd9d159ee879</guid>
            <category><![CDATA[gemini-enterprise]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[bigquery]]></category>
            <category><![CDATA[gemini]]></category>
            <dc:creator><![CDATA[Palakmundra]]></dc:creator>
            <pubDate>Tue, 30 Jun 2026 05:31:45 GMT</pubDate>
            <atom:updated>2026-06-30T05:31:45.499Z</atom:updated>
            <content:encoded><![CDATA[<p>In the era of Generative AI, Large Language Models are only as smart as the enterprise data you give them.</p><p>If your data lives in Google Cloud’s BigQuery, building a natural language search interface powered by <a href="https://cloud.google.com/gemini-enterprise">Gemini Enterprise</a> is a great next step to unlock Agentic capabilities with your organization’s data.</p><p>Translating raw tabular data into an intelligent chat interface isn’t just about writing SQL queries. It requires mapping your database infrastructure to an optimized search index, connecting it to Gemini Enterprise, and avoiding a few hidden API bottlenecks along the way.</p><p>Below demonstrates the Terraform required to deploy and automate a Gemini Enterprise grounded on BigQuery, along with the architectural deep dives and “gotchas” that you can potentially run into.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*2GmXsxRZru66rOTKolBdyA.png" /></figure><h3>The Architectural Components</h3><ul><li><strong>User Interface</strong>: A Gemini Enterprise Application that provides you an interface to submit your prompts.</li><li><strong>Original Data Source</strong>: Your structured data in a BigQuery Table.</li><li><strong>Data Connector</strong>: A resource that links BigQuery to Gemini Enterprise, automatically creating and updating an Entity Data Store.</li></ul><h3>BigQuery Ingestion Modes: One-Time vs. Periodic</h3><p>When grounding Gemini Enterprise in BigQuery, you can choose between two primary <a href="https://docs.cloud.google.com/gemini/enterprise/docs/connectors/connect-bigquery">ingestion strategies</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*UeU87ZxXWtD792V-HjUBCQ.png" /></figure><p>This guide focuses on <strong>Periodic Ingestion</strong> using a Data Connector, which is recommended for data that updates regularly.</p><p>If your data is static, or you prefer to trigger imports manually on-demand, you should use <strong>One-Time Ingestion</strong>. The architecture, Terraform resources, and the required API-trigger workaround for one-time imports has been documented in a separate post.</p><h3>Prerequisites</h3><p>Before getting started, make sure you have:</p><p><strong>1. Google Cloud Project &amp; APIs</strong></p><ul><li>You must have an active Google Cloud project with the following APIs enabled:</li><li><strong>Discovery Engine API</strong> (discoveryengine.googleapis.com): Powers the core search indexing, data store capabilities, and conversational infrastructure required for Gemini Enterprise.</li><li><strong>BigQuery API</strong> (bigquery.googleapis.com): Enables Discovery Engine connectors to query, read, and sync tabular data from your BigQuery datasets.</li></ul><p><strong>2. Licensing &amp; Tooling</strong></p><ul><li><strong>Gemini Enterprise Subscription</strong>: Your Google Cloud project or organization must have an active, purchased <a href="https://docs.cloud.google.com/gemini/enterprise/docs/licenses"><strong>Gemini Enterprise</strong> <strong>subscription tier.</strong></a></li><li>Your Google Cloud Organization ID</li><li>Terraform v1.3+ installed</li></ul><p><strong>3. IAM Permissions</strong></p><p>Your identity (or the service account running Terraform) will need:</p><p><em>Project level:</em></p><ul><li>Discovery Engine Admin (roles/discoveryengine.admin)</li><li>Project IAM Admin (roles/resourcemanager.projectIamAdmin)</li><li>BigQuery Data Viewer (roles/bigquery.dataViewer) access to the target BigQuery dataset and table.</li></ul><p><em>Cross-Project BigQuery Ingestion (If Applicable):</em></p><ul><li>If you intend to import data from a source Google Cloud project that is different from the project hosting your Gemini Enterprise data store, you must configure cross-project permissions.</li><li>Locate the Discovery Engine service account inside the gemini enterprise data store project:</li></ul><pre>service-PROJECT_NUMBER@gcp-sa-discoveryengine.iam.gserviceaccount.com</pre><ul><li>Grant this service account the following IAM roles in the external project containing your BigQuery data source (source google cloud project):<br>BigQuery Job User (roles/bigquery.jobUser)<br>BigQuery Data Editor (roles/bigquery.dataEditor)</li></ul><h3>The Terraform Structure</h3><p>We divide our configuration into three standard files: <strong>variables.tf</strong> (inputs), <strong>provider.tf</strong> (connections), and <strong>main.tf</strong> (resources).</p><h4>variables.tf</h4><p>To keep our configuration reusable, we define variables for project-specific details.</p><pre>variable &quot;project_id&quot; {<br>  type        = string<br>}<br><br>variable &quot;location&quot; {<br>  type        = string<br>}<br><br>variable &quot;datastore_id&quot; {<br>  type        = string<br>}<br><br>variable &quot;bq_dataset_id&quot; {<br>  type        = string<br>}<br><br>variable &quot;bq_table_id&quot; {<br>  type        = string<br>}<br><br>variable &quot;engine_id&quot; {<br>  type        = string<br>}</pre><h4>provider.tf</h4><p>We configure the Google providers. Note that some features of Gemini Enterprise often require the `google-beta` provider.</p><pre>terraform {<br>  required_version = &quot;&gt;= 1.0&quot;<br>  required_providers {<br>    google-beta = {<br>      source  = &quot;hashicorp/google-beta&quot;<br>      version = &quot;&gt;= 5.0.0&quot;<br>    }<br>  }<br>}<br><br>provider &quot;google-beta&quot; {<br>  project               = var.project_id<br>  region                = var.region<br>  user_project_override = true<br>  billing_project       = var.project_id<br>}</pre><h4>main.tf</h4><p>Here, we declare our search infrastructure, enforce structured indexing, and layer the Gemini chat configuration over it. Since the Terraform code uses a Data Connector with a refresh_interval , it leverages Periodic Ingestion. You never need to manually trigger syncs. Gemini Enterprise automatically initiates the first sync about an hour post-deployment and keeps your search store fresh per your `refresh_interval` configuration.</p><p><em>Step 1: Link BigQuery to Agent Search</em></p><ul><li>We create a <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/discovery_engine_data_connector">google_discovery_engine_data_connector</a>. This automatically provisions the underlying Data Store and sets up a sync schedule.</li></ul><p><em>Step 2: Create the Gemini Search Agent</em></p><ul><li>Next, we create the <a href="https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/discovery_engine_search_engine">google_discovery_engine_search_engine</a>.</li><li>We set the subscription tier to `SUBSCRIPTION_TIER_SEARCH_AND_ASSISTANT` to enable advanced Gemini features.</li><li>We dynamically link it to the Data Store ID generated by the connector in Step 1.</li></ul><p><em>Step 3: Secure the Agent</em></p><ul><li>Finally, we configure the Access Control (ACL) to use Google Workspace (`GSUITE`) for authentication.</li></ul><pre>resource &quot;google_discovery_engine_data_connector&quot; &quot;bq_link&quot; {<br>  provider                = google-beta<br>  project                 = var.project_id<br>  location                = var.location<br>  collection_id           = &quot;bq_connector_col_3&quot; #You can customize this string value <br>  collection_display_name = &quot;BQ Connector Collection&quot; #You can customize this string value <br>  data_source             = &quot;bigquery&quot;<br>  refresh_interval        = &quot;86400s&quot; #Syncs once every 24 hours<br><br>  params = {<br>    instance_uri = &quot;bigquery://${var.project_id}.${var.bq_dataset_id}&quot;<br>  }<br><br>  entities {<br>    entity_name = var.bq_table_id<br>  }<br>}<br><br>resource &quot;google_discovery_engine_search_engine&quot; &quot;bq_search_agent&quot; {<br>  provider      = google-beta<br>  project       = var.project_id<br>  engine_id     = var.engine_id<br>  collection_id = &quot;default_collection&quot; #Set default value<br>  location      = var.region<br>  display_name  = &quot;Gemini BigQuery Search Agent&quot;<br>  app_type      = &quot;APP_TYPE_INTRANET&quot;<br>  <br>  # Dynamically reference the data store ID automatically generated by the connector<br>  data_store_ids = [element(split(&quot;/&quot;, google_discovery_engine_data_connector.bq_link.entities[0].data_store), 7)]<br>  <br>  search_engine_config {<br>    search_tier    = &quot;SEARCH_TIER_ENTERPRISE&quot;<br>    search_add_ons = [&quot;SEARCH_ADD_ON_LLM&quot;]<br>    required_subscription_tier = &quot;SUBSCRIPTION_TIER_SEARCH_AND_ASSISTANT&quot;<br>  }<br>}<br><br>resource &quot;google_discovery_engine_acl_config&quot; &quot;identity_provider&quot; {<br>  provider = google-beta<br>  project  = var.project_id<br>  location = var.region<br>  idp_config {<br>    idp_type = &quot;GSUITE&quot;<br>  }<br>}</pre><p><strong>Note</strong>: <strong><em>Bypassing the Ingestion Control Plane Paradox</em></strong></p><p>When using <strong>google_discovery_engine_data_connector</strong> in a standard Terraform workflow, deploying to the <strong>default_collection</strong> will trigger an <strong>Error 409: Conflict </strong>because Google Cloud automatically pre-provisions a hidden data connector there. To bypass this without manual state intervention, you must isolate your pipeline by defining a custom collection ID in your configuration. Because this decoupled architecture separates your data connector from the primary search application, you must bridge the collection boundary by programmatically extracting the data store ID using the following snippet:</p><pre>data_store_ids = [element(split(&quot;/&quot;, google_discovery_engine_data_connector.&lt;DATA_CONNECTOR_NAME&gt;.entities[0].data_store), 7)]</pre><p>This expression parses the data connector’s output path to extract the raw, auto-generated Data Store ID. Passing this ID directly to your search app bridges the custom collection gap, allowing Gemini to securely index your BigQuery rows.</p><h3>Deployment &amp; Verification</h3><p>Deploying this configuration follows the standard Terraform lifecycle:</p><p>1. Initialize the working directory to pull down the required Google Beta binaries:</p><pre>terraform init</pre><p>2. Execute a dry-run verification preview to validate your HCL block syntax against your live GCP environment:</p><pre>terraform plan</pre><p>Once your plan step cleanly compiles and outputs `Plan: 3 to add, 0 to change, 0 to destroy`, your code logic is completely pristine.</p><p>3. Apply the configuration to provision the backend engines:</p><pre>terraform apply</pre><h4>Populating Data for Live Testing</h4><p>To test the end-to-end user experience, ensure your target BigQuery table is populated with structured metadata rows. You can easily run an initial data import via the Cloud Console using the Auto-detect Schema option to seed the table.</p><p>Once the data store and the connector are deployed, head to the Google Cloud Console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ygjN-GCouhPtWKE7hNid-w.png" /></figure><p>Under the Data Stores tab, click your newly automated data store, run a synchronization pipeline against your BigQuery table , and you can begin testing natural language queries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DL8haX6vwsabdmNeJTVGaQ.png" /></figure><h4>Testing the connector</h4><p>Now your BigQuery connected data store is fully provisioned. To access the live interface, return to the Gemini Enterprise app within the Google Cloud console and access your Gemini Enterprise application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y3ntK_kiJJfKx2XY0PXAsg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p9bj58S0uUeJeJoLGjAzqw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1001/1*cNyBtlJk7MAjVflWiC5yWA.png" /></figure><h3>Final Thoughts</h3><p>Deploying a Gemini Enterprise search interface over your structured corporate data doesn’t require a compromise between automated scheduling and deployment stability. By leveraging the right resources and explicitly aligning your configuration strings to mirror the platform’s native defaults, you can establish an automated, recurring data sync pipeline all in Terraform. This architecture ensures that your conversations with Gemini Enterprise deliver answers rooted in data you control.</p><p>Now that your enterprise data is cleanly grounded and your pipeline is fully automated, you have a repeatable, secure foundation to scale your organization’s AI initiatives.</p><p>Happy building! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd9d159ee879" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/gemini-enterprise-building-a-periodic-bigquery-search-agent-with-terraform-bd9d159ee879">Gemini Enterprise: Building a Periodic BigQuery Search Agent with Terraform</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving the One-to-Many Challenge: Multi-VPC Peering for NetApp DR]]></title>
            <link>https://medium.com/google-cloud/solving-the-one-to-many-challenge-multi-vpc-peering-for-netapp-dr-bf987609ea90?source=rss-9265b470b1c5------2</link>
            <guid isPermaLink="false">https://medium.com/p/bf987609ea90</guid>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[disaster-recovery]]></category>
            <category><![CDATA[vpc-peering]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[netapp]]></category>
            <dc:creator><![CDATA[Palakmundra]]></dc:creator>
            <pubDate>Wed, 11 Mar 2026 03:18:51 GMT</pubDate>
            <atom:updated>2026-03-11T03:18:51.381Z</atom:updated>
            <content:encoded><![CDATA[<p>Designing a robust High Availability (HA) and Disaster Recovery (DR) strategy is a cornerstone of cloud architecture. For organizations utilizing Google Cloud NetApp Volumes (GCNV), a common requirement involves replicating production data from a primary region (e.g., US-East) to a secondary region (e.g., US-Central).</p><p>However, complexity arises when the recovery architecture requires segmentation into two distinct environments: a non-production standby VPC for failover testing and a separate, dedicated DR VPC for actual disaster scenarios. This post explores a solution for achieving this topology using <strong>Multi-VPC Transitive Peering</strong>.</p><h3>The Challenge: One-to-Many Replication Limits</h3><p>The primary technical hurdle in this scenario is the requirement to replicate data from a single production volume to two separate destinations simultaneously. The ideal architecture demands parallel replication streams: one for the failover/standby VPC and another for the dedicated DR VPC.</p><p>However, Google Cloud NetApp Volumes does not natively support a one-to-many replication topology from a single source volume. While backend configurations for this might technically exist, they are not visible in the Cloud Console, API, or gcloud commands, making them unmanageable for the user.</p><h3>The Solution: Single Replication with Multi-VPC Peering</h3><p>To overcome this limitation without duplicating storage costs, the recommended design leverages <strong>Single Volume Replication combined with Service-Led Peering</strong>.</p><p>Instead of attempting two replication streams, the architecture replicates the primary NetApp volume to a <em>single</em> secondary GCNV volume in the recovery region. Both the non-production/standby VPC and the DR VPC then peer to this same central storage volume. This ensures that only one replication stream is actively maintained while serving multiple environments.</p><h4>High-Level Architecture</h4><ol><li><strong>Primary Region:</strong> Hosts the production NetApp volume.</li><li><strong>Secondary Region (Storage):</strong> Hosts the replicated GCNV volume in a tenant project.</li><li><strong>Secondary Region (Compute):</strong> Contains two separate VPCs (Standby and DR) that utilize transitive peering to access the single replicated volume.</li></ol><p>This approach supports cross-project replication, allowing the DR and non-prod VPCs to reside in different projects from the source if necessary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ERTPFyIisdSZLDxUvCzviA.png" /><figcaption>High level Architecture</figcaption></figure><h3>Implementation Strategy</h3><p>Implementing this solution requires a specific “handshake” process because establishing additional VPC peering connections (beyond the initial Private Services Access) is a support-led process.</p><h4>Step 1: Preparation</h4><p>Before initiating requests, gather the Project IDs and VPC network names for all participating projects. Crucially, ensure that the IP ranges of all VPCs — including the psaRange for NetApp Volumes — do not overlap to prevent peering failures.</p><h4>Step 2: The Peering Request</h4><p>You must initiate a peering request from your new VPC (e.g., the DR VPC) to the NetApp Volumes tenant project.</p><p>At this stage, the request state will remain <strong>INACTIVE</strong> until the handshake is completed by Google.</p><h4>Step 3: Support-Led Handshake</h4><p>Because there is currently no self-configurable option for multi-VPC peering, you must submit a support case. The request should be titled “Request for Transitive Peering Setup in Google Cloud NetApp Volumes” and include the details gathered in Step 1. The Google/NetApp SRE team will then accept the incoming peering requests on the tenant project to finalize the connection.</p><h4>Step 4: Routing and Security</h4><p>Once peering is established, the network connection is static. However, the customer is responsible for the “last mile” configuration. This includes updating routing tables and modifying firewall rules to ensure traffic flows correctly between the VPCs and the GCNV central storage. Properly configuring these export policies and routes ensures that when a failover occurs, the DR VPC can immediately leverage the existing peering connection.</p><h3>Critical Considerations</h3><ul><li><strong>GCVE Integration:</strong> If your architecture involves Google Cloud VMware Engine (GCVE), you must explicitly establish a network peering connection between GCVE and GCNV in the central region to enable communication between VMware workloads and the storage volumes.</li><li><strong>Automation vs. Manual Ops:</strong> While volume creation and replication can be handled via Terraform, the backend multi-VPC peering setup is a manual, one-time configuration via support.</li><li><strong>Cost Efficiency:</strong> This design is highly cost-efficient. By accessing a single replicated volume from multiple VPCs, you avoid paying for two identical copies of the data and the associated replication traffic.</li></ul><h3>Summary</h3><p><strong>Advantages:</strong></p><ul><li><strong>Cost Savings:</strong> Avoids duplicate storage and replication costs.</li><li><strong>Simplicity:</strong> Only one replication stream to manage.</li><li><strong>Resilience:</strong> Static configuration ensures connectivity is pre-existing during a disaster.</li></ul><p><strong>Trade-off:</strong></p><ul><li><strong>Support Dependency:</strong> Setup, edits, and deletions require a support ticket.</li><li><strong>PSA Limits:</strong> Subject to Google PSA peering limits and IP forwarding rules.</li><li><strong>Manual Setup:</strong> Cannot be fully automated via IaC tools like Terraform.</li></ul><p>By utilizing transitive peering, architects can bypass the one-to-many replication limitation, creating a flexible and cost-effective Disaster Recovery environment on Google Cloud.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf987609ea90" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/solving-the-one-to-many-challenge-multi-vpc-peering-for-netapp-dr-bf987609ea90">Solving the One-to-Many Challenge: Multi-VPC Peering for NetApp DR</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>