Get insights with the Data Insights agent

The Data Insights agent is an agent that's Made by Google. It gives you data insights from your BigQuery data. With the Data Insights agent, you don't need prior knowledge of SQL. This lets you make well-informed, data-driven business decisions and frees up data analysts to focus on more complex tasks.

This page describes how a Google Cloud project administrator can authorize, create, and deploy the Data Insights agent using Google Cloud console and REST API. This page also shows how an end user can use the agent.

Overview

The Data Insights agent is designed to do the following:

• Understand the user's intent: It analyzes the context of connected data sources and the user's natural language query to understand the user's goal
• Generate SQL: Based on this understanding, it converts the user's question into a syntactically and semantically correct SQL query.
• Retrieve data: It then executes the generated SQL to fetch the relevant data directly from the connected data source, a BigQuery dataset.
• Provide insights: It presents the retrieved data as visualizations, such as charts and tables, or as text-based summaries to answer the user's query.

Example queries that you can ask the Data Insights agent

Here are some examples queries that you can ask the Data Insights agent:

• Data aggregation and visualization:
  • "How did Q2 sales in the LATAM region this year compare to Q2 last year?"
  • "Plot a bar chart showing the comparison for each of the top 5 countries in the region."
• Trend analysis:
  • "How has outgoing call volume varied over the last 6 months, broken out by location?"
  • "Analyze the booking patterns for the hotels in Lisbon rated higher than 3 stars"
• Data mining:
  • "Which factors are correlated with the total sales value when a customer buys something? Give me a heatmap showing the relationship."
• Analysis and reporting:
  • "Summarize the opportunities and accounts table and create a short report highlighting any key trends."

Before you begin

To start using the Data Insights agent in Gemini Enterprise, follow these steps:

• Follow the instructions to get started with Gemini Enterprise.
• Follow the instructions to get licenses for Gemini Enterprise.
• Contact your Google account manager to access the Data Insights agent.
• Prepare your BigQuery data. For more information, see BigQuery documentation.
• To get the best out of the agent, understand the data in your BigQuery dataset.

Grant access to BigQuery data

To enable the Data Insights agent to view and query the BigQuery data, grant the Identity and Access Management (IAM) roles to the agent's users:

• BigQuery Data Viewer (roles/bigquery.dataViewer)
• BigQuery Job User (roles/bigquery.jobUser)
• BigQuery Metadata Viewer (roles/bigquery.metadataViewer)

Workflow

The overall workflow to set up and use the Data Insights agent is as follows:

• Obtain authorization details.
• Set up the agent using the Google Cloud console or using the REST API.
• Add or modify users and their permissions
• Change the working state of the agent instance
• Use the agent

Obtain authorization details

Follow these steps to set up authorization. The details you obtain are needed to authorize the Data Insights agent to connect to the BigQuery data.

1. In the Google Cloud console, go to the Credentials page in APIs & Services.

   Go to Credentials

2. Select the Google Cloud project that contains the BigQuery dataset that you want the agent to query.

3. Click Create credentials and select OAuth client ID.

4. In Application type, select Web application.

5. In the Authorized redirect URIs section, add the following URIs:

   • https://vertexaisearch.cloud.google.com/oauth-redirect
   • https://vertexaisearch.cloud.google.com/static/oauth/oauth.html

6. Click Create.

7. In the OAuth client created panel, click Download JSON. The downloaded JSON includes the following details for the selected Google Cloud project. You need these details to create an authorization resource:

   • Client ID: CLIENT_ID
   • Authorization URI:

     https://accounts.google.com/o/oauth2/v2/auth?client_id=CLIENT_ID&redirect_uri=https%3A%2F%2Fvertexaisearch.cloud.google.com%2Fstatic%2Foauth%2Foauth.html&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fbigquery&include_granted_scopes=true&response_type=code&access_type=offline&prompt=consent
     

   • Token URI: https://oauth2.googleapis.com/token
   • Client secret: CLIENT_SECRET

Set up the agent using Google Cloud console

This section shows how to authorize, create, and deploy an instance of the Data Insights agent using Google Cloud console. You can also add user permissions that determine who can access the created agent.

Authorize and create an agent instance

Authorize and create a Data Insights agent instance using these steps:

1. In Google Cloud console, go to Gemini Enterprise.

   Go to Gemini Enterprise Enterprise

2. Select an app in which you want to create the agent.

3. In the menu, click Agents.

   The Agents page displays the existing agents.

4. Click Add agent.

5. In the Create agent pane, click Create in the Data agent card.

6. In Authorizations, click Add authorization and enter the authorization details. For more information, see Obtain authorization.

7. Click Done.

8. Click Next.

9. Configure your agent as follows:

   1. Enter your agent name and description.
   2. In BigQuery dataset, click Browse and do one of the following:
      • Select an available dataset and click Select.
      • Enter the path to the required BigQuery dataset, click Search, select it, and then select Select.

   3. Optional: Click Show more for advanced options.

   4. Select the correct table access options. If you want to impose an allowlist or a blocklist, specify the paths to the restricted tables.

   5. Optional: Define the natural language query configuration to provide customizations specific to a natural language translation to SQL or Python code. You can also provide a SQL example using a natural language query, its expected SQL output, and its expected response. This improves the quality of the agent's outputs.

      • Schema description: a string in natural language that describes the schema of the BigQuery dataset.
      • Natural language query to SQL prompt: a query in natural language transformed into a SQL instruction.
      • Natural language query to Python prompt: a query in natural language transformed into a Python instruction.

   6. Optional: Add examples of natural language queries transformed to SQL queries:

      • Query: an example of a natural query that must be converted to a SQL query. For example, "What are the names and email addresses of the customers based in California"
      • Expected SQL: a string that illustrates an example SQL query corresponding to the natural language query. For example, suppose that you have a BigQuery table named customers. Then, your expected SQL query can be SELECT customer_name, email FROM customers WHERE state = 'California'.
      • Expected response: a string that supplies the expected answer for the query by executing the expected SQL query. For example:

      Here are the names and email addresses of your customers in California: \
      * Customer name: Lara B, Email address: [email protected] \
      * Customer name: Alex A, Email address: [email protected] \
      * Customer name: Bola C, Email address: [email protected] \
      

10. Click Create.

    The Data Insights agent instance appears in the Agents list.
    To start working with the agent, wait until the Agent state column shows Enabled for your instance.

Set up the agent using REST API

This section describes how to authorize, create, and deploy a Data Insights agent instance using the REST API.

Authorize the agent

As an administrator, create an authorization resource in Gemini Enterprise. This lets the Data Insights agent access the BigQuery data.

1. Create the authorization resource.

   REST

   The following sample shows how to create an authorization resource using the authorizations.create method.

   curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    -H "X-Goog-User-Project: PROJECT_NUMBER" \
    "https://discoveryengine.googleapis.com/v1alpha/projects/%3Cvar%20translate="no">PROJECT_NUMBER/locations/LOCATION/authorizations?authorizationId=AUTHORIZATION_ID" \
    -d '{
      "name": "projects/PROJECT_NUMBER/locations/LOCATION/authorizations/AUTHORIZATION_ID",
      "serverSideOauth2": {
        "clientId": "CLIENT_ID",
        "clientSecret": "CLIENT_SECRET",
        "authorizationUri": "AUTHORIZATION_URI",
        "tokenUri": "https://oauth2.googleapis.com/token"
   }
   }'
   

   Replace the following:

   • PROJECT_NUMBER: the number of your Google Cloud project.
   • LOCATION: the location of your Google Cloud project.
   • AUTHORIZATION_ID: an ID that you must provide to identify the authorization resource.
   • CLIENT_ID: the client ID that you obtained in the previous step.
   • CLIENT_SECRET: the client secret that you obtained in the previous step.
   • AUTHORIZATION_URI: the authorization URI that you obtained in the previous step.

   Response

   You should receive a JSON response similar to the following response. Note the AUTHORIZATION_RESOURCE_NAME, which you must use to create the Data Insights agent. The unique ID at the end of the AUTHORIZATION_RESOURCE_NAME is the AUTHORIZATION_ID.

   {
    "name": "AUTHORIZATION_RESOURCE_NAME",
    "serverSideOauth2": {
      "clientId": "CLIENT_ID",
      "clientSecret": "CLIENT_SECRET",
      "tokenUri": "https://oauth2.googleapis.com/token",
      "authorizationUri": "AUTHORIZATION_URI"
    }
   }
   

Create an agent instance

As a Google Cloud project administrator, you can create a Data Insights agent instance. This requires the project ID and the dataset ID of the BigQuery data that you want to query using your agent.

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  -H "X-Goog-User-Project: PROJECT_NUMBER" \
  "https://discoveryengine.googleapis.com/v1alpha/projects/%3Cvar%20translate="no">PROJECT_NUMBER/locations/LOCATION/collections/default_collection/engines/APP_ID/assistants/default_assistant/agents" \
  -d '{
    "displayName": "AGENT_DISPLAY_NAME",
    "description": "AGENT_DESCRIPTION",
    "icon": {
       "uri": "AGENT_ICON_URI"
     },
    "managed_agent_definition": {
      "tool_settings": {
        "tool_description": "AGENT_DESCRIPTION"
      },
      "data_science_agent_config": {
        "bq_project_id": "BIGQUERY_PROJECT_ID",
        "bq_dataset_id": "BIGQUERY_DATASET_ID"
      }
    },
    "authorization_config": {
      "tool_authorizations" : [
        "AUTHORIZATION_RESOURCE_NAME"
      ]
    }
  }'

{
  "name": "AGENT_RESOURCE_NAME",
  "displayName": "AGENT_DISPLAY_NAME",
  "description": "AGENT_DESCRIPTION",
  "icon": {
    "uri": "AGENT_ICON_URI"
  },
  "createTime": "CREATION_TIME",
  "managedAgentDefinition": {
    "toolSettings": {
      "toolDescription": "AGENT_DISPLAY_NAME"
    },
    "deploymentInfo": {},
    "dataScienceAgentConfig": {
      "bqProjectId": "BIGQUERY_PROJECT_ID",
      "bqDatasetId": "BIGQUERY_DATASET_ID"
    }
  },
  "authorization_config": {
    "tool_authorizations" : [
      "AUTHORIZATION_RESOURCE_NAME"
    ]
  },
  "state": "CONFIGURED"
}

Add advanced configurations for the agent

You can optionally define the nlQueryConfig field to provide customizations specific to natural language translation to SQL or Python code. You can also provide a SQL example using a natural language query, its expected SQL output, and its expected response. This improves the quality of the agent's outputs. The following code snippet shows how you can configure these advanced fields:

"dataScienceAgentConfig": {
  "nlQueryConfig": {
    "nl2sqlPrompt": "NL_TO_SQL_INSTRUCTIONS",
    "nl2pyPrompt": "NL_TO_PYTHON_INSTRUCTIONS",
    "nl2sqlExample": {
      "query": "EXAMPLE_NL_QUERY",
      "expectedSql": "EXPECTED_SQL_QUERY",
      "expectedResponse": "EXPECTED_SQL_RESPONSE"
    },
    "schemaDescription": "NL_DESCRIPTION_OF_BQ_DATASET"
  }
}

Replace the following:

• NL_TO_SQL_INSTRUCTIONS: a query in natural language transformed into a SQL instruction.
• NL_TO_PYTHON_INSTRUCTIONS: a query in natural language transformed into a Python instruction.
• EXAMPLE_NL_QUERY: an example of a natural language query that must be converted to a SQL query. For example, "What are the names and email addresses of the customers based in California"
• EXPECTED_SQL_QUERY: a string that illustrates an example SQL query corresponding to the natural query. For example, suppose that you have a BigQuery table named customers. Then, your expected SQL query can be "SELECT customer_name, email FROM customers WHERE state = 'California'".

• EXPECTED_SQL_RESPONSE: a string that supplies the expected answer for the query and the expected SQL query. For example:

  Here are the names and email addresses of your customers in California: \
  * Customer name: Lara B, Email address: [email protected] \
  * Customer name: Alex A, Email address: [email protected] \
  * Customer name: Bola C, Email address: [email protected] \
  

• NL_DESCRIPTION_OF_BQ_DATASET: a string in natural language that describes the schema of the BigQuery dataset.

Deploy the instance

After creating the Data Insights agent instance, as an administrator, you can deploy it so that the end users can use it.

Add or modify users and their permissions

Add or modify principals to your Data Insights agent instance and assign specific Identity and Access Management (IAM) roles to them, using these steps:

Change the working state of the instance

After you create the Data Insights agent instance, the agent is enabled, by default. You can change its working state to Preview, Disable, Suspend, or Delete using these steps:

Use the agent

Follow these steps to get data insights using your agent: