# Google AI Overview Scraper (`searchapi/google-ai-overview-scraper`) Actor

Scrapes Google AI Overviews (SGE) for any query. Extracts the overview text and HTML, model/provider, sources, inline references, follow-up/related questions, organic results, extraction confidence, and more.

- **URL**: https://apify.com/searchapi/google-ai-overview-scraper.md
- **Developed by:** [Search API](https://apify.com/searchapi) (community)
- **Categories:** AI, Developer tools, Automation
- **Stats:** 139 total users, 1 monthly users, 83.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.99 / 1,000 search results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google AI Overview Scraper

Extract Google AI Overview answers together with citations, related questions, organic-result context, and explicit no-overview status records. The Actor supports one query or a deduplicated list of queries and processes searches concurrently within a user-defined limit.

### Input

| Field | Required | Default | Description |
| --- | --- | --- | --- |
| `query` | No | — | Single search query, processed first. |
| `queries` | No | `[]` | Additional search queries. If both query inputs are empty, the Actor uses `What is quantum computing`. |
| `maxItems` | No | `10` | Maximum query records, from 1 to 100. |
| `maxConcurrency` | No | `3` | Concurrent Google searches, from 1 to 10. |
| `gl` | No | `us` | Two-letter Google country code. |
| `hl` | No | `en` | Google interface language code. |
| `maxRequestRetries` | No | `2` | Bounded retries for temporary transport failures. Explicit CAPTCHA pages are never retried. |
| `navigationTimeoutSecs` | No | `45` | Navigation timeout. |
| `requestHandlerTimeoutSecs` | No | `120` | Whole-page extraction timeout. |
| `resultsWaitSecs` | No | `15` | Wait for the Google result shell before extraction. |
| `proxyConfiguration` | No | `GOOGLE_SERP` | `GOOGLE_SERP` uses Apify's HTTP-only Google search transport. Standard/custom proxies use Playwright; Residential runs without a country align to `gl`. |

### Output

The 39-field schema includes:

- stable query identity, position, locale, search URL, and scrape timestamp;
- AI Overview text, sanitized HTML, character and token estimates, model/provider, confidence, and matched selector;
- normalized source citations and inline references;
- related/follow-up questions and up to ten organic results;
- `hasAiOverview` and `errorType` for successful searches where Google shows no overview.

Unavailable optional values are omitted recursively. Valid zero counts and `false` values are retained. Challenge pages, raw Google responses, cookies, authorization data, and proxy credentials are never stored.

### Extraction behavior

With `GOOGLE_SERP`, the Actor uses the proxy group's required HTTP search transport and parses the returned HTML directly. Other configurations use Playwright with a consistent desktop fingerprint per session, a locale and Residential country aligned with `hl`/`gl`, short-lived sessions, bounded concurrency, and session rotation. Both paths validate response status and content type before parsing. Explicit CAPTCHA or unusual-traffic pages fail closed with a `TARGET_BLOCKED` status and are not bypassed.

### Local verification

```sh
npm ci
npm test
npm run validate:dataset
apify validate-schema
apify run --purge
```

Google commonly challenges local and datacenter IP addresses. Local external access to `GOOGLE_SERP` may be disabled by the Apify account plan, so validate that transport in Apify Cloud. Authentication, CAPTCHA, and access controls are not bypassed.

# Actor input Schema

## `query` (type: `string`):

A single query to search. It is processed before any entries in Search Queries.

## `queries` (type: `array`):

List of search queries to scrape AI Overviews for.

## `maxItems` (type: `integer`):

Maximum number of results to return.

## `maxConcurrency` (type: `integer`):

Maximum query pages processed in parallel.

## `maxRequestRetries` (type: `integer`):

Retry temporary navigation, proxy, browser, rate-limit, and server failures up to this many times.

## `navigationTimeoutSecs` (type: `integer`):

Maximum time allowed for each Google page navigation.

## `requestHandlerTimeoutSecs` (type: `integer`):

Maximum time allowed to wait for and extract one Google results page.

## `resultsWaitSecs` (type: `integer`):

Maximum wait for the Google results shell before DOM extraction.

## `gl` (type: `string`):

Google country parameter (e.g. 'us', 'uk', 'in').

## `hl` (type: `string`):

Google language parameter (e.g. 'en', 'es', 'fr').

## `proxyConfiguration` (type: `object`):

GOOGLE\_SERP uses the supported HTTP-only search transport and is the default. Other proxy groups use the Playwright browser path; Residential country is aligned with gl when omitted.

## Actor input object example

```json
{
  "query": "What is quantum computing",
  "queries": [
    "What is quantum computing"
  ],
  "maxItems": 10,
  "maxConcurrency": 3,
  "maxRequestRetries": 2,
  "navigationTimeoutSecs": 45,
  "requestHandlerTimeoutSecs": 120,
  "resultsWaitSecs": 15,
  "gl": "us",
  "hl": "en",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "What is quantum computing",
    "queries": [
        "What is quantum computing"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("searchapi/google-ai-overview-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "What is quantum computing",
    "queries": ["What is quantum computing"],
}

# Run the Actor and wait for it to finish
run = client.actor("searchapi/google-ai-overview-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "What is quantum computing",
  "queries": [
    "What is quantum computing"
  ]
}' |
apify call searchapi/google-ai-overview-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,searchapi/google-ai-overview-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Vhf7jZIs0bniBxXF7/builds/jegufaGC49cJYTdEc/openapi.json