# Google AI Overview Scraper (`codingfrontend/google-ai-overview-scraper`) Actor

A robust, high-performance utility designed for developer automation, data integration, and AI training. Features built-in captcha bypass, headful/headless browser execution, and proxy support to scrape Google data seamlessly, reliably, and at scale.

- **URL**: https://apify.com/codingfrontend/google-ai-overview-scraper.md
- **Developed by:** [Coding Frontned](https://apify.com/codingfrontend) (community)
- **Categories:** Automation, Developer tools, AI
- **Stats:** 152 total users, 3 monthly users, 99.1% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.90 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google AI Overview Scraper

Collect genuine AI Overview answers and their public citations from Google Search. Queries with a normal search page but no overview are reported in `OUTPUT` and do not create placeholder dataset rows.

### Reliability contract

- Uses one bounded public Google result request per query; no browser fingerprint spoofing or hidden browser state.
- Writes a dataset row only when a recognizable AI Overview contains at least 50 characters of answer text.
- Distinguishes `found`, `not_present`, `explicit_unavailable`, `blocked`, `unrecognized`, and transport-failure boundaries.
- Treats normal no-overview and Google’s explicit “AI Overview is not available” response as verified query outcomes.
- Holds all rows until every requested query has a verified outcome; challenge, malformed, or transport failures fail atomically.
- Never stores raw HTML. Responses are capped at 6 MB and public answer text is capped at 20,000 characters.

The default `GOOGLE_SERP` proxy is recommended for public Google result pages. AI Overview availability varies by query, locale, time, and Google’s serving decisions.

### Input

| Field | Type | Default | Description |
|---|---|---:|---|
| `queries` | string\[] | required | One to 20 unique queries, up to 300 characters each. |
| `maxItems` | integer | `10` | Process at most this many input queries. |
| `gl` | string | `us` | Two-letter Google country code. |
| `hl` | string | `en` | Language code, optionally with country suffix. |
| `maxRequestRetries` | integer | `1` | Zero to two retries per query. |
| `requestTimeoutSecs` | integer | `60` | Hard 15–90 second request timeout. |
| `proxyConfiguration` | object | Google SERP proxy | Standard Apify proxy configuration. |

```json
{
  "queries": ["benefits of green tea"],
  "maxItems": 1,
  "gl": "us",
  "hl": "en",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["GOOGLE_SERP"]
  }
}
```

### Output

The dataset contains only genuine overview records:

```json
{
  "query": "example query",
  "aiOverviewText": "The source-backed overview text returned by Google...",
  "sources": [{ "title": "Example source", "url": "https://example.org/article" }],
  "relatedQuestions": ["A related question?"],
  "searchUrl": "https://www.google.com/search?q=example+query&gl=us&hl=en",
  "scrapedAt": "2026-08-31T00:00:00.000Z"
}
```

`OUTPUT` records counts and a per-query status even when the dataset is validly empty. An empty dataset therefore means Google did not serve an overview for the tested query—not that a placeholder row was lost.

# Actor input Schema

## `queries` (type: `array`):

One to 20 unique search queries.

## `maxItems` (type: `integer`):

Process at most this many input queries.

## `gl` (type: `string`):

Two-letter Google country code.

## `hl` (type: `string`):

Language code, optionally followed by a country code.

## `maxRequestRetries` (type: `integer`):

Retries for failed Google requests.

## `requestTimeoutSecs` (type: `integer`):

Hard timeout for each request.

## `proxyConfiguration` (type: `object`):

Standard Apify proxy settings. GOOGLE\_SERP is recommended.

## Actor input object example

```json
{
  "queries": [
    "What is quantum computing?"
  ],
  "maxItems": 10,
  "gl": "us",
  "hl": "en",
  "maxRequestRetries": 1,
  "requestTimeoutSecs": 60,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `output` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "What is quantum computing?"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "GOOGLE_SERP"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("codingfrontend/google-ai-overview-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["What is quantum computing?"],
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["GOOGLE_SERP"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("codingfrontend/google-ai-overview-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "What is quantum computing?"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  }
}' |
apify call codingfrontend/google-ai-overview-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,codingfrontend/google-ai-overview-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/H6QnTyeYGQ4GWrSb4/builds/PzWN76Zvpdu1a8iou/openapi.json