# Google Search Results Scraper (Question & Keyword Discovery) (`simpleapi/google-search-results-scraper`) Actor

Google Search Results Scraper for Question & Keyword Discovery extracts search results, related questions, autocomplete suggestions, titles, URLs, snippets, and keyword insights. Discover content opportunities, search intent, long-tail keywords, and topics for SEO and market research.

- **URL**: https://apify.com/simpleapi/google-search-results-scraper.md
- **Developed by:** [SimpleAPI](https://apify.com/simpleapi) (community)
- **Categories:** AI, Lead generation, Automation
- **Stats:** 3 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### Google Search Results Scraper — SERPs, Questions and Keyword Ideas

Google Search Results Scraper (Question & Keyword Discovery) scrapes live www.google.com results pages and returns `organicResults`, `paidResults`, `paidProducts` and `peopleAlsoAsk` for every seed keyword — plus a discovery pass most Google SERP scrapers skip: genuine `relatedSearches` pulled from Google's own `#botstuff` block, `autocompleteSuggestions` from Google's completion service, and `expandedQuestions` pulled straight off the results page. Every phrase collected is also merged into one deduplicated `keywordIdeas[]` array with its source attached. It's built for SEO teams, content strategists and PPC analysts who need real keyword and question coverage next to the SERP itself, not a second tool. No Google account or API key is required to run it.

### What is Google Search Results Scraper (Question & Keyword Discovery)?

It's an Apify Actor that submits your seed keywords to Google Search over a proxy, parses the resulting HTML, and returns one structured row per query per page — organic listings, paid ads, shopping products, People-Also-Ask questions — along with a second, dedicated fetch that reads Google's related-searches block, autocomplete completions, and question text. No login or Google account is needed; the Actor authenticates only against Apify's own proxy network. Everything is delivered as a standard Apify dataset that exports to JSON, CSV, Excel/XLSX, XML or RSS.

Key features:

- 🔎 **Flexible seeds** — plain keywords, Google search operators (`site:example.com`, `javascript OR python`), or a pasted `google.com/search` URL, one per line
- ❓ **Real People-Also-Ask questions** — collected straight from the results page, not simulated
- 🔁 **Genuine related searches and autocomplete** — parsed from Google's own `#botstuff` block and its `/complete/search` completion service, with an optional A–Z alphabet fan-out for autocomplete
- 💡 **Merged keyword ideas** — `keywordIdeas[]` unions related searches, autocomplete completions and PAA question text into one deduplicated list with provenance per phrase
- 🌍 **Country, language and location targeting** — `gl`, `hl`, `lr` and UULE location codes, plus `site:`, `intitle:`, `intext:`, `inurl:` and `filetype:` operators and date filters
- 🔒 **Automatic proxy escalation** — starts on Apify's Google SERP proxy and escalates to residential (3 retries, then sticky) whenever a page comes back without a result container

### What data can I extract with Google Search Results Scraper?

Every seed keyword returns a row with the base SERP fields plus the discovery fields the Actor adds on top. Field names below are the exact JSON keys pushed to the dataset — the default dataset view only shows 8 of them (`searchQueryTerm`, `resultsTotal`, `relatedQueries`, `aiOverview`, `paidResults`, `paidProducts`, `organicResults`, `peopleAlsoAsk`); every row carries the full set.

| Field | Example value | Use case |
| --- | --- | --- |
| `searchQuery` | `{"term": "best running shoes", "page": 1, "countryCode": "US", ...}` | Trace which query, page, country and language produced the row |
| `searchQueryTerm` | `"best running shoes"` | Group or filter rows by seed |
| `url` | `"https://www.google.com/search?q=..."` | The exact SERP URL that was fetched |
| `hasNextPage` | `true` | Detect whether a further page exists (respects `maxPagesPerQuery`) |
| `serpProviderCode` | `"O"` | Base actor's provider code, carried through unchanged |
| `resultsTotal` | `184000000` | Google's reported "About X results" count |
| `organicResults` | `[{"title": "...", "url": "...", "position": 1, ...}]` | The organic listings — title, url, description, position, site links |
| `relatedQueries` | `[{"title": "running shoes for flat feet", "url": "..."}]` | Base-compatible shape, now built from Google's real related-searches block |
| `suggestedResults` | `[{"title": "...", "url": "...", "type": "organic", "position": 1}]` | Same related-search data reshaped to match the base actor's field |
| `paidResults` | `[{"title": "...", "url": "...", "position": 1}]` | Text ads parsed from `#tads` |
| `paidProducts` | `[{"title": "...", "displayedUrl": "...", "prices": ["$129.99"]}]` | Shopping/product cards with parsed prices |
| `peopleAlsoAsk` | `[{"question": "What running shoes...", "answer": null, "url": "..."}]` | Base PAA field — question text and link only; `answer` is always `null` (see limits below) |
| `aiOverview` | `{"text": "...", "references": [...]}` or `null` | Only populated when `aiMode` is not off **and** `serpApiKey` is supplied |
| `customData` | `{"perplexitySearch": {...}, "chatGptSearch": {...}, ...}` | Echo of unimplemented add-on inputs, kept for schema compatibility |
| `htmlSnapshotUrl` | `"running_shoes_20260816_101500_p1.html"` | Key-value store record key of the first saved SERP page (when `saveHtmlToKeyValueStore` is on) |
| `html` | `"<!-- PAGE 1 -->..."` | Raw page HTML, only present when `saveHtml` is on |
| `relatedSearches` | `[{"position": 1, "query": "running shoes for flat feet", "url": "..."}]` | Google's genuine related-searches block, or `null` if Google served none |
| `relatedSearchesCount` | `8` | Row count of `relatedSearches` |
| `autocompleteSuggestions` | `[{"position": 1, "suggestion": "running shoes for women", "type": "QUERY", "seed": "running shoes"}]` | Completions from Google's `/complete/search` endpoint |
| `autocompleteSuggestionsCount` | `10` | Row count of `autocompleteSuggestions` |
| `expandedQuestions` | `[{"position": 1, "question": "What running shoes are best for beginners?"}]` | Real PAA question text, capped by `maxQuestionsPerQuery` |
| `expandedQuestionsCount` | `5` | Row count of `expandedQuestions` |
| `keywordIdeas` | `[{"position": 1, "keyword": "running shoes for flat feet", "sources": ["relatedSearch", "autocomplete"]}]` | Deduplicated union of every real phrase collected, with source(s) |
| `hasRelatedSearches` | `true` | Quick boolean check without counting the array |
| `hasExpandedQuestions` | `true` | Quick boolean check without counting the array |
| `discoveryPageUrl` | `"https://www.google.com/search?q=..."` | URL of the separate discovery request (no `udm=14`) |
| `discoveryNotes` | `["relatedSearches_selector=botstuff-ngTNl", "questions_selector=related-question-pair"]` | Diagnostic notes on which selector rung matched, or why discovery was skipped |
| `errorReason` | `null` | `null` on every charged row; set to a reason string only on the uncharged fallback row |

#### Questions and keyword ideas

`expandedQuestions`, `relatedSearches` and `autocompleteSuggestions` are collected on one extra request per seed (the discovery page, fetched without `udm=14`) and merged into every page row of that query, so raising `maxPagesPerQuery` never multiplies discovery requests. Answers to People-Also-Ask questions are **not** returned — see the limits note below. All three arrays feed `keywordIdeas[]`, so a content team can pull one flat, de-duplicated keyword list per seed with each phrase's origin (`relatedSearch`, `autocomplete`, `peopleAlsoAsk`) attached, instead of cross-referencing three separate fields by hand.

#### Targeting fields

`countryCode` (`gl`) and `languageCode` (`hl`) decide the market and language for every result, including which country's autocomplete and related searches come back. `site` / `relatedToSite`, `wordsInTitle` / `wordsInText` / `wordsInUrl`, and `fileTypes` compile into `site:`, `intitle:`, `intext:`, `inurl:` and `filetype:` operators appended to the query. `quickDateRange`, `beforeDate` and `afterDate` narrow by recency via Google's `tbs` parameter. Together these let you scope a seed to one country, one site, one file type or one time window without changing the seed text itself.

### How does Google Search Results Scraper differ from the official Google Search API?

Google's own **Custom Search JSON API** caps every query at 10 results per request and never returns more than 100 results total for any query, even with pagination — and it is closed to new customers and scheduled for full shutdown on January 1, 2027, per Google's own developer documentation. It is also built on a curated Programmable Search Engine, not the open web index, so it has no related-searches, People-Also-Ask or autocomplete surface at all. This Actor scrapes the actual www.google.com results page for any query — no Custom Search Engine to configure — and adds a dedicated discovery pass for the data the official API never exposed in the first place.

| Feature | Custom Search JSON API | Google Search Results Scraper |
| --- | --- | --- |
| New signups | Closed since 2025; full shutdown Jan 1, 2027 | Available now on Apify |
| Max results per query | 10 per request, 100 total (hard error beyond that) | Up to 100 per page (`resultsPerPage`), pages capped only by `maxPagesPerQuery` |
| Search scope | A configured Custom Search Engine (often site-restricted) | Any query against the live www.google.com index |
| Related searches / autocomplete / PAA | Not available | `relatedSearches`, `autocompleteSuggestions`, `expandedQuestions`, merged into `keywordIdeas[]` |
| AI Overview | Not available | Optional, via your own SerpApi key (`aiMode` + `serpApiKey`) |
| Query operators | Limited to CSE configuration | `site:`, `intitle:`, `intext:`, `inurl:`, `filetype:`, exact-match, date range |
| Pricing | $5 per 1,000 queries beyond 100 free/day | Pay-per-result on Apify (`row_result` event) |

#### Why is the Custom Search JSON API being shut down?

Google closed the Custom Search JSON API to new customers in 2025 and announced it will be fully discontinued on January 1, 2027, according to Google's own developer documentation. Even for existing customers, it was never a substitute for reading the real results page: it returns at most 100 results per query because it queries a curated Programmable Search Engine index rather than the open web, and it has no endpoint for related searches, People-Also-Ask questions or autocomplete at any price. This Actor fetches the genuine www.google.com results page instead, so it isn't affected by the shutdown and returns the discovery data the official API never had.

Use the Custom Search JSON API only if you already have a grandfathered account and a narrowly scoped Custom Search Engine you need to keep running before 2027. Use this Actor for open-web queries, ongoing keyword research, or any workflow that needs related searches, autocomplete or PAA questions alongside the SERP.

### How to use data extracted from Google Search?

#### SEO and content teams

Feed `queries` with a topic cluster's seed terms, turn on `expandPeopleAlsoAsk` and `includeKeywordIdeas`, and pull `keywordIdeas[]` plus `expandedQuestions[]` straight into a content brief. Each keyword idea carries its `sources` array, so a writer can see at a glance whether a phrase came from Google's own related-searches block, autocomplete, or a real PAA question — useful for prioritizing head terms versus long-tail question content without opening Google manually for every seed.

#### Agencies and service providers

Run the same seed list on a schedule across client accounts or verticals, and diff `relatedSearches` and `keywordIdeas[]` run over run to flag when Google surfaces new related terms or drops old ones for a client's target keywords. `discoveryNotes` records which selector matched (or why discovery was skipped), which is useful evidence when a client asks why a particular run returned fewer questions than usual.

#### Market research and intelligence

Compare `paidResults` and `paidProducts` counts across a basket of seed keywords to map which topics carry ad pressure, or track `resultsTotal` and `organicResults[].displayedUrl` domains over time to see which sites are gaining or losing organic share for a category. `focusOnPaidAds` forces a retry when a page parses zero ads, which matters because Google injects ad markup with JavaScript and the plain HTTP path often misses it on the first pass.

#### AI agents and automated pipelines

Because the Actor is a standard Apify Actor with a JSON input and output, it's directly callable as a tool from an agent framework or through Apify's MCP server — an agent researching a topic can call it with a batch of seed terms and get back organic sources, related questions, and keyword ideas to ground a RAG pipeline or a content-generation step, without scraping Google itself.

### 🔼 Input sample

`queries` is the only required field. Every other input keeps the base actor's name, type and default so existing integrations don't break; discovery-specific inputs are listed at the bottom.

| Parameter | Required | Type | Description | Example value |
| --- | --- | --- | --- | --- |
| `queries` | Yes | string | One seed per line — keywords, Google operators, or pasted `google.com/search` URLs. Keep every line under 32 words. | `"best running shoes\nhow to train for a marathon"` |
| `resultsPerPage` | No | integer | Sent as Google's `num` param. Default 100, min 1, max 100. Does not affect discovery blocks. | `20` |
| `maxPagesPerQuery` | No | integer | How many result pages to walk per seed. Default 1, min 1, no maximum. Discovery still runs once per seed. | `2` |
| `aiMode` | No | string | `aiModeOff` (default), `aiModeWithSearchResults`, or `aiModeOnly`. Non-off drops `udm=14` and, with `serpApiKey`, fills `aiOverview`. | `"aiModeOff"` |
| `serpApiKey` | No | string | Your own SerpApi key. Only used when `aiMode` is not off. | `""` |
| `perplexitySearch` | No | object | Not implemented — echoed into `customData` for backward compatibility. Default `{"enablePerplexity": false, "searchRecency": "", "returnImages": false, "returnRelatedQuestions": false}`. | default |
| `chatGptSearch` | No | object | Not implemented — echoed into `customData`. Default `{"enableChatGpt": false}`. | default |
| `maximumLeadsEnrichmentRecords` | No | integer | Not implemented — echoed into `customData`. Default 0, min 0. | `0` |
| `leadsEnrichmentDepartments` | No | array | Not implemented — echoed into `customData`. Default `[]`. | `[]` |
| `focusOnPaidAds` | No | boolean | Retries a page once on the same proxy if it parsed zero ads and zero shopping products. Default `false`. | `true` |
| `countryCode` | No | string | Sent as `gl`. Default `"us"`. Full ISO country enum. Also decides the market for autocomplete and related searches. | `"gb"` |
| `searchLanguage` | No | string | Restricts results to one language, sent as `lr=lang_xx`. No default (empty = none). | `"en"` |
| `languageCode` | No | string | Sent as `hl`. Default `"en"`. Drives the language of questions, related searches and autocomplete. | `"es"` |
| `locationUule` | No | string | Google UULE location code, sent as `uule`. | `""` |
| `forceExactMatch` | No | boolean | Wraps the seed in quotes for an exact-phrase search. Default `false`. | `false` |
| `site` | No | string | Adds `site:example.com`. Takes precedence over `relatedToSite`. | `"nike.com"` |
| `relatedToSite` | No | string | Adds `related:example.com`. Ignored if `site` is set. | `""` |
| `wordsInTitle` | No | array | Each entry becomes `intitle:"word"`. Default `[]`, max 32 items. | `["review"]` |
| `wordsInText` | No | array | Each entry becomes `intext:"word"`. Default `[]`, max 32 items. | `[]` |
| `wordsInUrl` | No | array | Each entry becomes `inurl:"word"`. Default `[]`, max 32 items. | `[]` |
| `quickDateRange` | No | string | `qdr:` value (`d7`, `w2`, `m6`, `y1`...). Don't combine with before/after dates. | `"m6"` |
| `beforeDate` | No | string | Absolute (`2024-05-03`) or relative (`"8 days"`) date, UTC. | `""` |
| `afterDate` | No | string | Absolute or relative date, UTC. | `""` |
| `fileTypes` | No | array | OR-joined into `filetype:` operators. Default `[]`, max 10 items. | `[]` |
| `mobileResults` | No | boolean | Switches the user-agent pool. `searchQuery.device` still always reports `DESKTOP`. Default `false`. | `false` |
| `includeUnfilteredResults` | No | boolean | Sends `tbs=li:1` — Google's VERBATIM switch, which narrows results. Default `false`. | `false` |
| `saveHtml` | No | boolean | Attaches raw SERP HTML to the `html` field on every row. Default `false`. | `false` |
| `saveHtmlToKeyValueStore` | No | boolean | Writes each SERP page to the run's key-value store; `htmlSnapshotUrl` carries the first page's key. Default `true`. | `true` |
| `includeIcons` | No | boolean | Adds an `icon` field to organic results whose card exposes a favicon URL. Default `false`. | `false` |
| `proxyConfiguration` | No | object | Apify proxy config. Defaults to the Google SERP proxy group, escalating to residential on a block. | `{"useApifyProxy": true}` |
| `includeRelatedSearches` | No | boolean | Collects Google's genuine related-searches block into `relatedSearches[]`. Default `true`. | `true` |
| `includeAutocompleteSuggestions` | No | boolean | Calls Google's completion service and attaches results as `autocompleteSuggestions[]`. Default `true`. | `true` |
| `autocompleteAlphabetFanOut` | No | boolean | Also requests `<seed> a` through `<seed> z` — adds 26 keyless requests per seed. Default `false`. | `false` |
| `expandPeopleAlsoAsk` | No | boolean | Collects real PAA questions into `expandedQuestions[]`. Default `true`. | `true` |
| `maxQuestionsPerQuery` | No | integer | Caps questions returned per seed. Default 5, min 0, max 100. Set 0 to skip questions. | `10` |
| `includeKeywordIdeas` | No | boolean | Builds the deduplicated `keywordIdeas[]` union. Default `true`. | `true` |
| `useFullResultsPage` | No | boolean | Issues one extra request per seed without `udm=14` to read PAA/related searches. Default `true`. Off makes discovery fields `null` except autocomplete. | `true` |
| `deduplicateResultsAcrossPages` | No | boolean | Default `false` keeps the base behavior (each page re-contains earlier pages' results). `true` makes each row carry only its own page. | `false` |
| `saveQuestionsToSeparateDataset` | No | boolean | Also mirrors question rows (uncharged) to a named `<runId>-questions` dataset. Default `false`. | `false` |

```json
{
  "queries": "best running shoes\nhow to train for a marathon",
  "resultsPerPage": 20,
  "maxPagesPerQuery": 1,
  "countryCode": "us",
  "languageCode": "en",
  "expandPeopleAlsoAsk": true,
  "maxQuestionsPerQuery": 5,
  "includeRelatedSearches": true,
  "includeAutocompleteSuggestions": true,
  "includeKeywordIdeas": true
}
```

**Common pitfall:** `useFullResultsPage` defaults to `true` and issues the one extra request per seed that `relatedSearches` and `expandedQuestions` depend on. Turn it off and the Actor sends exactly the same request as the base actor — `relatedSearches`, `expandedQuestions` and `keywordIdeas`'s non-autocomplete sources all come back `null`, with only `autocompleteSuggestions` still populated.

### 🔽 Output sample

Each seed keyword produces one dataset row per page (up to `maxPagesPerQuery`), delivered as typed JSON with a consistent schema across runs. Export straight from the Apify dataset to JSON, CSV, Excel/XLSX, XML or RSS.

```json
{
  "searchQuery": {
    "term": "best running shoes",
    "url": "https://www.google.com/search?q=best+running+shoes&gl=us&hl=en",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 18
  },
  "searchQueryTerm": "best running shoes",
  "url": "https://www.google.com/search?q=best+running+shoes&gl=us&hl=en",
  "hasNextPage": false,
  "serpProviderCode": "O",
  "resultsTotal": 184000000,
  "relatedQueries": [
    {"title": "running shoes for flat feet", "url": "https://www.google.com/search?q=running+shoes+for+flat+feet"}
  ],
  "paidResults": [],
  "paidProducts": [
    {"title": "Nike Pegasus 41", "displayedUrl": "nike.com", "prices": ["$139.97"]}
  ],
  "aiOverview": null,
  "organicResults": [
    {
      "title": "The Best Running Shoes of 2026",
      "url": "https://example.com/best-running-shoes",
      "displayedUrl": "example.com/best-running-shoes",
      "description": "We tested dozens of pairs to find...",
      "emphasizedKeywords": ["best running shoes"],
      "siteLinks": [],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    {"title": "running shoes for flat feet", "url": "https://www.google.com/search?q=running+shoes+for+flat+feet", "type": "organic", "position": 1}
  ],
  "peopleAlsoAsk": [
    {"answer": null, "question": "What running shoes do podiatrists recommend?", "title": "What running shoes do podiatrists recommend?", "url": null, "date": null}
  ],
  "customData": {
    "perplexitySearch": {"enablePerplexity": false, "searchRecency": null, "returnImages": false, "returnRelatedQuestions": false},
    "chatGptSearch": {"enableChatGpt": false},
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": "best_running_shoes_20260816_101500_p1.html",
  "relatedSearches": [
    {"position": 1, "query": "running shoes for flat feet", "url": "https://www.google.com/search?q=running+shoes+for+flat+feet"}
  ],
  "relatedSearchesCount": 8,
  "autocompleteSuggestions": [
    {"position": 1, "suggestion": "best running shoes for women", "type": "QUERY", "seed": "best running shoes"}
  ],
  "autocompleteSuggestionsCount": 10,
  "expandedQuestions": [
    {"position": 1, "question": "What running shoes do podiatrists recommend?"}
  ],
  "expandedQuestionsCount": 5,
  "keywordIdeas": [
    {"position": 1, "keyword": "running shoes for flat feet", "sources": ["relatedSearch"]},
    {"position": 2, "keyword": "best running shoes for women", "sources": ["autocomplete"]},
    {"position": 3, "keyword": "What running shoes do podiatrists recommend?", "sources": ["peopleAlsoAsk"]}
  ],
  "hasRelatedSearches": true,
  "hasExpandedQuestions": true,
  "discoveryPageUrl": "https://www.google.com/search?q=best+running+shoes&gl=us&hl=en",
  "discoveryNotes": ["relatedSearches_selector=botstuff-ngTNl", "questions_selector=related-question-pair"],
  "errorReason": null
}
```

### How do you filter and target specific search results?

The seed itself is the primary filter: a plain keyword returns Google's default ranking, while a query built from `site`, `wordsInTitle`, `wordsInUrl` or `fileTypes` narrows to a specific slice of the index — `site` takes precedence over `relatedToSite` if both are set, and each `filetype:` entry is OR-joined when you supply more than one. `countryCode` (`gl`) and `languageCode` (`hl`) scope both the SERP and the discovery data to a market — autocomplete and related searches change with them, not just the organic results. `quickDateRange`, `beforeDate` and `afterDate` add a recency window via `tbs`, but the source explicitly warns not to combine `quickDateRange` with the before/after pair. `resultsPerPage` (up to 100) and `maxPagesPerQuery` control volume per seed; `deduplicateResultsAcrossPages` decides whether each page row repeats earlier pages' organic results (the default) or carries only its own page.

```json
{ "queries": "site:reddit.com best running shoes", "site": "reddit.com" }
```

```json
{ "queries": "running shoes", "countryCode": "de", "languageCode": "de", "quickDateRange": "m6" }
```

```json
{ "queries": "running shoes review", "wordsInTitle": ["review"], "fileTypes": ["pdf"] }
```

### ▶️ Want to try other Google scrapers?

| Scraper | What it extracts |
| --- | --- |
| [Google Search Results (SERP) Scraper & Shopping Prices](https://apify.com) | Organic results, ads and shopping prices from a Google Search page |
| [Google Maps Reviews Scraper (Bad Review Finder)](https://apify.com) | Business reviews and ratings from Google Maps, flagged by rating |
| [Google Play Scraper: Country Pricing & In-App Purchases](https://apify.com) | App store listings, country pricing and in-app purchase data |
| [Google Scholar Scraper With Institution & Journal Data](https://apify.com) | Academic paper citations, authors, institutions and journals |
| [Baidu Search Scraper By Site & File Type](https://apify.com) | Organic results from Baidu, filterable by site and file type |
| [DuckDuckGo Scraper - Fresh News By Date & Source](https://apify.com) | Organic and news results from DuckDuckGo, filterable by date and source |

### How to extract Google Search data programmatically

The Actor runs like any Apify Actor: one API call starts a run, one Bearer token authenticates it, and structured JSON comes back through the dataset endpoint.

#### Python example

```python
from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("<YOUR_USERNAME>/google-search-results-scraper-question-and-keyword-discovery").call(
    run_input={
        "queries": "best running shoes\nmarathon training plan",
        "expandPeopleAlsoAsk": True,
        "includeKeywordIdeas": True,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["searchQueryTerm"], len(item["organicResults"]), item["expandedQuestionsCount"])
```

#### MCP for AI agents

Reachable through Apify's hosted MCP server at `https://mcp.apify.com` — an agent can `search-actors` for this Actor and `call-actor` by name without pre-configuring it, then use the returned `keywordIdeas[]` and `expandedQuestions[]` to ground a content-brief or RAG step. Compatible clients include Claude Desktop, Claude Code, Cursor, VS Code (GitHub Copilot agent mode) and the Apify CLI.

#### Export to spreadsheets or CRM

Download the dataset as CSV or Excel/XLSX directly from the Apify run, or fetch it via the API. Map `searchQueryTerm` to a keyword column, `organicResults[].url` / `.title` to a rankings sheet, and `keywordIdeas[].keyword` / `.sources` to a content-planning tab.

### Is it legal to scrape Google Search results?

Yes. Google Search results pages are public web pages — anyone with a browser can view the same organic listings, ads, related searches and questions this Actor collects, and public search-result data is not personal or private information. The relevant framework is Google's Terms of Service and general database-rights law, not GDPR, since these are business/product listings rather than personal data about identifiable individuals. Scraping public data is broadly legal in most jurisdictions (see *hiQ Labs v. LinkedIn*), but bulk automated access to Google specifically can still run into its Terms of Service, so avoid excessive request volume and don't republish scraped SERP content as your own. Consult legal counsel for commercial applications involving bulk storage of personal data.

### ❓ FAQ

#### What happens if Google delists or reranks a page between runs?

The Actor returns Google's SERP exactly as rendered at request time — there's no "delisted" flag, because organic ranking is a live snapshot, not a status field. Re-run the same seed on a schedule and compare `organicResults[].position` and `.url` across runs to detect ranking changes or a page dropping out of the results entirely.

#### Can I get People-Also-Ask questions along with the search results?

Yes — with `expandPeopleAlsoAsk` on (default), real question text is collected into `expandedQuestions[]`, capped by `maxQuestionsPerQuery` (default 5, max 100). Answer text is **not** returned: the async endpoint Google uses to render PAA answers pays out only to a cookied browser origin, so it was measured returning a 59-byte control frame with no payload on every available proxy. Any answer text you need has to come from the matching organic snippet, which is already in `organicResults`.

#### How accurate is the keyword and question data?

The Actor returns Google's related searches, autocomplete completions and PAA questions exactly as Google served them for that query, country and language at request time. Google's discovery surfaces change frequently and vary by market, so treat a single run as a snapshot — re-run periodically if you're tracking keyword trends rather than a point-in-time list.

#### How many results can I get per run?

There's no fixed cap on the number of seeds in `queries` or on `maxPagesPerQuery` (minimum 1, no maximum in the schema) — each additional page is a separate charged row. `resultsPerPage` is capped at 100 by Google's own `num` parameter, though Google applies its own filtering so the actual count per page often comes in lower.

#### How does the site: and intitle: operator targeting work?

`site` compiles to `site:example.com` and takes precedence if `relatedToSite` is also set; `wordsInTitle`, `wordsInText` and `wordsInUrl` each compile their entries into `intitle:"word"`, `intext:"word"` and `inurl:"word"` operators (up to 32 entries each), and multiple `fileTypes` are OR-joined into `filetype:` clauses. All of them are appended to the seed query before it's sent to Google, so you can combine several at once in a single request.

#### Does this Actor work with Claude, ChatGPT and AI agent frameworks?

Yes. It's reachable through Apify's hosted MCP server (`https://mcp.apify.com`), where an agent can call it by name via `call-actor` — compatible with Claude Desktop, Claude Code, Cursor and VS Code's agent mode. It's also callable as a plain HTTP endpoint from any agent framework that can make an authenticated API call.

#### How does this Actor compare to other Google SERP scrapers?

Most Google SERP scrapers on Apify parse `organicResults`, `paidResults` and the base `peopleAlsoAsk` field the same way. This Actor's distinct addition is the discovery layer: genuine `relatedSearches` read from Google's own related-searches block (not the base actor's practice of relabeling organic titles as related queries), real autocomplete completions, and PAA question text merged into one deduplicated `keywordIdeas[]` array — features not present in the base SERP actor this one extends.

#### Can I use this Actor without a Google API key or developer account?

Yes. No Google account, API key or Custom Search Engine setup is required — the Actor only needs an Apify account to run. The optional `serpApiKey` field is only needed if you want `aiOverview` populated, which uses your own SerpApi account and is billed separately by SerpApi.

### Conclusion

Google Search Results Scraper (Question & Keyword Discovery) returns the full Google SERP — organic, paid, shopping and AI Overview — alongside genuine related searches, autocomplete completions and People-Also-Ask questions merged into one keyword-ideas list, without needing a Custom Search Engine or a soon-to-be-retired API key. It's built for SEO teams, content strategists, and agencies who need real keyword and question coverage attached to the SERP in one run. Start it from the Apify Store with a list of seed keywords, or call it programmatically through the Apify API or MCP.

# Actor input Schema

## `queries` (type: `string`):

One seed per line. Plain keywords, Google operators (site:example.com, javascript OR python) or pasted google.com/search URLs all work. Each seed becomes one discovery job: SERP results plus its related searches, autocomplete completions and People-also-ask questions. Keep every line under 32 words.

## `resultsPerPage` (type: `integer`):

Passed to Google as the num parameter. Google applies its own filtering, so the returned count often differs. This setting does not affect the discovery blocks (related searches / questions / autocomplete).

## `maxPagesPerQuery` (type: `integer`):

How many result pages to walk per seed. Discovery runs once per seed regardless of this number, so raising it does not multiply discovery requests.

## `aiMode` (type: `string`):

Kept from the base actor. Selecting a non-off mode drops udm=14 from the primary request and, together with a SerpApi key below, fills aiOverview. The discovery blocks in this Actor do not depend on it - they use their own request.

## `serpApiKey` (type: `string`):

Optional. When set and AI Mode is not off, aiOverview is fetched through SerpApi using your own account. Leave empty to skip.

## `perplexitySearch` (type: `object`):

Retained for backward compatibility with the base actor's input. This Actor does not call Perplexity: the object is echoed verbatim into customData and has no other effect.

## `chatGptSearch` (type: `object`):

Retained for backward compatibility with the base actor's input. This Actor does not call ChatGPT: the object is echoed verbatim into customData and has no other effect.

## `maximumLeadsEnrichmentRecords` (type: `integer`):

Retained for backward compatibility with the base actor's input. No lead data is collected; the value is echoed into customData and has no other effect.

## `leadsEnrichmentDepartments` (type: `array`):

Retained for backward compatibility. Echoed into customData; no lead data is collected.

## `focusOnPaidAds` (type: `boolean`):

When enabled, a page that parsed zero ads and zero shopping products is fetched once more on the same proxy before the row is pushed. Google injects ad markup with JavaScript, so paidResults is frequently empty over the HTTP path.

## `countryCode` (type: `string`):

Sent to Google as the gl parameter. It also decides which country's autocomplete completions and related searches you get. The request always goes to www.google.com.

## `searchLanguage` (type: `string`):

Restricts results to pages written in one language. Sent as lr=lang\_xx.

## `languageCode` (type: `string`):

Sent as hl. It drives the language of the questions, related searches and autocomplete completions you get back, so set it to the market you are researching.

## `locationUule` (type: `string`):

Google UULE location code, sent as the uule parameter.

## `forceExactMatch` (type: `boolean`):

Wraps the whole seed in quotation marks for an exact-phrase search. Narrows results and usually shrinks the related-searches block.

## `site` (type: `string`):

Adds site:example.com. Takes precedence over 'Related to site'.

## `relatedToSite` (type: `string`):

Adds related:example.com. Ignored when 'Only this site' is filled in.

## `wordsInTitle` (type: `array`):

Each entry becomes intitle:"word".

## `wordsInText` (type: `array`):

Each entry becomes intext:"word".

## `wordsInUrl` (type: `array`):

Each entry becomes inurl:"word".

## `quickDateRange` (type: `string`):

d7, w2, m6, y1 ... sent as tbs=qdr:. Do not combine with the before/after dates.

## `beforeDate` (type: `string`):

Absolute (2024-05-03) or relative (8 days, 3 months). UTC.

## `afterDate` (type: `string`):

Absolute (2024-05-03) or relative (8 days, 3 months). UTC.

## `fileTypes` (type: `array`):

OR-joined into filetype: operators.

## `mobileResults` (type: `boolean`):

Switches the user-agent pool. Note that searchQuery.device is reported as DESKTOP either way, exactly as in the base actor.

## `includeUnfilteredResults` (type: `boolean`):

Sends tbs=li:1. Note this is Google's VERBATIM switch, which narrows rather than widens the result set - the name is inherited from the base actor's input schema.

## `saveHtml` (type: `boolean`):

Stores the raw HTML of every SERP page in the html field. Makes the dataset very large.

## `saveHtmlToKeyValueStore` (type: `boolean`):

Writes each SERP page to the run's key-value store. htmlSnapshotUrl then carries the record KEY of the first page.

## `includeIcons` (type: `boolean`):

Adds an icon field to organic results whose card exposes a favicon URL.

## `proxyConfiguration` (type: `object`):

Google SERP proxy by default, escalating to residential (3 retries, then sticky) when a page comes back without any result container. There is no no-proxy or datacenter tier.

## `includeRelatedSearches` (type: `boolean`):

Reads the genuine related-searches block Google renders at the bottom of the page (#botstuff). The numeric pagination strip that lives in the same node is excluded. Output: relatedSearches\[].

## `includeAutocompleteSuggestions` (type: `boolean`):

Calls Google's completion service for the seed and attaches the completions to the SERP row as autocompleteSuggestions\[]. This is one signal on the SERP row, not a standalone autocomplete export.

## `autocompleteAlphabetFanOut` (type: `boolean`):

Also requests completions for '<seed> a' through '<seed> z'. Adds 26 keyless requests per seed and typically multiplies the completion count several times over. Off by default.

## `expandPeopleAlsoAsk` (type: `boolean`):

Collects the real People-also-ask questions Google prints on the results page into expandedQuestions\[]. Answers are NOT returned: Google serves no answer-box or expansion payload to a server-side exit (measured - see the README limits section). The base peopleAlsoAsk field is left exactly as the base produced it.

## `maxQuestionsPerQuery` (type: `integer`):

Caps how many People-also-ask questions are returned per seed. Google normally prints 4-8 of them on one page. Set to 0 to skip questions entirely. Default 5.

## `includeKeywordIdeas` (type: `boolean`):

Builds one de-duplicated list of every real phrase Google returned - related searches, autocomplete completions and question texts - each with the sources it came from.

## `useFullResultsPage` (type: `boolean`):

The base actor sends udm=14 whenever AI Mode is off, and that page carries NO People-also-ask block and NO related searches. This option issues one extra request per seed WITHOUT udm=14 to read them. The primary SERP request is never modified. Turn this off to make the Actor issue exactly the same requests as the base actor - discovery fields then come back null except autocomplete.

## `deduplicateResultsAcrossPages` (type: `boolean`):

The base actor makes page N's row re-contain the organic results of pages 1..N, and every page is a separate charged row. Default false keeps that behaviour byte-for-byte. Set true to make each row carry only the results of its own page.

## `saveQuestionsToSeparateDataset` (type: `boolean`):

Also writes one row per question to a named dataset '<runId>-questions'. The mirror is never charged; the SERP rows in the default dataset are unaffected.

## Actor input object example

```json
{
  "queries": "javascript\ntypescript\npython",
  "resultsPerPage": 100,
  "maxPagesPerQuery": 1,
  "aiMode": "aiModeOff",
  "perplexitySearch": {
    "enablePerplexity": false,
    "searchRecency": "",
    "returnImages": false,
    "returnRelatedQuestions": false
  },
  "chatGptSearch": {
    "enableChatGpt": false
  },
  "maximumLeadsEnrichmentRecords": 0,
  "leadsEnrichmentDepartments": [],
  "focusOnPaidAds": false,
  "countryCode": "us",
  "languageCode": "en",
  "forceExactMatch": false,
  "wordsInTitle": [],
  "wordsInText": [],
  "wordsInUrl": [],
  "fileTypes": [],
  "mobileResults": false,
  "includeUnfilteredResults": false,
  "saveHtml": false,
  "saveHtmlToKeyValueStore": true,
  "includeIcons": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  },
  "includeRelatedSearches": true,
  "includeAutocompleteSuggestions": true,
  "autocompleteAlphabetFanOut": false,
  "expandPeopleAlsoAsk": true,
  "maxQuestionsPerQuery": 5,
  "includeKeywordIdeas": true,
  "useFullResultsPage": true,
  "deduplicateResultsAcrossPages": false,
  "saveQuestionsToSeparateDataset": false
}
```

# Actor output Schema

## `overview` (type: `string`):

Compact table: search query, results total, related queries, AI overview, paid results/products, organic results and People-also-ask.

## `organicResults` (type: `string`):

Search query and the organic listings parsed from the SERP.

## `paidResults` (type: `string`):

Search query and any text ads parsed from the SERP.

## `aiModeResults` (type: `string`):

Search query and the AI Overview, when aiMode is not off and a SerpApi key is supplied.

## `perplexitySearchResults` (type: `string`):

Search query and customData — Perplexity is not implemented; the input object is only echoed back.

## `chatGptSearchResults` (type: `string`):

Search query and customData — ChatGPT search is not implemented; the input object is only echoed back.

## `allFields` (type: `string`):

Every field on the dataset row, including the discovery fields (relatedSearches, autocompleteSuggestions, expandedQuestions, keywordIdeas) and diagnostics (discoveryNotes, errorReason).

## `questionsAndKeywords` (type: `string`):

One row per seed with the discovery counts and the merged, deduplicated keywordIdeas\[] list.

## `relatedSearches` (type: `string`):

Seed and Google's genuine related-searches block (#botstuff), or null when Google served none.

## `expandedQuestions` (type: `string`):

Seed and the real People-also-ask question text collected from the results page (answers are not returned).

## `autocompleteSuggestions` (type: `string`):

Seed and the completions returned by Google's autocomplete service.

## `htmlSnapshots` (type: `string`):

Raw HTML of each fetched SERP page, saved to the key-value store only when 🗄️ Store page HTML in the key-value store (saveHtmlToKeyValueStore) is on — the default. Each dataset row's htmlSnapshotUrl field carries the matching record's key.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": `javascript
typescript
python`,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "GOOGLE_SERP"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("simpleapi/google-search-results-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": """javascript
typescript
python""",
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["GOOGLE_SERP"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("simpleapi/google-search-results-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": "javascript\\ntypescript\\npython",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  }
}' |
apify call simpleapi/google-search-results-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,simpleapi/google-search-results-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vNYo6FDdYgCF44x4d/builds/GgvIO9t7wcqQGJAHy/openapi.json