# Google News Scraper (`cirkit/google-news-scraper`) Actor Fast Google News scraper. Search by keyword with time-range filter, scrape top stories per country, topic sections, geo headlines, or topic IDs. Extracts title, publisher, date, snippet, image, related coverage. Optional decode of the wrapped Google URL into the real publisher URL. - **URL**: https://apify.com/cirkit/google-news-scraper.md - **Developed by:** [Crikit](https://apify.com/cirkit) (community) - **Categories:** News, Automation - **Stats:** 7 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Google News Scraper **Scrape Google News** by keyword with a time-range filter, by country top stories, by topic section, or by geographic headline feed. This **Google News scraper** returns article title, publisher name, publish date, snippet, image, and related coverage — and can **decode Google's wrapped redirect URLs** into the real publisher links. No Google News API key, no login, and no headless browser. Results land in a structured dataset you can export to JSON, CSV, Excel, or XML, or pull straight from the Apify API. ### What is Google News Scraper? Google News Scraper is an [Apify Actor](https://apify.com/actors) that extracts news articles, top stories, and topic feeds from [Google News](https://news.google.com) and returns them as clean, structured records. URL decoding is the feature that makes this usable. Google News links are obfuscated redirects; without resolving them you cannot attribute coverage to a domain or fetch the article itself. ### What data can this Google News scraper extract? Every run writes one row per article. Full field list: | Field | Type | Description | | --- | --- | --- | | `articleId` | string | Google's stable CBM-base64 identifier for the article. Matches the path segment of googleNewsUrl. | | `title` | string | Article title, with the trailing ' - Publisher Name' suffix stripped. | | `googleNewsUrl` | string | The Google-wrapped article URL as it appears in the feed (news.google.com/rss/articles/CBM...). | | `publisherUrl` | string | The decoded publisher article URL (e.g. https://www.reuters.com/...). Only populated when resolvePublisherUrls=true. | | `publisherName` | string | Display name of the publisher (e.g. Reuters, The Wall Street Journal). | | `publisherHomeUrl` | string | Publisher homepage URL from the RSS attribute. | | `publishedAt` | string | Article publication time in ISO-8601 UTC. | | `imageUrl` | string | Article thumbnail URL when Google emits one in the feed. Note: Google News rarely populates per-item images in... | | `snippetHtml` | string | Raw HTML snippet from the RSS . Contains the primary article anchor and related-coverage anchors. | | `relatedCoverage` | array | Other articles Google has clustered under the same story. | | `query` | string | The search keyword that surfaced this record. Null for non-search feeds. | | `topic` | string | Editorial topic name (BUSINESS, TECHNOLOGY, ...). Null for non-topic feeds. | | `topicId` | string | Direct Google topic ID. Null when not used. | | `geo` | string | City or place name. Null for non-geo feeds. | | `feedKind` | string | Which kind of feed this record came from. | | `locale` | object | The Google News country edition the feed was scraped from. | | `scrapedAt` | string | Feed lastBuildDate in ISO-8601 UTC at the time of scrape. | ### How to scrape Google News news articles 1. Open [Google News Scraper](https://apify.com/cirkit/google-news-scraper) and click **Try for free**. 2. Add keywords to `queries`, or use `topics`, `geos`, or `topicIds` for section feeds. 3. Set `timeRange` to restrict how far back the search reaches. 4. Turn on `resolvePublisherUrls` to decode Google's wrapped links into real publisher URLs. 5. Set `locales` for language and country, and bound with `maxResults`. 6. Click **Start** and watch rows appear live in the **Output** tab. 7. Export the dataset as JSON, CSV, Excel, or XML — or fetch it from the [Apify API](https://docs.apify.com/api/v2) once the run finishes. ### Google News Scraper input options | Input | Type | What it does | Default | | --- | --- | --- | --- | | `queries` | array | Keywords or phrases to search Google News for. Supports Google operators: site:reuters.com, intitle:, OR, -exclude,... | `["tesla"]` | | `topics` | array | Editorial topic sections by name. Each scrapes the corresponding topic RSS feed. | | | `geos` | array | Free-text city or place names for local headlines (e.g. 'San Francisco', 'New York', 'Tokyo'). Google maps the name... | | | `topicIds` | array | Direct Google News topic IDs (base64 CAAq... strings from news.google.com URLs). Use when you want a specific Google... | | | `includeTopStories` | boolean | Also scrape each locale's editorial Top Stories feed. | `false` | | `locales` | array | Country editions to scrape. Each locale fans out across every query/topic/geo input. Defaults to US English. | `see schema` | | `timeRange` | string | Restrict search results to a recency window. Only applies to keyword searches. Use 'any' (or leave blank) for no... | `"any"` | | `resolvePublisherUrls` | boolean | When true, the actor decodes each Google News article URL into the publisher's real article URL (e.g.... | `false` | | `maxResultsPerTarget` | integer | Cap on items per single feed (one query in one locale, one topic in one locale, etc.). Google itself returns ~100... | `50` | | `maxResults` | integer | Hard cap across the entire run. Leave 0 or empty for unlimited. | `0` | | `maxConcurrency` | integer | Number of RSS feeds fetched in parallel. Higher values speed up multi-input runs. | `5` | | `decodeConcurrency` | integer | Number of publisher-URL decoder calls in parallel. Only relevant when resolvePublisherUrls=true. | `3` | Proxy is configured through the standard `proxyConfiguration` object; the defaults shipped with this Actor are already tuned for the target site. ### Example output ```json { "articleId": "a1b2c3d4", "title": "Example title", "googleNewsUrl": "https://example.com/item/12345", "publisherUrl": "https://example.com/item/12345", "publisherName": "Example title", "publisherHomeUrl": "https://example.com/item/12345", "publishedAt": "2026-08-30T12:00:00.000Z", "imageUrl": "https://example.com/item/12345", "snippetHtml": "...", "relatedCoverage": [ { "...": "..." } ], "query": "...", "topic": "...", "topicId": "a1b2c3d4", "geo": "...", "feedKind": "...", "locale": { "...": "..." }, "scrapedAt": "2026-08-30T12:00:00.000Z" } ``` ### How much does it cost to scrape Google News? This Actor runs on **pay per result** pricing: **$0.0015 per article** ($1.5 per 1,000). A one-time start fee of $0.00005 applies per run. | Results | Approximate cost | | --- | --- | | 1,000 | $1.50 | | 10,000 | $15.00 | | 100,000 | $150.00 | You pay for rows you actually receive, so a search that returns nothing costs nothing beyond the start fee. Every Apify account includes free monthly usage credit, so you can trial Google News Scraper at no cost. ### What can you build with Google News data? #### Run brand and reputation monitoring Schedule keyword runs for your brand and competitors and route new articles into Slack the moment coverage appears. #### Track a story across publishers `relatedCoverage` groups how different outlets are covering the same event — the fastest way to see framing differences. #### Build a news dataset for research or NLP Titles, snippets, publishers, and timestamps across thousands of articles form a clean corpus with no API cost per call. #### Monitor an industry or a competitor set Topic and geo feeds keep a persistent watch on a sector rather than only on named entities. #### Feed a newsletter or content-curation pipeline Decoded publisher URLs mean downstream fetching and summarising work without an extra redirect-resolution step. ### Google News scraping tips and limits - `resolvePublisherUrls` costs extra requests but is essential if you plan to fetch the articles or attribute coverage by domain. - `timeRange` is the key to cheap scheduled monitoring — restrict to the last hour or day and you only pay for genuinely new coverage. - `locales` changes both language and editorial selection substantially. Set it deliberately rather than accepting a default. - Google News rotates its top stories frequently. For a complete record, schedule frequent runs rather than one large daily pull. ### Integrations and automation Run Google News Scraper on a [schedule](https://docs.apify.com/platform/schedules) to keep a Google News dataset fresh, or trigger it from your own stack through the [Apify API](https://docs.apify.com/api/v2) and the official [JavaScript](https://docs.apify.com/sdk/js) and [Python](https://docs.apify.com/sdk/python) clients. Native [integrations](https://docs.apify.com/platform/integrations) push results into Google Sheets, Slack, Airtable, Zapier, Make, GitHub, or any webhook endpoint. The Actor also works as an [MCP](https://docs.apify.com/platform/integrations/mcp) tool, so an AI agent can call it directly. ### Related scrapers | Actor | What it does | | --- | --- | | [Google Trends Scraper](https://apify.com/cirkit/google-trends-scraper) | Search demand behind the stories you are tracking. | | [Reddit Posts Scraper - Subreddit, Search, User Posts API](https://apify.com/cirkit/reddit-posts-scraper) | Where a story is being discussed on Reddit. | | [YouTube Search Scraper](https://apify.com/cirkit/youtube-search-scraper) | Video coverage of the same topics. | | [ClinicalTrials.gov Study Search Scraper](https://apify.com/cirkit/clinicaltrials-study-search-scraper) | Primary sources behind health and science reporting. | ### Frequently asked questions #### Why are Google News URLs encoded, and can I get the real link? Google News wraps outbound links in a redirect. Turn on `resolvePublisherUrls` and the Actor decodes them into the publisher's actual article URL in `publisherUrl`. #### Can I scrape news for a specific country? Yes. Use `geos` for geographic headline feeds and `locales` to set the country and language edition. #### Do I need a Google News API key? No. Google News Scraper reads publicly available Google News pages and endpoints directly, so there is no key to obtain, no OAuth app to register, and no account to connect. #### Can I export Google News data to CSV, Excel, or Google Sheets? Yes. Every run stores results in an Apify dataset that exports to JSON, JSONL, CSV, Excel, XML, or HTML with one click, and the Google Sheets integration writes rows straight into a spreadsheet. #### Can I run Google News Scraper on a schedule? Yes. Apify schedules run the Actor hourly, daily, weekly, or on any cron expression, and webhooks can notify your systems the moment a run finishes. #### Is it legal to scrape Google News? Scraping publicly available data is broadly lawful in the US and EU, but the answer depends on what you collect and how you use it. This Actor is built to gather public news headlines, snippets, and metadata that Google News publishes publicly — note that article full text remains the publisher's copyright. Personal data carries extra obligations under GDPR and CCPA. Read Apify's guide on [whether web scraping is legal](https://blog.apify.com/is-web-scraping-legal/) and take your own legal advice for your use case. ### Support and feedback Found a bug, a missing field, or a Google News page shape this Actor does not handle yet? Open an issue from the **Issues** tab on the Actor page. Feature requests and custom-scraper enquiries are welcome there too. # Actor input Schema ## `queries` (type: `array`): Keywords or phrases to search Google News for. Supports Google operators: site:reuters.com, intitle:, OR, -exclude, "exact phrase". ## `topics` (type: `array`): Editorial topic sections by name. Each scrapes the corresponding topic RSS feed. ## `geos` (type: `array`): Free-text city or place names for local headlines (e.g. 'San Francisco', 'New York', 'Tokyo'). Google maps the name to its internal geo topic ID. ## `topicIds` (type: `array`): Direct Google News topic IDs (base64 CAAq... strings from news.google.com URLs). Use when you want a specific Google News topic page. ## `includeTopStories` (type: `boolean`): Also scrape each locale's editorial Top Stories feed. ## `locales` (type: `array`): Country editions to scrape. Each locale fans out across every query/topic/geo input. Defaults to US English. ## `timeRange` (type: `string`): Restrict search results to a recency window. Only applies to keyword searches. Use 'any' (or leave blank) for no restriction. ## `resolvePublisherUrls` (type: `boolean`): When true, the actor decodes each Google News article URL into the publisher's real article URL (e.g. https://www.barrons.com/...). Adds one extra HTTP request per article. Disable for the cheapest, fastest run. ## `maxResultsPerTarget` (type: `integer`): Cap on items per single feed (one query in one locale, one topic in one locale, etc.). Google itself returns ~100 per feed; values above 200 are clamped. ## `maxResults` (type: `integer`): Hard cap across the entire run. Leave 0 or empty for unlimited. ## `maxConcurrency` (type: `integer`): Number of RSS feeds fetched in parallel. Higher values speed up multi-input runs. ## `decodeConcurrency` (type: `integer`): Number of publisher-URL decoder calls in parallel. Only relevant when resolvePublisherUrls=true. ## `proxyConfiguration` (type: `object`): Apify proxy configuration. Defaults to RESIDENTIAL US. Google News RSS has no anti-bot, but residential IPs avoid geo-routing surprises across locales. ## Actor input object example ```json { "queries": [ "tesla" ], "topics": [], "geos": [], "topicIds": [], "includeTopStories": false, "locales": [ { "hl": "en-US", "gl": "US", "ceid": "US:en" } ], "timeRange": "any", "resolvePublisherUrls": false, "maxResultsPerTarget": 50, "maxResults": 0, "maxConcurrency": 5, "decodeConcurrency": 3, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } } ``` # Actor output Schema ## `dataset` (type: `string`): Dataset of Google News article records. ## `datasetCsv` (type: `string`): Same dataset rendered as CSV. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "tesla" ], "locales": [ { "hl": "en-US", "gl": "US", "ceid": "US:en" } ], "maxResultsPerTarget": 50, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }; // Run the Actor and wait for it to finish const run = await client.actor("cirkit/google-news-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["tesla"], "locales": [{ "hl": "en-US", "gl": "US", "ceid": "US:en", }], "maxResultsPerTarget": 50, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "US", }, } # Run the Actor and wait for it to finish run = client.actor("cirkit/google-news-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "tesla" ], "locales": [ { "hl": "en-US", "gl": "US", "ceid": "US:en" } ], "maxResultsPerTarget": 50, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }' | apify call cirkit/google-news-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,cirkit/google-news-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/UkUQOoHgE4qLwUdi2/builds/YdfRJ2le4kj5QgxT4/openapi.json