# Guardian Article Search Scraper (`parseforge/guardian-content-search-scraper`) Actor

Scrapes Guardian articles by search query, section, tag, or date range. Returns headline, URL, publication date, and optional full body text, liveblog blocks, references, and rights metadata.

- **URL**: https://apify.com/parseforge/guardian-content-search-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** News, Business, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $29.62 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Guardian Article Search Scraper

**Scrape Guardian articles from any search query, section, or date range, up to a million per run.** Every article comes with its headline, URL, publication date, and full body text when you need it. No API key or login. Export to CSV, JSON, Excel, or XML.

The Guardian's official API needs a developer key and rate-limits you. This reads the public article search and archive directly, filtered by section, tag, contributor, date, or production office, and returns each match in one fixed schema. It is the fastest way to turn Guardian journalism into a dataset for research, media monitoring, or content analysis.

| Who uses it | What they scrape The Guardian for |
|---|---|
| Market researchers | Which topics and narratives The Guardian is covering this week |
| Media monitoring teams | Every article that mentions a brand, person, or issue |
| Academic researchers | A clean corpus of articles for text analysis or discourse studies |
| Content marketers | What The Guardian has published on a subject before pitching a story |
| Data journalists | A structured archive of articles to cross-reference with other datasets |

### What it does

This Actor collects Guardian articles by search query, section, tag, or date range, and returns each one as a flat row with headline, URL, publication date, and optional full body text.

- 🔍 **Search queries:** free-text search with AND, OR, NOT, and quoted phrases.
- 📅 **Date range:** fromDate and toDate filters for historical or recent coverage.
- 🏷️ **Section and tag filters:** narrow to world, business, football, climate change, or any Guardian tag.
- 📄 **Full body text:** include the complete article text and HTML when you need it.
- 🧩 **Liveblog blocks:** include structured blocks for liveblogs, body content, and embedded media.
- 🔗 **References and rights:** include external IDs like ISBN or IMDb, and syndication rights metadata.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with The Guardian data

**📈 Track topics and trends.**

A market researcher runs a query for 'electric vehicles' with a date range of the last month, then exports the results to CSV to see which narratives are gaining traction.

**📰 Monitor brand mentions.**

A PR team sets a query for their company name and a tag filter for 'business', then schedules the Actor daily to catch every new article.

**🎓 Build a research corpus.**

An academic uses the section filter for 'environment' and includes full body text, then downloads the dataset as JSON for text analysis.

**📊 Compare coverage across editions.**

A data journalist filters by production office to compare how the UK, US, and Australia editions cover the same story.

**📚 Archive a section.**

A librarian sets the section to 'books' and maxItems to 100000, then exports the entire archive to XML for preservation.

### Why choose this scraper

| | What you get |
|---|---|
| **No API key** | The Guardian's official API requires a developer key and rate limits. This Actor reads the public search directly. |
| **Up to a million articles** | Set maxItems as high as you need for large-scale archive pulls. |
| **Full body text** | Include the complete article text and HTML in each record, or disable it for lighter runs. |
| **Structured liveblogs** | Include the blocks object for liveblog updates, body blocks, and embedded media. |
| **References and rights** | Include external IDs like ISBN, MusicBrainz, or IMDb, and syndication rights metadata. |

### How it compares

No other Store actor targets The Guardian the same way, so the honest comparison is with the alternatives teams actually weigh.

| | Guardian Article Search Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When The Guardian changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor from a search query, section, tag, or date range, alone or together, and filters run as each article is read so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10
}
```

A larger pull:

```json
{
 "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.0395 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $3.95 |
| 1,000 results | $39.50 |
| 10,000 results | $395.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Guardian Article Search Scraper](https://apify.com/parseforge/guardian-content-search-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to The Guardian through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/guardian-content-search-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check your query and filters. If you set a tag that does not exist or a date range with no articles, the Actor returns an empty dataset. Try removing filters one by one to isolate the issue.

**Why is the run slow?**

Including full body text and liveblog blocks increases the amount of data per article and slows the run. Disable includeBodyText and includeBlocks if you only need metadata.

**Why did the run stop before reaching maxItems?**

The Actor stops when there are no more matching articles. Your filters may be too restrictive. Broaden the query, remove the tag, or widen the date range.

**Why are some articles missing body text?**

Some Guardian content, like interactive pieces or videos, may not have a traditional body text field. The Actor returns whatever is available in the public data.

**Can I scrape comments?**

No. This Actor only scrapes articles from The Guardian's search and archive. It does not collect reader comments.

### FAQ

| Question | Answer |
|---|---|
| Do I need a Guardian API key? | No. This Actor reads the public Guardian article search and archive directly, so no API key or login is required. |
| Can I get the full text of each article? | Yes. Set includeBodyText to true to include the complete article body text and HTML in each record. Disable it for lighter records. |
| How many articles can I scrape in one run? | You can set maxItems up to 1,000,000 articles per run. The Actor will collect until it reaches that number or runs out of matching articles. |
| Can I filter by section or tag? | Yes. Use the section filter to pick one of 30 sections like world, business, or football. Use the tag filter for specific tags like environment/climate-change or profile/jonathan-freedland. |
| Can I search with Boolean operators? | Yes. The query field supports AND, OR, NOT, and quoted phrases, like The Guardian's own search. |
| Can I filter by date? | Yes. Set fromDate and toDate in YYYY-MM-DD format to include only articles published within that range. |
| Can I filter by production office? | Yes. Choose UK, US, Australia, or International to limit results to a specific edition. |
| Can I sort results? | Yes. Use orderBy to sort by newest, oldest, or relevance to your search query. |
| Can I include liveblog blocks? | Yes. Set includeBlocks to true to include the structured blocks object for liveblog updates, body blocks, and embedded media. |
| Can I include references and rights? | Yes. Set includeReferences to true to include external IDs like ISBN or IMDb. Set includeRights to true to include syndication and reuse rights metadata. |
| What export formats are supported? | You can export your dataset to CSV, JSON, Excel, or XML from the Apify platform. |
| Is this legal? | You are responsible for complying with The Guardian's terms of service and applicable laws. This Actor only accesses publicly available data. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Guardian News & Media Limited. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `query` (type: `string`):

Free-text search across The Guardian's archive. Supports operators: AND, OR, NOT, quoted phrases. Leave empty to browse the latest articles.

## `maxItems` (type: `integer`):

How many articles to collect per run.

## `sectionCaptionFilters` (type: `string`):

Refine the search by section, tag, contributor, date, or production office.

## `section` (type: `string`):

Pick one section. Choose 'all' to search every section.

## `tag` (type: `string`):

Filter by tag (e.g. 'environment/climate-change', 'football/premierleague', 'profile/jonathan-freedland'). Use comma to combine.

## `fromDate` (type: `string`):

Earliest publication date (YYYY-MM-DD). Articles older than this are excluded.

## `toDate` (type: `string`):

Latest publication date (YYYY-MM-DD). Articles newer than this are excluded.

## `productionOffice` (type: `string`):

Edition responsible for the article: UK, US, AU, or international.

## `orderBy` (type: `string`):

Sort articles by newest, oldest, or relevance to the search query.

## `lang` (type: `string`):

Limit to articles in a specific language code (en, fr, es, de, ar, etc.).

## `starRating` (type: `integer`):

Only include reviews with this minimum star rating (1-5). Best for film, TV, music, restaurant reviews.

## `includeBodyText` (type: `boolean`):

Include the article's full body text and HTML in each record. Disable for lighter records.

## `includeBlocks` (type: `boolean`):

Include the structured 'blocks' object (liveblog updates, body blocks, embedded media) for each article. Useful for liveblog scraping.

## `includeReferences` (type: `boolean`):

Include the article's external reference IDs (ISBN, MusicBrainz, IMDb, etc.) when present.

## `includeRights` (type: `boolean`):

Include each article's syndication and reuse rights metadata.

## Actor input object example

```json
{
  "query": "",
  "maxItems": 10,
  "section": "all",
  "tag": "",
  "productionOffice": "any",
  "orderBy": "newest",
  "lang": "",
  "includeBodyText": true,
  "includeBlocks": false,
  "includeReferences": false,
  "includeRights": false
}
```

# Actor output Schema

## `overview` (type: `string`):

Overview of scraped data

## `fullData` (type: `string`):

Complete dataset

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "tag": "",
    "lang": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/guardian-content-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "tag": "",
    "lang": "",
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/guardian-content-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "tag": "",
  "lang": ""
}' |
apify call parseforge/guardian-content-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/guardian-content-search-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/S4oa9sMsH7VKm7H6m/builds/K8F4GYdCtrXDGQtbv/openapi.json