# Yahoo News Scraper (`searchapi/yahoo-news-scraper`) Actor

Scrapes Yahoo News for any query. Extracts the full canonical news-vertical schema: title, source, snippet, dates, thumbnail, images, category, subcategory, tags, language, authors, sponsored flag, and more.

- **URL**: https://apify.com/searchapi/yahoo-news-scraper.md
- **Developed by:** [Search API](https://apify.com/searchapi) (community)
- **Categories:** News, Automation, Developer tools
- **Stats:** 1 total users, 0 monthly users, 41.4% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.99 / 1,000 search results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Yahoo News Scraper

Search public Yahoo News results and export normalized, API-ready article records. The Actor uses Yahoo's server-rendered result cards, decodes Yahoo tracking redirects into real article URLs, supports one query or a fair batch, and completes typical runs in seconds without launching a browser.

### Features

- Single-query and deduplicated multiple-query modes
- Fair interleaving and global/per-query limits
- Real Yahoo pagination with up to 50 pages per query
- US, UK, Canada, Australia, and India regional Yahoo search hosts
- Source-backed past-day, past-week, past-month, and custom date filters
- Yahoo relevance order or newest ordering across fetched candidates
- Stable article IDs and canonical URL deduplication
- Publisher and “via Yahoo News/Finance/Sports” separation
- Thumbnails, dimensions, alternative text, favicons, snippets, highlights, and normalized publication times
- Direct, Apify Residential/Datacenter, and custom proxy operation
- Bounded retries, status/content validation, block detection, and clean no-results handling

### Input

Provide `query`, `queries`, or both. Duplicate queries are removed case-insensitively.

| Field | Description | Default |
| --- | --- | --- |
| `query` | One Yahoo News query | — |
| `queries` | Up to 20 queries | — |
| `maxItems` | Maximum records across all queries | `100` |
| `maxItemsPerQuery` | Optional cap for each query | fair share of `maxItems` |
| `maxPages` | Maximum pages inspected per query, 1–50 | `10` |
| `country` | Two-letter country code | `us` |
| `language` | Language or language-country locale | `en` |
| `timeRange` | `any`, `day`, `week`, or `month` | `any` |
| `startDate` | Inclusive `YYYY-MM-DD` publication date | — |
| `endDate` | Inclusive `YYYY-MM-DD` publication date | — |
| `sortBy` | `relevance` or `newest` | `relevance` |
| `maxConcurrency` | Concurrent Yahoo page requests, 1–10 | `5` |
| `maxRequestRetries` | Temporary retries, 0–3 | `2` |
| `requestTimeoutSecs` | Per-request timeout | `30` |
| `proxyConfiguration` | Apify or custom proxy configuration | direct |

Example batch input:

```json
{
  "queries": ["artificial intelligence", "climate technology"],
  "maxItems": 20,
  "maxItemsPerQuery": 10,
  "maxPages": 5,
  "country": "us",
  "language": "en",
  "timeRange": "week",
  "sortBy": "newest",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}
```

Set `maxItems` to `1` for a single search item.

### Output

The dataset schema documents 52 possible fields. A normal result can include:

- stable ID and global, query, page, and page-card positions
- decoded article URL and destination domain
- article title, snippet, description, and highlighted query terms
- publisher, original Yahoo source label, and Yahoo distribution property
- thumbnail URL, alternative text, title, dimensions, and aspect ratio
- publisher favicon
- normalized publication time, exact Yahoo label, relative time, and age in seconds
- Yahoo-hosted and explicit sponsorship flags
- exact query, region, language, filters, ordering, retrieval method, and aggregate search metadata

Unavailable optional values are omitted instead of being emitted as `null`, empty strings, zero counts, or false flags. The Actor does not fabricate authors, categories, paywall status, engagement counts, or full article content that Yahoo's search cards do not expose.

### Filter and ordering behavior

Yahoo's former `age` and `s=4` URL parameters no longer change the current News Search cards. The Actor therefore verifies Yahoo's displayed publication labels, normalizes them against one stable run timestamp, and enforces recency/custom dates after extraction. Records without a usable Yahoo timestamp are excluded when a date filter is active.

`newest` sorts the candidates fetched within `maxPages`; it does not claim to sort Yahoo's entire index. `relevance` preserves Yahoo's page and card order.

Country codes `us`, `gb`, `ca`, `au`, and `in` select their regional Yahoo search hosts. Other valid two-letter codes use the global host with locale headers and remain visible in dataset context.

### Performance and resilience

Yahoo serves the result cards in the initial HTML, so the Actor fetches and parses that response directly. Independent pages and queries run concurrently with one stable proxy session per query. Responses are checked for status, HTML content type, final Yahoo host, maximum size, challenge content, and current result structure before parsing or storing data.

Challenge pages are never retried or stored. Temporary rate limits, network errors, and server failures use bounded exponential backoff.

### Proxy notes

Use Apify Residential or Datacenter proxies when your direct network is rate-limited. `GOOGLE_SERP` cannot be used because it only supports selected Google Search/Shopping requests, not Yahoo.

### Local run

```bash
npm ci
apify run --purge --input-file INPUT.json
```

Results are written to `storage/datasets/default`.

### Limitations

Yahoo News search cards contain summaries rather than full publisher article bodies. Some records omit thumbnails, favicons, highlights, or “via” labels because Yahoo does not supply them. Search inventory and regional ranking can change over time. The Actor does not bypass authentication, paywalls, CAPTCHAs, or access controls.

# Actor input Schema

## `query` (type: `string`):

One query to enter in Yahoo News Search.

## `queries` (type: `array`):

Optional batch of up to 20 queries. Duplicate queries are removed and results are interleaved fairly.

## `maxItems` (type: `integer`):

Maximum records across all submitted queries.

## `maxItemsPerQuery` (type: `integer`):

Optional per-query cap. By default each query receives a fair share of maxItems.

## `maxPages` (type: `integer`):

Maximum Yahoo result pages inspected for each query. Each page currently contains about ten cards.

## `country` (type: `string`):

Two-letter country code. US, GB, CA, AU, and IN use their regional Yahoo News Search hosts; other codes use Yahoo's global host and locale headers.

## `language` (type: `string`):

Two-letter language code used in Yahoo's live language parameter and request locale.

## `timeRange` (type: `string`):

Filter Yahoo's displayed publication timestamps after extraction. Legacy 1d, 7d, and 30d inputs remain accepted.

## `startDate` (type: `string`):

Optional inclusive date in YYYY-MM-DD format, enforced against Yahoo's displayed timestamp.

## `endDate` (type: `string`):

Optional inclusive date in YYYY-MM-DD format, enforced against Yahoo's displayed timestamp.

## `sortBy` (type: `string`):

Preserve Yahoo relevance order or sort fetched candidates by Yahoo's displayed publication timestamp.

## `maxConcurrency` (type: `integer`):

Maximum Yahoo result pages fetched at once.

## `maxRequestRetries` (type: `integer`):

Bounded retries for temporary network, rate-limit, and server errors. Challenge pages are never retried.

## `requestTimeoutSecs` (type: `integer`):

Timeout for each Yahoo result-page request.

## `proxyConfiguration` (type: `object`):

Direct, Apify Residential, or custom proxy settings. GOOGLE\_SERP is not applicable to Yahoo.

## `debug` (type: `boolean`):

Log safe structural diagnostics without HTML, response bodies, tracking URLs, or secrets.

## Actor input object example

```json
{
  "query": "artificial intelligence",
  "maxItems": 100,
  "maxPages": 10,
  "country": "us",
  "language": "en",
  "timeRange": "any",
  "sortBy": "relevance",
  "maxConcurrency": 5,
  "maxRequestRetries": 2,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "debug": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "artificial intelligence",
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("searchapi/yahoo-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "artificial intelligence",
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("searchapi/yahoo-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "artificial intelligence",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call searchapi/yahoo-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,searchapi/yahoo-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/kvH4nKnFfJ74rzDH6/builds/BXnYLShe2rzgzWFvx/openapi.json