# WSJ Articles Scraper (`parseforge/wsj-articles-scraper`) Actor

Scrapes the latest Wall Street Journal articles from the public headline feed. Returns each article as a flat row with headline, URL, and publication timestamp.

- **URL**: https://apify.com/parseforge/wsj-articles-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** News
- **Stats:** 22 total users, 6 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### WSJ Articles Scraper

**Scrape WSJ articles from the latest headlines feed, up to a million per run.** Each article returns its headline, URL, and publication timestamp. No login or API key required. Export to CSV, JSON, Excel, or XML.

The Wall Street Journal paywall blocks casual reading and its official API is not built for bulk collection. This actor reads the public article feed directly, collecting the latest headlines and metadata without authentication. Set a maximum article count and let the scraper pull every available headline into one clean dataset.

| Who uses it | What they scrape Wall Street Journal for |
|---|---|
| Financial analysts | Monitor breaking business and market headlines as they appear on WSJ. |
| News aggregators | Pull the latest WSJ articles into a daily news digest or newsletter. |
| Data journalists | Track which topics and companies the Wall Street Journal is covering over time. |
| Academic researchers | Build a corpus of WSJ headlines for media bias or framing studies. |

### What it does

This Actor collects the latest Wall Street Journal articles from the public feed and returns each one as a flat row with its headline, URL, and timestamp.

- 📰 **Latest headlines feed:** scrapes the most recent articles as they appear on the public WSJ listing.
- 🔢 **Configurable volume:** set a maximum number of articles from 1 up to 1,000,000 per run.
- 📄 **Flat row output:** every article is one row with a consistent schema, ready for analysis.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Wall Street Journal data

**📈 Monitor market-moving headlines.**

A financial analyst runs the scraper hourly to catch every WSJ headline about mergers, earnings, and Fed policy before the market reacts.

**📰 Build a daily news roundup.**

A newsletter curator pulls the latest 50 WSJ articles each morning and filters them by topic for a subscriber email.

**🔍 Track corporate coverage.**

A PR professional collects WSJ articles mentioning a set of Fortune 500 companies to measure share of voice over a quarter.

**📚 Feed a research dataset.**

An academic researcher scrapes WSJ headlines daily for six months to study how business media frames economic uncertainty.

### Why choose this scraper

| | What you get |
|---|---|
| **No paywall bypass** | Reads the same public headline feed any visitor sees, so no login or subscription is needed. |
| **Fixed schema** | Every article arrives with the same fields, making it safe for automated pipelines and dashboards. |
| **High throughput** | Collect up to a million articles in a single run, limited only by what the feed currently offers. |

### How it compares

No other Store actor targets Wall Street Journal the same way, so the honest comparison is with the alternatives teams actually weigh.

| | WSJ Articles Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Wall Street Journal changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor by setting a maximum article count. The scraper reads the latest feed until it reaches your limit, so only the newest headlines land in your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10
}
```

A larger pull:

```json
{
 "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [WSJ Articles Scraper](https://apify.com/parseforge/wsj-articles-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Wall Street Journal through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/wsj-articles-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting fewer articles than the maxItems I set?**

The actor stops when it reaches your maxItems limit or when there are no more articles in the feed, whichever comes first. If the feed has fewer articles than your limit, you will get everything available.

**Why do I only get headlines and not the full article text?**

This actor reads the public headline feed, which does not include full body text. Full articles are behind the WSJ paywall and cannot be collected without a subscription and a different approach.

**The run finished but my dataset is empty. What happened?**

Check that the WSJ public feed is reachable from your network. A temporary site block or an IP restriction can cause an empty result. Try running from a different Apify datacenter or with a proxy.

**Can I scrape articles from a specific WSJ section like Markets or Tech?**

The actor currently reads the main latest-articles feed. Section-specific feeds are not supported in this version. You can filter by headline text after the run.

### FAQ

| Question | Answer |
|---|---|
| Does this scraper bypass the WSJ paywall? | No. It reads only the public article listing that any visitor can see. Full article body text behind the paywall is not collected. |
| Do I need a WSJ subscription or API key? | No subscription or API key is required. The actor reads the publicly accessible headline feed. |
| What data does each article row contain? | Each row includes the article headline, the WSJ URL, and the publication timestamp. The exact fields are shown in the sample output on the actor's page. |
| How many articles can I scrape in one run? | You set the maximum with the maxItems input, anywhere from 1 to 1,000,000. The actual number returned depends on how many articles are currently in the feed. |
| How often is the WSJ article feed updated? | The feed updates continuously throughout the day as new articles are published. Running the actor more frequently catches new headlines sooner. |
| Can I filter articles by topic or keyword? | The actor collects the latest articles without keyword filtering. You can filter the resulting dataset after the run using the headline text. |
| Can I get the full body text of articles? | No. This actor collects only what is available in the public headline feed. Full article text is behind the WSJ paywall and is not included. |
| What export formats are supported? | You can export your dataset in CSV, JSON, Excel, or XML from the Apify platform after the run completes. |
| Is this actor affiliated with the Wall Street Journal? | No. This is an independent scraper built on the Apify platform. It is not affiliated with or endorsed by Dow Jones or the Wall Street Journal. |
| Can I schedule this scraper to run automatically? | Yes. Use Apify's scheduler to run the actor on a recurring interval, such as every hour or every morning. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Dow Jones & Company, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

Maximum number of articles to collect per run.

## Actor input object example

```json
{
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/wsj-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "maxItems": 10 }

# Run the Actor and wait for it to finish
run = client.actor("parseforge/wsj-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10
}' |
apify call parseforge/wsj-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/wsj-articles-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Lvc6v4D2aiUHJug7T/builds/pSLRTEaltri8j4DEa/openapi.json