# Substack Newsletter Scraper (`boundary/substack-newsletter-scraper`) Actor

Scrape Substack newsletter posts — titles, content, reactions, comments, tags, and author data. Supports custom domains. No login needed.

- **URL**: https://apify.com/boundary/substack-newsletter-scraper.md
- **Developed by:** [Boundary](https://apify.com/boundary) (community)
- **Categories:** News, Social media
- **Stats:** 9 total users, 0 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 posts

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Substack Newsletter Scraper

Scrape posts from any Substack newsletter — titles, full content, reactions, comments, tags, author data, and more. Works with custom domains. No login or API key needed.

### Features

- Extracts **full post data**: title, subtitle, full HTML content, description, word count, cover image, tags
- Scrapes **engagement metrics**: reactions, comments (with text and reactions), restacks
- Includes **author info**: name, handle, bio, profile image, Twitter handle
- Includes **newsletter info**: name, subdomain, custom domain, logo, description
- Captures **podcast and audio**: podcast URLs, TTS audio URLs when available
- Accepts **flexible input**: Substack URLs or custom domains — just paste from your browser
- Handles **pagination** automatically — scrape hundreds or thousands of posts
- **No authentication required** — uses Substack's public API
- Supports **multiple newsletters** in a single run
- Lightweight — uses HTTP requests only, no browser needed

### How much does it cost?

With Substack Newsletter Scraper, you can scrape **1,000 posts for $2.00** in Apify platform credits ($0.002 per post).

Running on the free plan? You can scrape several hundred posts per month at no cost.

### Input

| Field | Description | Default |
|-------|-------------|---------|
| **Newsletter URLs** | Paste newsletter URLs from your browser (e.g. `https://www.lennysnewsletter.com`). Works with both Substack subdomains (`name.substack.com`) and custom domains. | Required |
| **Post Limit** | Maximum posts per newsletter (0 = unlimited) | 100 |
| **Not Older Than** | Stop scraping when posts are older than this date (YYYY-MM-DD) | No limit |

### Output example

Each post is saved as a JSON object:

```json
{
  "postId": 190767098,
  "title": "BOOM: Senate Votes to Block Private Equity from Buying Homes",
  "subtitle": "For 15 years, elites have fought against homeownership...",
  "slug": "boom-senate-votes-to-block-private",
  "url": "https://www.thebignewsletter.com/p/boom-senate-votes-to-block-private",
  "date": "2026-03-13T22:39:57.833Z",
  "updatedAt": "2026-03-13T23:39:58.809Z",
  "type": "newsletter",
  "audience": "everyone",
  "description": "For 15 years, elites have fought against homeownership...",
  "contentHtml": "<p>Until the Iran war, the main political pressure on...</p>",
  "contentText": "Until the Iran war, the main political pressure on...",
  "wordCount": 2040,
  "coverImageUrl": "https://substackcdn.com/image/fetch/...",
  "sectionName": null,
  "reactionCount": 320,
  "reactions": { "❤": 320 },
  "commentCount": 8,
  "restackCount": 42,
  "tags": ["Private equity", "housing", "Wall Street"],
  "authorName": "Matt Stoller",
  "authorHandle": "mattstoller",
  "authorBio": "Matt Stoller is Research Director for the American Economic...",
  "authorImageUrl": "https://bucketeer-e05bbc84-baa3-437e.../915fa1b4-....png",
  "authorTwitter": null,
  "newsletterName": "BIG by Matt Stoller",
  "newsletterSubdomain": "mattstoller",
  "newsletterDomain": "www.thebignewsletter.com",
  "newsletterLogoUrl": "https://bucketeer-e05bbc84-baa3-437e.../c12cbcf7-....png",
  "newsletterDescription": "The history and politics of monopoly power.",
  "podcastUrl": null,
  "podcastDuration": null,
  "ttsAudioUrl": "https://substack-video.s3.amazonaws.com/video_upload/post/...",
  "comments": [
    {
      "name": "marku52",
      "body": "I don't understand how they intend to keep Wall Street from...",
      "date": "2026-03-14T00:36:45.063Z",
      "reactions": { "❤": 20 }
    }
  ],
  "scrapedAt": "2026-03-14T11:40:28.777Z"
}
```

### All output fields

| Field | Type | Description |
|-------|------|-------------|
| `postId` | number | Substack post ID |
| `title` | string | Post title |
| `subtitle` | string | Post subtitle |
| `slug` | string | URL slug |
| `url` | string | Full post URL |
| `date` | string | Publish date (ISO 8601) |
| `updatedAt` | string | Last update date (ISO 8601) |
| `type` | string | Post type (`newsletter` or `podcast`) |
| `audience` | string | `everyone` = free, `only_paid` = paywalled, `only_free` = free-only |
| `description` | string | Short description / preview text |
| `contentHtml` | string | Full HTML content for free posts. Teaser (first few paragraphs) for paid posts. |
| `contentText` | string | Truncated plain text (~200 chars) |
| `wordCount` | number | Word count |
| `coverImageUrl` | string | Cover image URL |
| `sectionName` | string | Newsletter section name |
| `reactionCount` | number | Total reaction count |
| `reactions` | object | Reactions by emoji (e.g. `{"❤": 320}`) |
| `commentCount` | number | Number of comments |
| `restackCount` | number | Number of restacks |
| `tags` | array | Post tags |
| `authorName` | string | Author display name |
| `authorHandle` | string | Author handle |
| `authorBio` | string | Author bio |
| `authorImageUrl` | string | Author profile image URL |
| `authorTwitter` | string | Author Twitter/X handle |
| `newsletterName` | string | Newsletter name |
| `newsletterSubdomain` | string | Substack subdomain |
| `newsletterDomain` | string | Custom domain (or Substack subdomain) |
| `newsletterLogoUrl` | string | Newsletter logo URL |
| `newsletterDescription` | string | Newsletter tagline |
| `podcastUrl` | string | Podcast audio URL |
| `podcastDuration` | number | Podcast duration in seconds |
| `ttsAudioUrl` | string | AI-generated text-to-speech audio URL |
| `comments` | array | Comments with name, body, date, and reactions |
| `scrapedAt` | string | When the data was scraped (ISO 8601) |

### About paid content

By default, this scraper only collects **free posts** (`audience: "everyone"`). Paid/paywalled posts are skipped.

If you turn on **"Include Paid Posts"**, the scraper will attempt to fetch paywalled posts too. However, Substack's public API has limitations for paid content:

- **Content is partial** — you'll get a teaser (first few paragraphs), not the full article. This is a Substack limitation, not ours.
- **Some newsletters block paid posts entirely** — the API returns HTTP 403 for paid posts on certain newsletters. These posts are skipped automatically.
- **Comments may be empty** — if comment permissions are set to paid subscribers only, the API returns no comments.
- **No TTS audio** — text-to-speech audio is not available for paid posts.

You can always check the `audience` field in the output to see whether a post is free (`everyone`) or paywalled (`only_paid`).

### Tips

- **Custom domains work the same** — `www.lennysnewsletter.com` and `lenny.substack.com` both work. Just paste the URL from your browser.
- **Use "Not Older Than" to save costs** — for recurring scrapes, set a date cutoff so you only fetch new posts.
- **Post limit is per newsletter** — if you scrape 3 newsletters with a limit of 100, you'll get up to 300 posts total.
- **TTS audio** — many free Substack posts have AI-generated audio versions. The `ttsAudioUrl` field captures these when available.

# Actor input Schema

## `newsletterUrls` (type: `array`):

Substack newsletter URLs to scrape. Accepts substack URLs (https://newsletter.substack.com) or custom domains (https://www.lennysnewsletter.com). Paste the URL from your browser.

## `postLimit` (type: `integer`):

Maximum number of posts to scrape per newsletter, starting from the latest. Set to 0 for unlimited — this will scrape the entire newsletter archive, which may take a long time for large newsletters.

## `notOlderThan` (type: `string`):

Stop scraping when posts are older than this date (YYYY-MM-DD). Scrapes from the latest post backwards until this date is reached. Leave empty for no date cutoff.

## Actor input object example

```json
{
  "newsletterUrls": [
    "https://www.lennysnewsletter.com/"
  ],
  "postLimit": 100
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "newsletterUrls": [
        "https://www.lennysnewsletter.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("boundary/substack-newsletter-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "newsletterUrls": ["https://www.lennysnewsletter.com/"] }

# Run the Actor and wait for it to finish
run = client.actor("boundary/substack-newsletter-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "newsletterUrls": [
    "https://www.lennysnewsletter.com/"
  ]
}' |
apify call boundary/substack-newsletter-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,boundary/substack-newsletter-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/lOanh3pdcClwAjQto/builds/RxcIXATFbwNrgey81/openapi.json