# Substack Scraper — Newsletters, Posts & Creator Leads (`scrapesage/substack-scraper`) Actor

Scrape Substack: search newsletters by keyword, browse category leaderboards, pull full publication profiles (subscribers, paid pricing, podcast), posts, authors and the recommendation network. Turn creators into leads with contact emails. Monitoring mode. No API key, no browser.

- **URL**: https://apify.com/scrapesage/substack-scraper.md
- **Developed by:** [Scrape Sage](https://apify.com/scrapesage) (community)
- **Categories:** Lead generation, Social media, News
- **Stats:** 7 total users, 5 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.20 / 1,000 publication scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Substack Scraper — Newsletters, Posts & Creator Leads (Subscribers, Pricing, Emails)

> **Disclaimer:** This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by Substack Inc. or any of its subsidiaries. All trademarks mentioned are the property of their respective owners. "Substack" is referenced only to describe the publicly available website this Actor collects data from.

Extract **complete Substack data** — search newsletters by keyword, browse **category leaderboards**, and pull the fields other scrapers miss: **free-subscriber counts, paid-subscriber tiers, real paid pricing (monthly / yearly / founding), podcast details, the recommendation network, and full author profiles**. Optionally turn every creator into a **ready-to-contact lead** by crawling their own website for **contact emails, phone, and socials**.

No login, no cookies, no browser — fast first-party JSON extraction with 99%+ reliability.

### Why this Substack scraper?

Most Substack scrapers return a thin slice — a title, a date, maybe a subscriber number. This actor reads Substack's own public API and ships the **richest dataset in the category**, across newsletters, posts and authors in one run:

| Data | Typical scrapers | This actor |
|---|---|---|
| Search by keyword + category leaderboards | partial | ✅ both |
| Free subscriber count | partial | ✅ |
| Paid-subscriber tier (e.g. "Thousands of paid subscribers") | ❌ | ✅ |
| Real paid pricing — monthly / yearly / founding + currency | ❌ | ✅ |
| Accepts sponsorships (ad-sales signal) | ❌ | ✅ |
| Podcast title / description / flags | ❌ | ✅ |
| Recommendation network (who recommends whom) | ❌ | ✅ opt-in |
| Posts — reactions, restacks, comments, word count | partial | ✅ opt-in |
| Full post content (HTML + plain text) | ❌ | ✅ opt-in |
| Author profiles — followers, bio, external links, all publications | ❌ | ✅ opt-in |
| Creator **contact emails** (from their website) | ❌ | ✅ opt-in |
| Lead score (0–100) per newsletter | ❌ | ✅ |
| **No start fee** | ❌ many charge per run | ✅ pay per result only |

### Use cases

- **Creator & newsletter lead generation** — Substack creators are active buyers and sellers: they want tools, sponsors, cross-promotion, and ghostwriters. Score them by audience (`freeSubscriberCount`, `paidSubscriberTier`) and reach them directly (`supportEmail`, `contactEmails`).
- **Sponsorship & ad-sales prospecting** — find paid newsletters that `acceptsSponsorships`, ranked by subscriber tier and niche, with contact data attached.
- **Market & competitor research** — track category leaderboards, paid pricing, posting cadence, and engagement (reactions, restacks, comments) across any topic.
- **Content & trend analysis** — pull posts with full content for summarization, RAG, sentiment, and topic modeling.
- **Influencer / partnership discovery** — map the recommendation network to find who the top newsletters endorse.

### How to use

1. [Sign up for Apify](https://console.apify.com/sign-up) — the free plan is enough to try this actor.
2. Open the **Substack Scraper**, enter search queries and/or categories (or paste Substack URLs), and click **Start**.
3. Watch results stream into the dataset table.
4. **Export** as JSON, CSV, Excel, XML, or RSS — or pull results programmatically via the [Apify API](https://docs.apify.com/api/v2).

### Input

```json
{
    "searchQueries": ["artificial intelligence"],
    "categories": ["Technology", "Business"],
    "maxPublications": 200,
    "includePosts": true,
    "maxPostsPerPublication": 20,
    "includeRecommendations": true,
    "includeAuthorProfiles": true,
    "enrichContactEmails": true,
    "onlyPaidPublications": false,
    "minFreeSubscribers": 1000
}
```

- **searchQueries** — keywords to search publications (each returns full newsletter profiles).
- **categories** — category leaderboards by name (`Technology`, `Business`, `Finance`, `Culture`, `U.S. Politics`, `Food & Drink`, `Sports`, …) or numeric id.
- **startUrls** — direct publication URLs (`https://newsletter.substack.com` or a custom domain), post URLs (`.../p/the-slug`), or author profiles (`https://substack.com/@handle`).
- **maxPublications** *(default 100)* — cap on unique publications from search + categories.
- **includePosts** / **maxPostsPerPublication** / **includePostContent** — add recent posts, and optionally their full HTML + plain text.
- **includeRecommendations** *(default false)* — add each newsletter's recommendation network as a `recommends` array.
- **includeAuthorProfiles** *(default false)* — emit one author record per unique creator (followers, bio, links, all publications).
- **enrichContactEmails** *(default false)* — crawl the publication's own website (home + about/contact, max 3 pages) for emails, phone, and extra socials. Substack never exposes emails — this is the only way to get them.
- **onlyPaidPublications** / **minFreeSubscribers** — filters.
- **monitorMode** *(default false)* — emit only publications/posts not seen in previous runs (see below).

### Output

One record per newsletter (`type: "publication"`), plus optional post records (`type: "post"`) and author records (`type: "author"`):

```json
{
    "type": "publication",
    "id": 89120,
    "name": "Astral Codex Ten",
    "subdomain": "astralcodexten",
    "url": "https://www.astralcodexten.com/",
    "customDomain": "www.astralcodexten.com",
    "publicationType": "newsletter",
    "tagline": "P(A|B) = [P(A)*P(B|A)]/P(B), all the rest is commentary",
    "authorName": "Scott Alexander",
    "authorHandle": "astralcodexten",
    "authorBio": "Psychiatrist, blogger…",
    "freeSubscriberCount": 91000,
    "paidSubscriberTier": "Thousands of paid subscribers",
    "bestsellerTier": 1000,
    "isPaid": true,
    "currency": "USD",
    "monthlyPrice": 10,
    "yearlyPrice": 100,
    "foundingPrice": 300,
    "acceptsSponsorships": false,
    "hasPodcast": true,
    "supportEmail": "astralcodexten@substack.com",
    "website": "https://www.astralcodexten.com/",
    "contactEmails": ["scott@slatestarcodex.com"],
    "contactSocials": { "twitter": "https://twitter.com/slatestarcodex" },
    "recommends": [
        { "name": "Slow Boring", "subdomain": "slowboring", "url": "https://www.slowboring.com/" }
    ],
    "leadScore": 86,
    "category": "Technology",
    "searchQuery": "artificial intelligence",
    "scrapedAt": "2026-06-14T12:00:00.000Z"
}
```

### Monitoring mode

Turn on **monitorMode** to make the actor remember every publication and post it has already returned (in a named key-value store) and emit **only new ones** on the next run. Combine it with [Apify Schedules](https://docs.apify.com/platform/schedules) to:

- watch a category or keyword for **newly launched newsletters**,
- alert on **new posts** from a set of newsletters you track,
- keep a CRM topped up with **fresh creator leads**.

Monitoring mode is independent of the scheduler: Schedules decide *when* a run starts; monitoring decides *what counts as new*. Use a distinct `monitorStoreName` per tracked target to keep histories separate.

### How much does it cost to scrape Substack?

This Actor uses Apify's **pay-per-event** pricing: you are charged only for the results it delivers, with no monthly rental and no start fee. The events it can charge are:

- **Publication Scraped** - One newsletter profile: subscribers, paid pricing, podcast, sponsorship signal, support email, recommendation network + 0-100 lead score.
- **Post Scraped** - One post: date, audience, reactions, restacks, comments, word count, podcast - plus full content when enabled.
- **Author Profile Scraped** - One creator profile: followers, subscriber band, bestseller tier, external links, all publications.
- **Creator Lead Enriched** - Crawls a publication's own website for contact emails, phone & socials (the lead wedge); charged only when it runs.

The current price of each event is shown on the **Pricing** tab of this page. Set a maximum total charge on the run if you want a hard cap on spend, and use the input limits to control how much the Actor fetches.

### Automate & schedule

Run this actor on autopilot and pull results into your own stack:

- **[Apify API](https://docs.apify.com/api/v2)** — start runs, fetch datasets, and manage schedules over REST.
- **[apify-client for JavaScript](https://docs.apify.com/api/client/js/)** and **[apify-client for Python](https://docs.apify.com/api/client/python/)** — official SDKs.
- **[Schedules](https://docs.apify.com/platform/schedules)** — run it hourly/daily/weekly to monitor new newsletters, posts, or leads.
- **[Webhooks](https://docs.apify.com/platform/integrations/webhooks)** — trigger downstream actions (CRM import, Slack alert, email sequence) the moment a run finishes.

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'MY_APIFY_TOKEN' });

const run = await client.actor('scrapesage/substack-scraper').call({
    searchQueries: ['fintech'],
    categories: ['Finance'],
    maxPublications: 200,
    enrichContactEmails: true,
    onlyPaidPublications: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Got ${items.length} newsletters & creator leads`);
```

### Integrate with any app

Connect the dataset to 5,000+ apps — no code required:

- **[Make](https://docs.apify.com/platform/integrations/make)** — multi-step automation scenarios.
- **[Zapier](https://docs.apify.com/platform/integrations/zapier)** — push new creator leads straight into your CRM.
- **[Slack](https://docs.apify.com/platform/integrations/slack)** — get notified when a monitored search finds new newsletters.
- **[Google Drive / Sheets](https://docs.apify.com/platform/integrations/drive)** — auto-export every run to a spreadsheet.
- **[Airbyte](https://docs.apify.com/platform/integrations/airbyte)** — pipe results into your data warehouse.
- **[GitHub](https://docs.apify.com/platform/integrations/github)** — trigger runs from commits or releases.

### Use with AI assistants (MCP)

The output is clean, LLM-ready JSON. You can call this actor from Claude, ChatGPT, or any agent framework through the **[Apify MCP server](https://docs.apify.com/platform/integrations/mcp)** — ask your assistant to "find the top AI newsletters on Substack and list their contact emails" and let it run this scraper for you.

### Agent-ready: autonomous payments (x402 & Skyfire)

This actor is **agent-ready** — AI agents can discover it, run it, and **pay for it autonomously**, with no Apify account and no human in the loop. It uses [pay-per-event](https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event) pricing and [limited permissions](https://docs.apify.com/platform/actors/development/permissions), so it qualifies for Apify's agentic-payment standards:

- **[x402](https://docs.apify.com/platform/integrations/x402)** — an open, HTTP-native payment protocol. Agents pay per run in USDC on the Base network directly through the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp) — no account, no API key.
- **[Skyfire](https://docs.apify.com/platform/integrations/skyfire)** — agent-to-service payments for fully autonomous AI-agent workflows.

Building an AI agent, MCP tool, or autonomous data pipeline? This scraper is ready to plug in and pay as it goes.

### More scrapers from scrapesage

Build a complete **creator & event lead-gen stack**:

- **[Eventbrite Scraper](https://apify.com/scrapesage/eventbrite-scraper)** — events + organizer leads (prices, emails, socials).
- **[Sched Conference Scraper](https://apify.com/scrapesage/sched-conference-scraper)** — sessions, speakers & sponsors from Sched event sites.
- **[Whova Event Scraper](https://apify.com/scrapesage/whova-event-scraper)** — attendees, agendas, and sponsors from Whova event apps.
- **[Swapcard Exhibitor Scraper](https://apify.com/scrapesage/swapcard-exhibitor-scraper)** — exhibitor lists and booth data from Swapcard trade shows.
- **[Facebook Ad Library Scraper](https://apify.com/scrapesage/facebook-ad-library-scraper)** — competitor ad intelligence (Meta + Instagram).
- **[Google Ads Transparency Scraper](https://apify.com/scrapesage/google-ads-transparency-scraper)** — who's advertising what on Google.
- **[LinkedIn Jobs Scraper](https://apify.com/scrapesage/linkedin-jobs-scraper)** — job postings as hiring-intent signals.
- **[Bark Listing Scraper](https://apify.com/scrapesage/bark-listing-scraper)** — service-provider leads from Bark.
- **[Airbnb Scraper](https://apify.com/scrapesage/airbnb-scraper)** — listings, prices, and availability.

### Tips

- **Exhaust a niche**: combine `searchQueries` (keywords) with `categories` (leaderboards) to cover both long-tail and top newsletters; raise `maxPublications`.
- **Best leads**: set `onlyPaidPublications: true` + `minFreeSubscribers` + `enrichContactEmails: true` to get monetizing creators with real contact data and a high `leadScore`.
- **Cost control**: posts, recommendations, author profiles and email enrichment are all opt-in, so you only pay for what you turn on; email enrichment only runs for publications that actually have a website.
- **Monitoring**: combine `monitorMode` with [Schedules](https://docs.apify.com/platform/schedules) to track only new newsletters/posts.

### FAQ

**How do I scrape the top newsletters in a topic?** Put the category name in `categories` (e.g. `Technology`, `Finance`) to pull its leaderboard, and/or add keywords to `searchQueries`.

**Where do the emails come from?** Never from Substack (they don't publish creator emails). With `enrichContactEmails` on, the actor visits the newsletter's own public website and extracts publicly listed contact emails — the same thing a human visitor would see. Many newsletters also expose a `supportEmail` directly.

**Does it expose exact paid-subscriber counts?** Substack hides exact paid counts, but publishes a tier band (e.g. "Hundreds/Thousands of paid subscribers") which this actor returns as `paidSubscriberTier`, plus the exact `freeSubscriberCount` for most newsletters.

**Can I export to Google Sheets, CSV, or Excel?** Yes — one click in the dataset view, or automatically on every run via the [Google Drive integration](https://docs.apify.com/platform/integrations/drive).

**Is scraping Substack legal?** This actor collects publicly available data only. You are responsible for using the data in compliance with applicable laws (GDPR/CCPA for personal data) and Substack's terms.

**A field is null — why?** Some newsletters genuinely don't publish a price (free-only), a website, or a podcast. Fields are `null` only when the data doesn't exist, not because the scraper skipped them.

### Data & lawful use

This Actor reads only what Substack publishes to logged-out visitors: it does not log in, use cookies or session tokens, create accounts, or reach anything behind a sign-in. Names, handles, bios and engagement figures are public, but they relate to identifiable people, so treat the output as personal data. If you are in the EU or UK you are the data controller for what you do with it: have a lawful basis (usually legitimate interest for research, marketing analytics or B2B prospecting), honour access and deletion requests, and do not use the output for spam or unsolicited messaging.

Under [Apify's Standard Actor Contract](https://docs.apify.com/legal/standard-actor-contract), which governs your use of this Actor, you are the controller of any personal data in your input and output and scrapesage acts only as your processor: that data is processed solely to run your job, written only to your own Apify storage, never used for any other purpose and never shared onward. If you need help with a data-subject request that involves this Actor's output, open an issue on the Issues tab.

### Disclaimer

**This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by Substack Inc. or any of its subsidiaries. All trademarks mentioned are the property of their respective owners.**

"Substack" and any related marks are the property of their respective owners and are used here only in a descriptive, nominative sense - to identify the publicly accessible website from which this Actor collects data. This Actor is not an official Substack product, is not authorised or certified by Substack Inc., and does not distribute Substack software. It collects only publicly available information; you are responsible for ensuring your use of that data complies with applicable laws, regulations and the terms of the source website.

### Need help?

Open an issue on the actor's **Issues** tab, or visit the [Apify help center](https://help.apify.com/). Feature requests are welcome — this actor is actively maintained.

# Actor input Schema

## `searchQueries` (type: `array`):

Keywords to search Substack publications, e.g. <code>artificial intelligence</code>, <code>marketing</code>, <code>crypto</code>. Each query returns matching newsletters with full profiles. Combine with categories and start URLs.

## `categories` (type: `array`):

Browse top-ranked publications in these categories (the Substack leaderboard). Use names like <code>Technology</code>, <code>Business</code>, <code>Finance</code>, <code>Culture</code>, <code>U.S. Politics</code>, <code>Food & Drink</code>, <code>Sports</code>, or a numeric category id.

## `startUrls` (type: `array`):

Direct Substack URLs: a publication (<code>https://newsletter.substack.com</code> or a custom domain), a post (<code>.../p/the-slug</code>), or an author profile (<code>https://substack.com/@handle</code>). Mixed lists are fine.

## `maxPublications` (type: `integer`):

Cap the number of unique publications collected across all search queries and categories. Start URLs are always processed.

## `includePosts` (type: `boolean`):

For each publication, also emit its recent posts (title, date, audience, reactions, restacks, comments, word count, podcast info).

## `maxPostsPerPublication` (type: `integer`):

How many recent posts to return per publication when 'Include posts' is on.

## `includePostContent` (type: `boolean`):

Add the full post body (HTML and plain text) to each post record. Increases dataset size; leave off for metadata-only.

## `includeRecommendations` (type: `boolean`):

Add the publications each newsletter recommends (the Substack growth/recommendation graph) as a 'recommends' array on the publication record.

## `includeAuthorProfiles` (type: `boolean`):

Emit one author record per unique creator (bio, follower count, subscriber band, bestseller tier, external links, all their publications). Also used for author start URLs.

## `enrichContactEmails` (type: `boolean`):

Crawl the publication's own website (custom domain home + about/contact, max 3 pages) for contact emails, phone and extra social links. Substack does not expose emails — this is the only way to get them. Adds a lead score.

## `onlyPaidPublications` (type: `boolean`):

Keep only publications that offer a paid subscription (they have monetization budget and sponsorship intent).

## `minFreeSubscribers` (type: `integer`):

Keep only publications with at least this many free subscribers (Substack publishes a free-subscriber count for most newsletters). 0 = no filter.

## `monitorMode` (type: `boolean`):

Remember publications and posts already returned and emit ONLY ones not seen in previous runs — ideal for scheduled runs that track new newsletters or new posts. Works alongside Apify Schedules.

## `monitorStoreName` (type: `string`):

Named key-value store that holds the 'already seen' ids for monitoring mode. Use a different name per tracked target to keep histories separate.

## `maxConcurrency` (type: `integer`):

Maximum parallel requests. Substack's public endpoints are tolerant; 5 is a safe, fast default.

## `proxyConfiguration` (type: `object`):

Proxy settings. Substack serves clean JSON from datacenter IPs, so the default Apify proxy is plenty.

## `urlsFromFile` (type: `string`):

Paste a list of URLs (one per line), OR one link to a .txt/.csv file, Google Sheet or Google Drive file containing them. Lets you import many Start URLs at once instead of typing each. Google Sheet/Drive share links are handled automatically.

## Actor input object example

```json
{
  "searchQueries": [
    "artificial intelligence"
  ],
  "maxPublications": 100,
  "includePosts": false,
  "maxPostsPerPublication": 20,
  "includePostContent": false,
  "includeRecommendations": false,
  "includeAuthorProfiles": false,
  "enrichContactEmails": false,
  "onlyPaidPublications": false,
  "minFreeSubscribers": 0,
  "monitorMode": false,
  "monitorStoreName": "substack-monitor",
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

All scraped records in the default dataset. Publication rows carry the full profile, subscriber counts, paid pricing, lead score and contact fields; post rows carry engagement and optional content; author rows carry creator profiles and external links.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "artificial intelligence"
    ],
    "urlsFromFile": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapesage/substack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["artificial intelligence"],
    "urlsFromFile": "",
}

# Run the Actor and wait for it to finish
run = client.actor("scrapesage/substack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "artificial intelligence"
  ],
  "urlsFromFile": ""
}' |
apify call scrapesage/substack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrapesage/substack-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/w9P9Jokpf6EojMuAZ/builds/SKok3oASWgEa6NEwy/openapi.json