# Subreddit Scraper - Whole Subreddits, No 1k Post Cap ($1.5/1k) (`harshmaur/reddit-subreddit-scraper`) Actor

Scrape entire subreddits — thousands of posts per community, far beyond Reddit's ~1,000-post listing cap, by combining every sort and time window. Optional comments per post. Archive communities or build ML datasets. No API key. CSV/Excel/JSON. From $1.50 per 1,000 posts.

- **URL**: https://apify.com/harshmaur/reddit-subreddit-scraper.md
- **Developed by:** [Harsh Maur](https://apify.com/harshmaur) (community)
- **Categories:** AI, Lead generation, Social media
- **Stats:** 78 total users, 7 monthly users, 96.3% runs succeeded, 5 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.50 / 1,000 result saveds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Subreddit Scraper — Scrape an Entire Subreddit Past the 1,000-Post Limit (No API Key)

**Download all posts from a subreddit — thousands per community, far beyond Reddit's ~1,000-post listing cap. Optional comments, no API key, no login. From $1.50 per 1,000 results.**

[![Try on Apify](https://img.shields.io/badge/Apify-Try%20Now-00C7B7?style=for-the-badge\&logo=apify)](https://apify.com/harshmaur/reddit-subreddit-scraper) [![Input Schema](https://img.shields.io/badge/Input-Schema-blue?style=for-the-badge)](https://apify.com/harshmaur/reddit-subreddit-scraper/input-schema) [![API Docs](https://img.shields.io/badge/API-Docs-orange?style=for-the-badge)](https://apify.com/harshmaur/reddit-subreddit-scraper/api)

[What it does](#what-does-subreddit-scraper-do) • [How to use](#how-to-scrape-an-entire-subreddit) • [1,000-post limit](#how-subreddit-scraper-bypasses-reddits-1000-post-listing-cap) • [Input](#input-example) • [Output](#output-example) • [Pricing](#how-much-does-it-cost-to-scrape-a-subreddit) • [FAQ](#faq)

***

### What does Subreddit Scraper do?

**Subreddit Scraper** is a purpose-built subreddit scraper that takes a community name — `AskReddit`, `r/AskReddit`, or a full URL — and pulls as much of that subreddit as possible into one deduplicated dataset. Where an ordinary scrape stops at the ~1,000 posts Reddit exposes per listing, this actor walks **every sort order across every time window plus per-flair listings**, merging the results so you keep going long after a single listing runs dry.

- 🗂️ **Deep coverage** — combines new, top, and controversial sorts over hour/day/week/month/year/all-time windows, then adds flair-filtered listings on top
- 💬 **Optional comments** — flip one switch to also pull each post's comment thread
- 🔑 **Zero setup** — no Reddit account, no OAuth app, no API key; sidesteps Reddit's 600 requests/10 min API ceiling entirely
- 📊 **70+ fields per post** — full text and HTML, media and galleries, flair, flags, and derived engagement metrics
- 🔄 **Export and automate** — JSON, CSV, Excel, XML, HTML downloads; integrates with n8n, Zapier, Make, the REST API, and MCP
- 💰 **Pay per result** — from $1.50 per 1,000 results, no monthly fee

If you want keyword search, single posts, or user profiles, grab the all-in-one [Reddit Scraper](https://apify.com/harshmaur/reddit-scraper) instead — this actor does one thing deeply: whole communities.

***

### How to scrape an entire subreddit

No code needed — a deep subreddit pull takes five steps:

1. [Sign up for Apify](https://apify.com/) for free (or log in)
2. Open [Subreddit Scraper](https://apify.com/harshmaur/reddit-subreddit-scraper)
3. Type the communities you want into **Subreddits** — any format works: `AskReddit`, `r/AskReddit`, or `https://www.reddit.com/r/AskReddit/`
4. Set **Max posts** (default 1,000 — raise it up to 50,000 to go deep) and optionally enable **Scrape comments for each post**
5. Press **Start**, then download the dataset as JSON, CSV, Excel, XML, or HTML

> 💡 **Tip:** Start with a small `maxPostsCount` to verify the output shape, then rerun with a high cap for the full archive pull.

***

### How Subreddit Scraper bypasses Reddit's 1,000-post listing cap

Reddit never serves more than roughly **1,000 posts per listing**. Page through r/technology/new as long as you like — the feed simply ends. The same wall applies to top, hot, and controversial, which is why most subreddit scrapers quietly top out around a thousand items no matter what limit you ask for.

This actor's workaround is a **union of listings instead of one listing**:

1. **Every sort order** — it crawls `new`, `top`, and `controversial` separately
2. **Every time window** — `top` and `controversial` are fetched for hour, day, week, month, year, and all-time, and each window surfaces a different slice of history
3. **Every flair** — the subreddit's flair-filtered listings are crawled too, reaching posts that never appeared in any global sort
4. **Deduplication** — overlapping results are merged by post ID, so each post appears exactly once in your dataset

Each listing is individually capped at ~1,000 posts, but their *union* is not — on active communities this routinely yields several times more than any single listing could. Honest caveat: it still cannot guarantee literally every post ever submitted; very old posts that fall outside all current listings stay out of reach for any scraper.

***

### Input example

The full reference lives on the [Input Schema tab](https://apify.com/harshmaur/reddit-subreddit-scraper/input-schema). A typical deep-archive run looks like this:

```json
{
  "subreddits": ["r/buildapc", "MachineLearning", "https://www.reddit.com/r/cscareerquestions/"],
  "maxPostsCount": 20000,
  "crawlCommentsPerPost": true,
  "maxCommentsPerPost": 25,
  "includeNSFW": false,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

| Field | Type | What it controls |
| --- | --- | --- |
| `subreddits` | array | One or more communities in any format — bare name, `r/name`, or full URL |
| `maxPostsCount` | integer | Total posts to save across all subreddits (default 1,000, max 50,000) |
| `crawlCommentsPerPost` | boolean | Also collect comments for every post found (default off — adds run time) |
| `maxCommentsPerPost` | integer | Comment cap per post when comment crawling is on (default 10) |
| `includeNSFW` | boolean | Keep posts marked 18+ in the results (default off) |
| `proxy` | object | Proxy settings — Apify residential proxies recommended |

***

### Output example

Every post arrives as a structured item with 70+ fields, downloadable as **JSON, CSV, Excel, XML, or HTML**. A trimmed example:

```json
{
  "dataType": "post",
  "id": "t3_1xyz789",
  "parsedId": "1xyz789",
  "title": "Finally finished my first PC build — lessons learned",
  "body": "After three months of part-hunting, here is everything I wish I had known...",
  "bodyHtml": "<p>After three months of part-hunting...</p>",
  "postType": "text",
  "flair": "Build Complete",
  "authorName": "first_time_builder",
  "authorId": "t2_abc123",
  "communityName": "r/buildapc",
  "parsedCommunityName": "buildapc",
  "subredditName": "buildapc",
  "subredditId": "t5_2s3nh",
  "subredditSubscribers": 7200000,
  "score": 2841,
  "upVotes": 2841,
  "upvoteRatio": 0.96,
  "commentsCount": 312,
  "postUrl": "https://www.reddit.com/r/buildapc/comments/1xyz789/",
  "nsfw": false,
  "over18": false,
  "isSelf": true,
  "locked": false,
  "stickied": false,
  "archived": false,
  "removedByCategory": null,
  "isRobotIndexable": true,
  "totalAwardsReceived": 1,
  "domain": "self.buildapc",
  "mediaType": "text",
  "hasMedia": false,
  "isVideo": false,
  "isGallery": false,
  "ageHours": 52.4,
  "scorePerHour": 54.22,
  "commentsPerHour": 5.95,
  "engagementTotal": 3153,
  "isHighEngagement": true,
  "wordCount": 540,
  "createdAt": "2026-06-08T09:15:00.000Z",
  "crawledAt": "2026-06-10T13:40:00.000Z"
}
```

Posts with media additionally populate `galleryImages`, `mediaAssets`, `videoUrl`, `thumbnail`, and the raw Reddit objects (`media`, `secureMedia`, `mediaMetadata`, `galleryData`). With comment crawling enabled, comment items (`dataType: "comment"`) carry the full thread context — `parentId`, `depth`, `isSubmitter`, `controversiality`, plus the parent post's title and URL.

Watching a community for takedowns? When a post shows up as `[removed]`, `removedByCategory` records why — `moderator` for a mod removal, `automod_filtered` for one held for approval, `reddit`/`content_takedown` for a Reddit-side action, or `deleted` when the author pulled it themselves — and `isRobotIndexable` drops to `false`. Perfect for auditing what a subreddit's mods take down over time.

***

### What can you do with a full subreddit dataset?

- 🔬 **Community research** — study how a community's topics, norms, and vocabulary evolved across thousands of posts, not just last week's front page
- 🤖 **ML and NLP datasets** — turn a niche subreddit into a labeled training corpus; flair and engagement fields come free as features
- 📈 **Trend analysis** — chart `scorePerHour` and posting volume over time to spot when a topic took off inside a community
- 🗄️ **Archiving** — archive a subreddit before it goes private, gets banned, or its mods wipe history; export to CSV and keep it forever
- 🕵️ **Competitor community analysis** — pull a rival product's subreddit end to end and mine complaints, feature requests, and churn signals

***

### How much does it cost to scrape a subreddit?

Subreddit Scraper is **pay-per-result** — no subscription, no platform fee:

- **Actor start:** $0.02 per run
- **Result stored:** $0.0015 each

So a 1,000-post pull costs **$1.52**, a 10,000-post archive **$15.02**, and a 50,000-post maximum run **$75.02**. You pay only for items actually saved.

> ⏱️ **Honest note on run time:** depth has a cost in requests. Because the actor walks many listings per subreddit (sorts × time windows × flairs) before deduplicating, a deep run takes noticeably longer than a quick scrape of one listing. Budget accordingly for large `maxPostsCount` values — especially with comments enabled.

***

### FAQ

#### How many posts can I get from one subreddit?

It varies by community. Each Reddit listing maxes out near 1,000 posts, but because this actor merges new, top, and controversial across six time windows plus flair listings, active subreddits commonly yield **several thousand unique posts** — often 3-10× what a single-listing scrape returns. Quiet subreddits with few total posts will simply return everything they have.

#### Does it get around the 1,000 post limit?

Yes, in the only way that actually works: by unioning many capped listings rather than fighting the cap on one. You will get far more than 1,000 posts from any sizable subreddit. It does **not** guarantee every post ever made — content too old to appear in any current listing is unreachable for any scraper, including this one.

#### Can I download a whole subreddit to CSV?

Yes. After the run finishes, export the dataset straight from Apify as **CSV** (or Excel, JSON, XML, HTML). Each post becomes one row with 70+ columns — ready for spreadsheets, pandas, or BI tools.

#### Do I need a Reddit API key or account?

No. There is no OAuth flow, no login, and no Reddit account involved. The actor reads publicly available data directly, which also means Reddit's 600 requests/10 min API quota never applies to your runs.

#### Can it scrape comments too?

Yes — enable `crawlCommentsPerPost` and set `maxCommentsPerPost` to control depth. Be aware this multiplies the work: comments are fetched per post, so a 10,000-post run with comments takes substantially longer. For comment-first jobs, [Reddit Comments Scraper](https://apify.com/harshmaur/reddit-comments-scraper) is the better fit.

#### Can I scrape several subreddits in one run?

Yes. `subreddits` accepts a list, and `maxPostsCount` caps the total saved across all of them. Mix formats freely — `gaming`, `r/pcgaming`, and full URLs in the same array all work.

#### Does it include NSFW posts?

Only if you ask. 18+ posts are excluded by default; set `includeNSFW: true` to keep them in the dataset (they are flagged via the `nsfw` and `over18` fields either way).

#### Is scraping a subreddit legal?

Collecting publicly visible data is generally permissible, but you are responsible for using it lawfully: respect Reddit's terms, applicable privacy regulations, and the communities you study. Apify's guide on the [enforceability of terms of use](https://blog.apify.com/enforceability-of-terms-of-use/) is a good starting point.

***

### More Reddit scrapers

| Actor | When to use it instead |
| --- | --- |
| [Reddit Scraper](https://apify.com/harshmaur/reddit-scraper) | The all-in-one: keyword search, direct URLs, users, and subreddits in a single actor |
| [Reddit Comments Scraper](https://apify.com/harshmaur/reddit-comments-scraper) | You start from specific posts and want complete comment threads, nested replies included |
| [Reddit Search Scraper](https://apify.com/harshmaur/reddit-search-scraper) | You're hunting a topic or brand across all of Reddit rather than inside one community |
| [Reddit User Scraper](https://apify.com/harshmaur/reddit-user-scraper) | You need a redditor's profile, post history, and comment history |

***

[![Try Subreddit Scraper](https://img.shields.io/badge/🚀_Try_Subreddit_Scraper-00C7B7?style=for-the-badge)](https://apify.com/harshmaur/reddit-subreddit-scraper)

*Built by [Harsh Maur](https://github.com/harshmaur) • Questions? [Contact support](mailto:harshmaur@gmail.com)*

# Actor input Schema

## `subreddits` (type: `array`):

Enter one or more subreddits in any format: AskReddit, r/AskReddit, or https://www.reddit.com/r/AskReddit/.

## `postedAfter` (type: `string`):

Only keep posts created on or after this date (YYYY-MM-DD, e.g. 2025-01-01, or full ISO-8601, UTC). When set, the deep scrape walks the subreddit newest-first and stops at this edge. Reddit caps each listing at ~1,000 posts, so a window far in the past may be only partly reachable on very large subreddits.

## `postedBefore` (type: `string`):

Only keep posts created on or before this date (YYYY-MM-DD, e.g. 2025-12-31, or full ISO-8601); a plain date covers the whole day in UTC.

## `commentedAfter` (type: `string`):

Only keep comments created on or after this date (YYYY-MM-DD or ISO-8601, UTC). Applies when 'Scrape comments for each post' is enabled.

## `commentedBefore` (type: `string`):

Only keep comments created on or before this date (YYYY-MM-DD or ISO-8601, UTC).

## `onlyWithFlair` (type: `boolean`):

When enabled, keep only posts that have a flair assigned and drop flairless ones.

## `maxPostsCount` (type: `integer`):

Maximum number of posts to save across all subreddits. Raise this to pull entire communities. Max: 50000.

## `crawlCommentsPerPost` (type: `boolean`):

When enabled, comments are also collected for every post found. This significantly increases run time and data output.

## `maxCommentsPerPost` (type: `integer`):

Maximum comments to scrape per individual post (when 'Scrape comments for each post' is enabled).

## `includeNSFW` (type: `boolean`):

Include posts marked as Not Safe For Work in results.

## `sentimentAnalysis` (type: `boolean`):

Adds AI-judged fields to every post and comment: `sentimentLabel` (positive / negative / neutral / mixed / uncertain), `sentimentScore` (-1 to 1), `intent` (question, seeking\_recommendation, complaint, praise, purchase\_intent, comparison, announcement, self\_promotion, discussion), `emotion` (joy, excitement, anger, frustration, sadness, disappointment, fear, surprise, neutral), `entities` (brands, products and companies mentioned in the text) and — for results found via Search Keywords — `relevanceScore` (0 to 1 against your query). The model understands Reddit slang and sarcasm; bot and moderator boilerplate is labelled `uncertain` free of charge. Each analyzed result is billed as one `analyzed_item` event — see the Pricing tab. Paid add-on: requires a paid Apify plan and is available on this pay-per-event listing only — ignored on free plans.

## `contentAnalysis` (type: `boolean`):

Classifies each post into a `contentCategory` (News, Politics, Technology, Gaming, Entertainment, Sports, Finance, Shopping & Products, Relationships & Personal, Humor & Memes, Help & Support, Community & Meta, Adult, Other) using its title, body, subreddit and flair. Each analyzed result is billed as one `analyzed_item` event (a post with both toggles on is still charged only once). Paid add-on: requires a paid Apify plan; ignored on free plans.

## `customLabels` (type: `object`):

Define your own labels as `{"label_name": "plain-language instruction"}` and the AI evaluates each one on every analyzed post and comment, returning them under `customLabels` on each result. Example: `{"mentions_pricing": "true if the text complains about pricing or cost", "competitor": "name the competing product mentioned, or none"}`. Up to 5 labels; names must be short identifiers (letters, digits, underscore). Each label evaluated on each result is billed as one `custom_label` event — 3 labels on 1,000 results = 3,000 events. An evaluation that finds nothing (an empty/null answer, when your instruction allows it) is still an evaluation and is charged; nothing is charged for results the AI could not process. Paid add-on: requires a paid Apify plan; ignored on free plans.

## `proxy` (type: `object`):

Either use Apify proxy, or provide your own proxy servers.

## Actor input object example

```json
{
  "subreddits": [
    "r/AskReddit"
  ],
  "onlyWithFlair": false,
  "maxPostsCount": 1000,
  "crawlCommentsPerPost": false,
  "maxCommentsPerPost": 200,
  "includeNSFW": false,
  "sentimentAnalysis": false,
  "contentAnalysis": false,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

All scraped Reddit data including posts, comments, communities, and user profiles. Use 'dataType' field to filter by type.

## `runSummaryMap` (type: `string`):

Visual report of this run: how many posts, comments and communities were saved, what was skipped and why, and any input we adjusted for you.

## `runSummaryJson` (type: `string`):

The same run summary as machine-readable JSON, for pipelines and monitoring.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "subreddits": [
        "r/AskReddit"
    ],
    "maxPostsCount": 1000,
    "maxCommentsPerPost": 200,
    "proxy": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("harshmaur/reddit-subreddit-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "subreddits": ["r/AskReddit"],
    "maxPostsCount": 1000,
    "maxCommentsPerPost": 200,
    "proxy": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("harshmaur/reddit-subreddit-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "subreddits": [
    "r/AskReddit"
  ],
  "maxPostsCount": 1000,
  "maxCommentsPerPost": 200,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call harshmaur/reddit-subreddit-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,harshmaur/reddit-subreddit-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/kkR3ztZEkOQLinNj9/builds/1xQJOiP69Bvnt7KCH/openapi.json
