# Subreddit Scraper - Deep Crawl Posts, Media & Nested Comments (`mikolabs/subreddit-scraper-deep-crawl-posts-media-nested-comments`) Actor

Extract complete subreddit feeds (Hot, New, Top, Rising, Controversial), historical post archives, media galleries, flairs, and nested discussion comment trees from any Reddit community. Download clean, structured JSON/CSV data with AI sentiment and content taxonomy enrichment.

- **URL**: https://apify.com/mikolabs/subreddit-scraper-deep-crawl-posts-media-nested-comments.md
- **Developed by:** [Mikolabs](https://apify.com/mikolabs) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.60 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Subreddit Scraper — Deep Crawl Posts, Media & Nested Comments

Extract complete subreddit feeds (Hot, New, Top, Rising, Controversial), historical post archives, media galleries, flairs, and nested discussion comment trees from any [Reddit](https://www.reddit.com/) community. Download clean, structured JSON/CSV data with AI sentiment and content taxonomy enrichment.

### Overview

Subreddit Scraper is an enterprise-grade data extraction tool built specifically for monitoring and deep-crawling entire Reddit communities. Provide one or more subreddit names (e.g. `technology`, `startups`, `AskReddit`), select your sorting order, and extract thousands of posts and discussion threads with automatic pagination.

When comment extraction is enabled, the actor also collects the full discussion thread for every post, including replies, author karma, upvote counts, and AI sentiment scoring.

***

### Why Use Subreddit Scraper

- **Community & Niche Research**: Analyze audience interests, common pain points, and viral topics across specific subreddits.
- **Brand & Product Monitoring**: Track customer feedback, brand mentions, and product discussions inside industry communities.
- **Trend Forecasting**: Monitor `Rising` and `Hot` feeds to spot emerging trends before they hit mainstream social media.
- **Content & SEO Ideation**: Mine high-upvote posts (`Top` of the year/all-time) to generate engaging article and video ideas.
- **Longitudinal Datasets**: Crawl historical subreddit archives for academic discourse research and market intelligence.

***

### Pricing & Plans (No Hidden Fees)

Transparent and predictable pricing with **no extra proxy costs, no setup fees, and no hidden maintenance charges**.

#### Tiered Pricing Structure

| Tier / Discount Level | Price per 1,000 Items | Effective Savings | Minimum Scrape |
| :--- | :--- | :--- | :--- |
| **No Discount (Standard / Pay-As-You-Go)** | **$4.00 / 1,000 items** | Standard Rate | 1 item |
| **🥉 Bronze Discount** | **$2.00 / 1,000 items** | **50% OFF** | 20 items |
| **🥈 Silver Discount** | **$1.80 / 1,000 items** | **55% OFF** | 20 items |
| **🥇 Gold Discount** | **$1.60 / 1,000 items** | **60% OFF** | 20 items |

#### Plan Comparison

| Feature | Free Tier | Subscriber / Paid Tier |
| :--- | :--- | :--- |
| **Free Daily Allowance** | **20 items / run (4 runs / day free)** | Unlimited |
| **Pricing** | $4.00 / 1,000 results (or free allowance) | **Down to $1.60 / 1,000 results** |
| **Additional Fees** | **$0.00 (No extra fees)** | **$0.00 (No extra fees)** |
| **Proxy / Bandwidth Costs** | Included ($0.00) | Included ($0.00) |
| **Deep Crawl Pagination** | ✅ | ✅ |
| **Comments Extraction per Post** | ✅ | ✅ |
| **AI Sentiment & Taxonomy** | ✅ | ✅ |
| **Granular Filters** | ✅ | ✅ |
| **Run Summary Dashboard** | ✅ | ✅ |

*Free users can extract up to 20 items per run (4 runs/day) completely free. Upgrade for volume discounts down to $1.60 / 1,000 items with zero hidden fees.*

***

### How to Use — Step by Step

1. **Enter Subreddit Names**: Input one or more subreddits in `Subreddits to Scrape` (e.g. `["startups", "SaaS"]`).
2. **Choose Sort & Timeframe**: Select feed sorting (`hot`, `new`, `top`, `rising`, `controversial`) and time window (`all`, `year`, `month`, `week`, `day`).
3. **Configure Comments Extraction (Optional)**: Toggle `Scrape Comments for Each Post` and specify how many comments per thread to collect.
4. **Set Filters & AI Analytics**: Apply date ranges, score minimums, flair filters, and toggle `AI Sentiment Analysis` or `AI Content Taxonomy`.
5. **Click Start**: Download results in JSON, CSV, Excel, XML, or HTML table format.

***

### Input Parameters

| Parameter | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `subreddits` | string\[] | `["technology", "startups"]` | List of subreddit names to scrape (without `r/`). |
| `sort` | string | `hot` | Feed sort: `hot`, `new`, `top`, `rising`, `controversial`. |
| `timeframe` | string | `all` | Time window for top/controversial: `all`, `year`, `month`, `week`, `day`, `hour`. |
| `fullSubredditMode` | boolean | `false` | Deep crawl mode — paginates historical pages for maximum coverage. |
| `maxPostsPerSubreddit` | integer | `100` | Maximum posts to collect per subreddit. |
| `maxTotalItems` | integer | `500` | Safety ceiling for total items (posts + comments). |
| `includeComments` | boolean | `false` | Extract full comment threads for each collected post. |
| `maxCommentsPerPost` | integer | `25` | Maximum comments per post thread. |
| `commentsSort` | string | `confidence` | Comment ranking order: `confidence` (Best), `top`, `new`, `controversial`, `old`, `qa`. |
| `flattenComments` | boolean | `false` | When true, pushes each comment as a separate dataset row. |
| `sentiment_analysis` | boolean | `false` | Adds sentiment score, confidence, and label to each post and comment. |
| `content_analysis` | boolean | `false` | Classifies posts against an enterprise topic taxonomy. |
| `postType` | string | `all` | Filter by media format: `all`, `text`, `image`, `video`, `gallery`, `link`. |
| `minScore` | integer | – | Only keep posts with at least this upvote score. |
| `minComments` | integer | – | Only keep posts with at least this many comments. |
| `postsCreatedAfter` | string | – | Only keep posts created on or after date (`YYYY-MM-DD`). |
| `postsCreatedBefore` | string | – | Only keep posts created on or before date (`YYYY-MM-DD`). |
| `flairContains` | string | – | Only keep posts matching this flair keyword. |
| `titleContains` | string | – | Only keep posts whose title contains this keyword. |
| `excludeStickied` | boolean | `false` | Exclude pinned moderator announcements. |
| `excludeKeywords` | string\[] | – | Exclude posts containing any of these keywords. |
| `includeNsfw` | boolean | `true` | Include NSFW/18+ content in output. |

***

### Example Output: Post Record with Nested Comments

```json
{
    "kind": "post",
    "id": "1hvoazn",
    "title": "My best cheesecake so far",
    "body": "Found my new favorite recipe (no water bath).",
    "author": "ClearlyBulky",
    "score": 3489,
    "upvote_ratio": 1.0,
    "num_comments": 43,
    "subreddit": "Baking",
    "created_utc": "2025-01-07T10:09:56.000Z",
    "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/",
    "permalink": "/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/",
    "flair": "Recipe",
    "media_type": "gallery",
    "sentiment_score": 2,
    "sentiment_label": "positive",
    "content_category_label": "Desserts & Baking",
    "content_category_path": ["Food & Drink", "Desserts & Baking"],
    "comments_count_scraped": 1,
    "comments": [
        {
            "kind": "comment",
            "id": "m5un6bj",
            "author": "BakingFanatic",
            "score": 76,
            "depth": 0,
            "body": "This looks absolutely incredible! Can you share the recipe?",
            "sentiment_label": "positive"
        }
    ]
}
```

***

### API Access

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("YOUR_ACTOR_ID").call(run_input={
    "subreddits": ["startups", "SaaS"],
    "sort": "top",
    "timeframe": "month",
    "includeComments": True,
    "maxCommentsPerPost": 20,
    "sentiment_analysis": True,
    "content_analysis": True,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"[{item.get('content_category_label')}] {item['title']} ({item['score']} upvotes)")
```

***

### Support

For help or feature requests, use the **Issues** tab on the actor page in Apify Console.

# Actor input Schema

## `subreddits` (type: `array`):

List of subreddit names (without r/) to scrape.

## `sort` (type: `string`):

How to sort posts in the subreddit feed.

## `timeframe` (type: `string`):

Time window for Top or Controversial posts.

## `fullSubredditMode` (type: `boolean`):

Deep crawl mode — paginates as far back as Reddit allows for maximum historical post extraction.

## `maxPostsPerSubreddit` (type: `integer`):

Maximum number of posts to collect per subreddit.

## `maxTotalItems` (type: `integer`):

Safety ceiling for total items (posts + comments) across the entire run.

## `includeComments` (type: `boolean`):

When enabled, extracts the comment thread and replies for every scraped post.

## `maxCommentsPerPost` (type: `integer`):

Maximum number of comments to extract per post thread.

## `commentsSort` (type: `string`):

Ranking method for comments in threads.

## `flattenComments` (type: `boolean`):

When true, pushes each comment as its own dataset row instead of nesting under parent post.

## `sentiment_analysis` (type: `boolean`):

Enriches each post and comment with sentiment score, confidence, and label (positive, negative, neutral, mixed, uncertain).

## `content_analysis` (type: `boolean`):

Classifies posts against an enterprise topic taxonomy (category label and hierarchical path).

## `postType` (type: `string`):

Filter posts by media type.

## `minScore` (type: `integer`):

Only keep posts with at least this score.

## `minComments` (type: `integer`):

Only keep posts with at least this many comments.

## `postsCreatedAfter` (type: `string`):

Only keep posts created on or after this date (YYYY-MM-DD).

## `postsCreatedBefore` (type: `string`):

Only keep posts created on or before this date (YYYY-MM-DD).

## `flairContains` (type: `string`):

Only keep posts with flair text containing this keyword.

## `titleContains` (type: `string`):

Only keep posts whose title contains this keyword.

## `excludeStickied` (type: `boolean`):

Exclude moderator announcements and pinned posts.

## `excludeKeywords` (type: `array`):

Exclude posts containing any of these keywords.

## `includeNsfw` (type: `boolean`):

Include posts marked as NSFW/18+ in output dataset.

## Actor input object example

```json
{
  "subreddits": [
    "technology",
    "startups"
  ],
  "sort": "hot",
  "timeframe": "all",
  "fullSubredditMode": false,
  "maxPostsPerSubreddit": 100,
  "maxTotalItems": 500,
  "includeComments": false,
  "maxCommentsPerPost": 25,
  "commentsSort": "confidence",
  "flattenComments": false,
  "sentiment_analysis": false,
  "content_analysis": false,
  "postType": "all",
  "excludeStickied": false,
  "includeNsfw": true
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "subreddits": [
        "technology",
        "startups"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("mikolabs/subreddit-scraper-deep-crawl-posts-media-nested-comments").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "subreddits": [
        "technology",
        "startups",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("mikolabs/subreddit-scraper-deep-crawl-posts-media-nested-comments").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "subreddits": [
    "technology",
    "startups"
  ]
}' |
apify call mikolabs/subreddit-scraper-deep-crawl-posts-media-nested-comments --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,mikolabs/subreddit-scraper-deep-crawl-posts-media-nested-comments"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/abewUXWb5xj9igwIK/builds/TbAgvDyGFwPJOMaHf/openapi.json