# Discourse Forum Topics Scraper (`parseforge/discourse-forum-topics-scraper`) Actor

Scrapes topic listings from public Discourse forums. Returns each topic's title, author, reply count, views, and timestamps as a flat row, filtered by category and sort order.

- **URL**: https://apify.com/parseforge/discourse-forum-topics-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Social media, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Discourse Forum Topics Scraper

**Scrape topics from any public Discourse forum, up to a million per run.** Collect titles, authors, reply counts, and timestamps from the latest, top, new, or unread feeds. No API key required. Export to CSV, JSON, Excel, or XML.

Discourse forums power thousands of community sites, but pulling topic lists for analysis usually means manual browsing or writing custom API scripts. This Actor reads the public topic feeds directly from any Discourse instance. You get a clean, structured dataset of discussions, filtered by category and sorted by activity, ready for your reports.

| Who uses it | What they scrape Discourse for |
|---|---|
| Community managers | Monitor which discussions are trending across their Discourse forum each day. |
| Market researchers | Track conversation themes and product mentions within niche community forums. |
| Data analysts | Build a time-series dataset of topic activity to measure community health. |
| Competitive intelligence teams | Watch competitor community forums for feature requests and user pain points. |

### What it does

This Actor collects topic listings from a public Discourse forum's latest, top, new, or unread feeds and returns each topic as a flat row.

- 📋 **Flexible feeds:** scrape from the latest, top, new, or unread topic lists with one setting.
- 🏷️ **Category filter:** narrow the scrape to a single category by providing its URL slug.
- 📊 **Structured output:** every row includes the topic title, author, reply count, views, and timestamps.
- ⚙️ **Volume control:** set a hard limit from 1 to 1,000,000 topics per run to match your needs.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Discourse data

**📈 Monitor community engagement.**

A community manager scrapes the top weekly topics from their forum to report on active threads and member participation to stakeholders.

**🔍 Research niche discussions.**

A market analyst scrapes a specific product category on a public forum to identify common user questions and sentiment trends.

**📅 Archive topic history.**

A data engineer runs a daily scrape of the latest topics to build a searchable archive of community knowledge over time.

**🛡️ Track competitor forums.**

A product manager scrapes a competitor's feedback category to spot feature gaps and recurring complaints.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API setup** | Reads the same public feeds a browser sees, so you skip app registration and OAuth. |
| **Any Discourse site** | Works on thousands of forums, from company support sites to open-source communities. |
| **Fixed schema** | Every run returns the same fields, making it safe for automated pipelines and dashboards. |

### How it compares

No other Store actor targets Discourse the same way, so the honest comparison is with the alternatives teams actually weigh.

| | Discourse Forum Topics Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Discourse changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor with a forum base URL and an optional category slug, and set the sort order to target the most relevant discussions. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "forumUrl": "https://meta.discourse.org/",
  "maxItems": 10
}
```

A larger pull:

```json
{
  "forumUrl": "https://meta.discourse.org/",
  "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Discourse Forum Topics Scraper](https://apify.com/parseforge/discourse-forum-topics-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Discourse through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/discourse-forum-topics-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**The Actor returns zero results even though I can see topics on the forum.**

Check that the forum URL is correct and does not include a trailing slash or a path to a specific topic. Also verify that the category slug, if provided, matches exactly what appears in the forum's URL.

**I get an error that the page could not be loaded.**

The forum might be blocking automated access, or the URL might be incorrect. Confirm the forum is publicly accessible by opening it in a private browser window. If it uses Cloudflare or similar protection, the Actor may not be able to access it.

**The data has fewer topics than I requested in 'Maximum topics'.**

The Actor can only return topics that exist in the selected feed. If the forum has fewer topics than your max setting, you will get all available topics. Try switching the sort order to 'latest' or removing the category filter to capture more.

**The category filter is not working.**

Ensure you are using the category slug, not the full category name. The slug is the lowercase, hyphenated version found in the URL. For example, use 'feature-requests' instead of 'Feature Requests'.

**The run times out before collecting all topics.**

Lower the 'Maximum topics' value and run the Actor in smaller batches. You can also increase the run timeout in the Actor's advanced settings if you are on a paid Apify plan.

### FAQ

| Question | Answer |
|---|---|
| Do I need an API key or login to scrape a Discourse forum? | No. This Actor reads the public topic listing pages that are visible to anyone visiting the forum in a browser. If the forum is public, you can scrape it without authentication. |
| Can I scrape private or hidden Discourse categories? | No. The Actor can only access categories that are publicly visible without a login. Private groups and restricted categories are not available. |
| How do I find the category slug for a specific forum section? | Navigate to the category page on the forum and look at the URL. The slug is the last part of the path, for example 'support' in https://meta.discourse.org/c/support/. |
| What is the difference between the 'latest', 'top', 'new', and 'unread' sort options? | These correspond to the standard Discourse topic list filters. 'Latest' shows recently active topics, 'top' shows the highest-scoring topics, 'new' shows newly created topics, and 'unread' shows topics with unread posts for a logged-in user. |
| Does this Actor scrape the full body text of each post inside a topic? | No. This Actor scrapes the topic listing page, which gives you the topic title, author, reply count, views, and timestamps. It does not enter each topic to extract the full post content. |
| Can I scrape multiple categories in a single run? | The input accepts a single category slug per run. To scrape multiple categories, run the Actor multiple times with different category slugs, or leave the slug empty to scrape all public categories together. |
| What format is the exported data in? | You can export your results in CSV, JSON, Excel, or XML formats directly from the Apify dataset tab. |
| Is there a rate limit I should worry about? | The Actor runs at a reasonable speed to avoid overwhelming smaller forums. If you need to scrape a very large forum quickly, you can adjust the max items setting and run it in batches. |
| Will this work with any site that uses Discourse? | It works with standard Discourse instances that expose the default topic list endpoints. Heavily customized themes or very old Discourse versions might have a different HTML structure, but the vast majority of forums are supported. |
| How do I get the full URL of my Discourse forum? | Use the base URL you see in your browser's address bar when you visit the forum's homepage, for example https://community.example.com. Do not include a trailing slash or a path to a specific topic. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Civilized Discourse Construction Kit, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `forumUrl` (type: `string`):

Base URL of the Discourse forum.

## `maxItems` (type: `integer`):

How many forum topics to collect per run.

## `sort` (type: `string`):

Discourse listing endpoint to query.

## `categorySlug` (type: `string`):

Optional category slug.

## Actor input object example

```json
{
  "forumUrl": "https://meta.discourse.org/",
  "maxItems": 10,
  "sort": "latest"
}
```

# Actor output Schema

## `results` (type: `string`):

Scraped Discourse Forum Topics records

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "forumUrl": "https://meta.discourse.org/",
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/discourse-forum-topics-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "forumUrl": "https://meta.discourse.org/",
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/discourse-forum-topics-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "forumUrl": "https://meta.discourse.org/",
  "maxItems": 10
}' |
apify call parseforge/discourse-forum-topics-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/discourse-forum-topics-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/geOe5sunZhDQhj0wj/builds/aPGseuYmdQhlq5mZk/openapi.json