# Lemmy Scraper: Posts, Comments, Communities & Users (`perconey/lemmy-scraper`) Actor

Scrape any Lemmy instance (lemmy.world, lemmy.ml, beehaw.org and other Lemmyverse nodes) via the official /api/v3/\* REST API. Posts with upvote/downvote counts, comment trees, communities with subscriber counts, user profiles, full-text search. No auth, no proxies. Pay per result.

- **URL**: https://apify.com/perconey/lemmy-scraper.md
- **Developed by:** [Perconey](https://apify.com/perconey) (community)
- **Categories:** Social media, Developer tools, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 result items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### What does Lemmy Scraper do?

**Lemmy Scraper** pulls structured data from any [Lemmy](https://join-lemmy.org/) instance via the **official `/api/v3/*` REST API**. Posts with **upvote AND downvote counts** (unlike Reddit or HN, Lemmy exposes both), full comment trees, communities with subscriber and activity counts, user profiles, full-text search. The actor calls the documented public API directly - no browser, no proxies, no cookies, no anti-bot fight. One actor works with **every Lemmy-protocol server** in the Lemmyverse: lemmy.world, lemmy.ml, beehaw.org, sh.itjust.works, lemm.ee, feddit.org, programming.dev, and hundreds more.

Try it instantly: pick **getPosts**, leave instance as `https://lemmy.world`, click Start. You get the current Hot feed (30 posts) with creator, community, score, upvote/downvote, comment count in under 5 seconds for $0.03.

### Why use Lemmy Scraper?

- **Federation researchers**: Compare community dynamics across instances. The same `technology` community name exists on multiple instances with different culture, content, and activity.
- **Trend analysts**: Both upvotes AND downvotes are exposed. Compute controversy ratios that platforms like HN and Reddit hide.
- **Community managers**: Track mentions of your project across federated Lemmy. Schedule daily `searchPosts` runs with your product name.
- **OSS maintainers**: Lemmy's user base is heavily developer-skewed. `getCommunityPosts` for `programming@programming.dev`, `linuxmemes@lemmy.world`, `selfhosted@lemmy.world` is a free signal on what dev tools are getting attention.
- **Academic researchers**: Lemmy is a federated decentralized social network on ActivityPub - rich material for network science studies. Per-instance counts via `getSiteInfo` give you the federation graph.
- **Reddit refugees / content migrators**: Identify Lemmy communities that mirror Reddit subreddits you care about. Move your subscription pattern to the Lemmyverse.

### How to use Lemmy Scraper

1. Open the **Input** tab.
2. Pick an **action** from the dropdown. `getPosts` is the simplest starting point.
3. Set **instance** (default `https://lemmy.world`). To scrape a different server, paste its URL.
4. For community/user/post/search actions, fill **queries** (one entry per line). For getPosts / listCommunities / getSiteInfo, leave queries empty.
5. Tune **sort** (Hot / New / Active / TopWeek / TopAll / etc), **maxItems**, and `searchType` if using searchPosts.
6. Toggle `includeComments` for getPostDetail or `includeSubmissions` for getUserProfile to get the deep dive.
7. Click **Start**.

#### Query format by action

Action | Query format
\--- | ---
getSiteInfo | leave empty (uses the instance field)
getPosts | leave empty
listCommunities | leave empty
getCommunityDetail | `technology` or `technology@lemmy.world` or `https://lemmy.world/c/technology`
getCommunityPosts | same as getCommunityDetail
getUserProfile | `nutomic` or `nutomic@lemmy.ml` (auto-routes to user's home instance)
getPostDetail | post id (e.g. `1`) or full URL (e.g. `https://lemmy.world/post/1234`)
searchPosts | free-text search query

### Input

Field | Required | Description
\--- | --- | ---
`action` | yes | Which API call to make. Eight options.
`instance` | yes | Lemmy instance URL. Default https://lemmy.world.
`queries` | sometimes | Required for community/user/post/search actions. Empty for site-info / posts feed / list-communities.
`maxItems` | no | Max items per query. Default 30.
`sort` | no | Hot / Active / New / TopDay / TopWeek / TopMonth / TopYear / TopAll / MostComments / NewComments. Default Hot.
`searchType` | no | searchPosts only: Posts / Comments / Communities / Users / All. Default Posts.
`includeComments` | no | getPostDetail only: also walk the comment tree.
`includeSubmissions` | no | getUserProfile only: also fetch recent posts and comments by the user.
`token` | no | Lemmy JWT for actions an instance restricts to logged-in users (rare).

### Output

Every item carries `_type` (`post` / `comment` / `community` / `user` / `site` / `error`) plus `_action` and `_instance` for filtering.

```json
{
    "_type": "post",
    "_action": "getPosts",
    "_instance": "https://lemmy.world/",
    "id": 9876543,
    "name": "Rust 1.85 released - 2024 edition stable",
    "url": "https://blog.rust-lang.org/...",
    "body": "...",
    "creator_id": 12345,
    "community_id": 678,
    "published": "2026-05-12T10:30:00.000Z",
    "nsfw": false,
    "score": 542,
    "upvotes": 569,
    "downvotes": 27,
    "comments": 89,
    "post_url": "https://lemmy.world/post/9876543",
    "creator": { "id": 12345, "name": "rustacean", "actor_id": "https://lemmy.world/u/rustacean" },
    "community": { "id": 678, "name": "rust", "title": "Rust", "actor_id": "https://lemmy.world/c/rust" }
}
```

You can download the dataset in JSON, CSV, XML, Excel, RSS or HTML format from the Output tab.

### Data fields

Type | Key fields
\--- | ---
`post` | id, name (title), url, body, score, upvotes, downvotes, comments, published, nsfw, locked, post\_url, creator, community
`comment` | id, post\_id, content, path, score, upvotes, downvotes, child\_count, published, creator, community
`community` | id, name, title, description, subscribers, posts, comments, users\_active\_day/week/month/half\_year, icon, banner, actor\_id, nsfw
`user` | id, name, display\_name, bio, avatar, banner, post\_count, post\_score, comment\_count, comment\_score, banned, published
`site` | name, description, users, posts, comments, communities, users\_active\_day/month, admins (array), version, federated\_instances count

### Pricing

**Pay-per-result: $0.001 per item.** No flat monthly fee.

Cost examples:

- Daily Hot feed (30 posts): **$0.03**
- 1,000 posts from `technology@lemmy.world` for content research: **$1.00**
- 100 user profiles with submissions (~4000 items): **$4.00**
- Full comment tree of a 500-comment thread: **$0.50**

### Tips

- **Cross-instance routing is automatic for users and communities.** If you write `nutomic@lemmy.ml`, the actor queries lemmy.ml's API (not the default instance), because that's where nutomic's account record lives. Same for communities: `technology@lemmy.world` queries lemmy.world. This catches a common gotcha where Lemmy user records 404 when queried on the wrong instance.
- **Upvotes and downvotes are both exposed.** Compute `controversy = min(up, down) / max(up, down)` for finding heated threads.
- **Comment trees can be huge.** Use maxItems to cap the BFS walk. The actor stops as soon as the budget is reached.
- **searchPosts supports searchType: All** to search across posts + comments + communities + users in one call. The result mix is shaped as posts in the dataset (other types currently flattened the same way; filter by `_type`).
- **Federation aware**: post/community IDs are NOT portable across instances. ID 1234 on lemmy.world is a different post than ID 1234 on lemmy.ml. Always pass full URLs to `getPostDetail` so the actor extracts the correct (instance, id) pair.

### FAQ, disclaimers, support

**Is this legal?** The actor calls each Lemmy instance's official public REST API with documented endpoints. Public read access is the design intent of the AGPL-licensed Lemmy software. We send a clear User-Agent identifying the actor and honor rate-limit / Retry-After headers.

**Does it work with Mbin / kbin?** Mbin (kbin's active fork) is an alternative Threadiverse server that mostly speaks the same Lemmy API. Most actions should work; trending or community-specific endpoints may differ. Open an Issue if you hit one.

**Why is getUserProfile failing for a user?** Lemmy user records live on the user's HOME instance. If you queried `nutomic` against `lemmy.world`, the actor returns a friendly hint to retry as `nutomic@lemmy.ml`. The actor auto-routes when you include the home-instance suffix.

**Will I get rate-limited?** Lemmy's per-IP rate limits are generous for read-only traffic and the actor backs off on 429 / Retry-After. For very heavy scraping consider donating to the instance you're hitting most.

**Bug or feature request?** Open an Issue on the actor's Issues tab. I usually respond within a day.

**Need a scraper for Mastodon, Bluesky, Stack Overflow, Hacker News?** See my other actors at https://apify.com/perconey, or open an Issue for a federated platform you need.

# Actor input Schema

## `action` (type: `string`):

Pick the type of data. getSiteInfo / getPosts / listCommunities need no queries. The others require at least one entry in queries.

## `instance` (type: `string`):

Which Lemmy-protocol server to query. Default https://lemmy.world. Other major instances: https://lemmy.ml, https://beehaw.org, https://sh.itjust.works, https://lemm.ee, https://feddit.org. For getCommunityDetail / getCommunityPosts / getUserProfile, paste the username or community with its home instance (e.g. nutomic@lemmy.ml) and the actor routes the call automatically.

## `queries` (type: `array`):

Depends on action. getSiteInfo / getPosts / listCommunities: leave empty. getCommunityDetail / getCommunityPosts: community name (e.g. 'technology' or 'technology@lemmy.world' or full URL). getUserProfile: username (e.g. 'nutomic' or 'nutomic@lemmy.ml'). getPostDetail: numeric post id or full URL (https://lemmy.world/post/...). searchPosts: free text.

## `maxItems` (type: `integer`):

Stop after this many items per query. For getPostDetail+includeComments and getUserProfile+includeSubmissions this caps the secondary fetch.

## `sort` (type: `string`):

Sort key for list/feed/search actions. Valid: Hot, Active, New, Old, TopDay, TopWeek, TopMonth, TopYear, TopAll, MostComments, NewComments. Default Hot for posts, TopAll for communities.

## `searchType` (type: `string`):

What to search for. Posts is the default. All searches across posts + comments + communities + users in one call.

## `includeComments` (type: `boolean`):

Walk the post's comment tree. Each comment counts as one more result-item, capped by maxItems.

## `includeSubmissions` (type: `boolean`):

After the user profile, also fetch their recent posts and comments. Each counts toward maxItems.

## `token` (type: `string`):

Lemmy session JWT. Only needed for actions that the instance restricts to logged-in users (some lock followers/saved). Almost all public reads work anonymously.

## Actor input object example

```json
{
  "action": "getPosts",
  "instance": "https://lemmy.world/",
  "queries": [],
  "maxItems": 30,
  "sort": "Hot",
  "searchType": "Posts",
  "includeComments": false,
  "includeSubmissions": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "instance": "https://lemmy.world/",
    "queries": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("perconey/lemmy-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "instance": "https://lemmy.world/",
    "queries": [],
}

# Run the Actor and wait for it to finish
run = client.actor("perconey/lemmy-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "instance": "https://lemmy.world/",
  "queries": []
}' |
apify call perconey/lemmy-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,perconey/lemmy-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/23OI4aKLQvmTn0ZNW/builds/AuIrMhlWyUxabaL23/openapi.json