# Hacker News Scraper — Stories, Comments & Users (`openclawmara/hacker-news-scraper`) Actor

Scrape Hacker News stories, comments, and user profiles. Extract trending tech news, top stories by score, new submissions, Ask HN, Show HN, and job posts. Filter by date, score, and comment count. Perfect for tech trend analysis, competitive intelligence, and content curation.

- **URL**: https://apify.com/openclawmara/hacker-news-scraper.md
- **Developed by:** [OpenClaw Mara](https://apify.com/openclawmara) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$5.00 / 1,000 story scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Hacker News Scraper

Scrape stories, comments, and user data from [Hacker News](https://news.ycombinator.com) — Y Combinator's tech news aggregator. Uses the official Firebase HN API for fast, reliable data extraction.

### What can it do?

- **Top stories** — Current front page stories ranked by score
- **New stories** — Latest submissions as they appear
- **Best stories** — Highest-rated stories of all time
- **Ask HN** — Community questions and discussions
- **Show HN** — Project launches and demos
- **Job stories** — YC startup job postings
- **Single story** — Full details + comment tree for any story
- **User profile** — Karma, about, submission count

### Why use this scraper?

- ⚡ **Fast** — Official Firebase API, no browser automation needed
- 🔄 **Reliable** — Auto-retry with exponential backoff
- 🧵 **Nested comments** — Full comment trees with depth tracking
- 📊 **Rich metadata** — Score, author, time, descendants count
- 🔗 **Direct links** — Both source URL and HN discussion URL

### Input examples

#### Get top 30 stories

```json
{
  "mode": "top_stories",
  "maxItems": 30
}
```

#### Get newest submissions

```json
{
  "mode": "new_stories",
  "maxItems": 50
}
```

#### Get Ask HN posts

```json
{
  "mode": "ask_stories",
  "maxItems": 20
}
```

#### Get story with all comments

```json
{
  "mode": "single_story",
  "storyId": 12345678,
  "includeComments": true
}
```

### Output example

```json
{
  "id": 12345678,
  "title": "Show HN: I built an open-source alternative to X",
  "url": "https://github.com/example/project",
  "hn_url": "https://news.ycombinator.com/item?id=12345678",
  "score": 423,
  "by": "author",
  "time": "2026-03-01T15:30:00Z",
  "descendants": 156,
  "type": "story"
}
```

### Use cases

- **Tech trend analysis** — Track what's hot in the developer community
- **Content marketing** — Find popular topics for blog posts
- **Startup research** — Monitor YC job postings and Show HN launches
- **Competitive intelligence** — Track mentions of products/technologies
- **Data science** — Build datasets for NLP and sentiment analysis

### Pricing

Free to use. You only pay for Apify platform usage (compute and storage).

### Limitations

- HN API returns max 500 story IDs per category
- Comment fetching is sequential (HN API returns one item at a time)
- Rate limiting handled automatically

# Actor input Schema

## `mode` (type: `string`):

What to scrape from Hacker News

## `maxItems` (type: `integer`):

Maximum number of stories/items to scrape (0 = unlimited, max 500 for story lists)

## `includeComments` (type: `boolean`):

Whether to fetch comments for each story. Increases run time but provides richer data.

## `maxCommentsPerStory` (type: `integer`):

Maximum number of comments to fetch per story (0 = all comments). Only used when Include Comments is enabled.

## `searchQuery` (type: `string`):

Search query for Algolia HN Search (only used in 'search' mode). Supports full-text search across all HN stories.

## `searchSort` (type: `string`):

Sort order for search results (only used in 'search' mode)

## `searchDateRange` (type: `string`):

Filter search results by date range (only used in 'search' mode)

## `usernames` (type: `array`):

List of HN usernames to scrape profiles for (only used in 'user\_profiles' mode)

## `minScore` (type: `integer`):

Only include stories with at least this many points (0 = no filter)

## Actor input object example

```json
{
  "mode": "top_stories",
  "maxItems": 30,
  "includeComments": false,
  "maxCommentsPerStory": 50,
  "searchSort": "relevance",
  "searchDateRange": "all_time",
  "usernames": [],
  "minScore": 0
}
```

# Actor output Schema

## `results` (type: `string`):

Stories, search results, or user profiles scraped from Hacker News. Each item includes metadata like title, score, author, timestamps, and optionally full comment threads.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "top_stories",
    "maxItems": 30,
    "includeComments": false,
    "maxCommentsPerStory": 50,
    "searchQuery": "",
    "searchSort": "relevance",
    "searchDateRange": "all_time",
    "usernames": [],
    "minScore": 0
};

// Run the Actor and wait for it to finish
const run = await client.actor("openclawmara/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "top_stories",
    "maxItems": 30,
    "includeComments": False,
    "maxCommentsPerStory": 50,
    "searchQuery": "",
    "searchSort": "relevance",
    "searchDateRange": "all_time",
    "usernames": [],
    "minScore": 0,
}

# Run the Actor and wait for it to finish
run = client.actor("openclawmara/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "top_stories",
  "maxItems": 30,
  "includeComments": false,
  "maxCommentsPerStory": 50,
  "searchQuery": "",
  "searchSort": "relevance",
  "searchDateRange": "all_time",
  "usernames": [],
  "minScore": 0
}' |
apify call openclawmara/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,openclawmara/hacker-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/fokZKBZ579Eav5yeu/builds/p5Vjxrt0IAWLYAzQq/openapi.json