# Hacker News Scraper (`parseforge/hacker-news-scraper`) Actor

Scrapes Hacker News stories from six feeds (top, new, best, ask, show, jobs) and returns each story as a flat row with title, URL, score, author, and comment count.

- **URL**: https://apify.com/parseforge/hacker-news-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, News, Other
- **Stats:** 3 total users, 1 monthly users, 100.0% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $11.99 / 1,000 result items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Hacker News Scraper

**Scrape Hacker News stories from top, new, best, ask, show, or job lists, up to a million per run.** Each story comes with its title, URL, score, author, and comment count. No API key required. Export to CSV, JSON, Excel, or XML.

Hacker News is the tech industry's water cooler, but manually tracking trending stories across its multiple feeds is a full-time job. This scraper reads the public story lists directly, letting you pull the top, new, best, ask, show, or job posts on demand. Filter by story type and set a maximum count, and every match lands in a clean, flat dataset ready for analysis.

| Who uses it | What they scrape Hacker News for |
|---|---|
| Product managers | Monitor Show HN launches to spot new competitors and tools in their space. |
| Market researchers | Track which technologies and topics are gaining traction among early adopters. |
| Content marketers | Identify high-performing story angles and headlines that resonate with a technical audience. |
| Job seekers | Collect the latest Y Combinator startup job postings from the jobs feed. |

### What it does

This Actor collects Hacker News stories from a selected feed and returns each one as a flat row with its title, URL, score, author, and comment count.

- 📋 **Six story feeds:** top, new, best, ask, show, and job stories, each selectable with one click.
- 🔢 **Configurable volume:** set a maximum from 1 to 1,000,000 stories per run to control dataset size.
- 📄 **Flat output schema:** every story returns as a single row with title, URL, score, author, and comment count.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Hacker News data

**📈 Track trending technologies.**

A market researcher pulls the top 500 stories daily to chart which programming languages and frameworks are gaining mindshare among developers.

**🔍 Monitor competitor launches.**

A product manager scrapes Show HN every week, filtering for their industry keywords to catch new product announcements early.

**💡 Generate content ideas.**

A content marketer collects the month's best stories to analyze which headlines and topics drove the highest engagement for their next newsletter.

**💼 Aggregate startup jobs.**

A job seeker scrapes the jobs feed to build a personal pipeline of the latest Y Combinator startup openings in one spreadsheet.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API key** | Reads the public Hacker News feeds directly with no registration or authentication. |
| **Fixed schema** | Every story, regardless of feed, returns the same predictable fields for easy processing. |
| **High throughput** | Collect up to a million stories in a single run for large-scale trend analysis. |

### How it compares

No other Store actor targets Hacker News the same way, so the honest comparison is with the alternatives teams actually weigh.

| | Hacker News Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Hacker News changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor by selecting one of six story feeds and a maximum number of stories to collect. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "maxItems": 10
}
```

A larger pull:

```json
{
  "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.01599 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $1.60 |
| 1,000 results | $15.99 |
| 10,000 results | $159.90 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Hacker News Scraper](https://apify.com/parseforge/hacker-news-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Hacker News through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/hacker-news-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting fewer stories than my maxItems setting?**

The Hacker News feeds contain a finite number of stories. If you set maxItems to 1,000 but the 'ask' feed only has 200 stories, the run will finish after collecting all 200.

**The actor is running but returning no results.**

Check that you have selected a valid story type. If the Hacker News site is temporarily unreachable, wait a few minutes and retry the run.

**Some story fields are empty in my output.**

Certain fields like 'author' may be blank for some story types, such as job postings. This is expected behavior based on the data Hacker News provides.

**Can I get the full article text behind each story?**

This scraper collects the story metadata from Hacker News lists, not the linked article content. To get full article text, you would need a separate web scraping step for each external URL.

**The run is taking a long time with a high maxItems value.**

Collecting hundreds of thousands of stories requires many sequential requests to respect the site. Increase the run timeout in your task settings if needed, or reduce maxItems.

### FAQ

| Question | Answer |
|---|---|
| Do I need a Hacker News account or API key to use this? | No. This scraper reads the public Hacker News story feeds directly and requires no login, account, or API key. |
| What data fields does each story row contain? | Each row includes the story title, URL, score, author username, and comment count, returned in a consistent flat schema. |
| Can I scrape comments as well as stories? | This Actor is focused on story lists. It returns the comment count for each story but does not scrape the full comment threads. |
| How many stories can I collect in one run? | You can set the maximum anywhere from 1 to 1,000,000 stories. The Actor will stop once it reaches your limit. |
| Which story feeds are available? | You can choose from top, new, best, ask, show, and job stories using the story type dropdown in the input. |
| How often should I run this scraper? | The front page changes frequently. Running it every 15 to 30 minutes with a scheduled Apify task will capture the latest stories as they rise. |
| Can I filter stories by keyword or domain? | This Actor collects all stories from the selected feed up to your max count. For keyword or domain filtering, you can process the output dataset in a downstream step. |
| What output formats are supported? | You can export your results to CSV, JSON, Excel, or XML from the Apify dataset tab. |
| Is this an official Hacker News API? | No, this is an unofficial scraper that reads the public web feeds. It is not affiliated with Hacker News or Y Combinator. |
| Can I scrape historical Hacker News data? | This Actor reads the current story lists. For historical data, you would need to run it on a schedule and accumulate results over time. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Y Combinator. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `storyType` (type: `string`):

Which list of stories to scrape.

## `maxItems` (type: `integer`):

Maximum number of stories to collect per run.

## Actor input object example

```json
{
  "storyType": "top",
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "maxItems": 10 }

# Run the Actor and wait for it to finish
run = client.actor("parseforge/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10
}' |
apify call parseforge/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/hacker-news-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7lHgv9iIrszYTkSyC/builds/6rWnu8K4cVfzFqP6l/openapi.json