# Hacker News Items Scraper (`parseforge/hackernews-items-scraper`) Actor

Collects Hacker News items from the top, new, best, ask, show, or job story lists. Each item is returned as a flat row with title, URL, score, author, and comment count.

- **URL**: https://apify.com/parseforge/hackernews-items-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** News, Social media
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Hacker News Items Scraper

**Scrape Hacker News items from any story list, up to a million per run.** Every item comes with its title, URL, score, author, and comment count. No API key or login. Export to CSV, JSON, Excel, or XML.

Hacker News is the tech industry's water cooler, but the official Firebase API rate-limits you and returns raw nested data. This Actor reads the public story feeds directly, lets you pick any list from Top to Jobs, and returns each item in one flat, predictable row.

| Who uses it | What they scrape Hacker News for |
|---|---|
| Product managers | Spot which tools and startups are gaining traction on Show HN this month. |
| Market researchers | Track the topics and technologies the engineering community cares about right now. |
| Job seekers | Pull the latest Who is hiring? posts and YC-backed job listings into a spreadsheet. |
| Content curators | Build a daily digest of the highest-scoring articles without manual copying. |

### What it does

This Actor collects Hacker News items from the top, new, best, ask, show, or job story lists and returns each one as a flat row.

- 📋 **Six story lists:** top, new, best, ask, show, and job stories, each as a separate run.
- 🔢 **Controlled volume:** set a hard cap from 1 to 1,000,000 items so you never overspend.
- 📄 **Flat output:** every item is a single row with title, URL, score, author, and comment count.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Hacker News data

**📈 Track rising startups.**

A VC analyst runs the Actor on Show HN every Monday, exports the top 100 items to CSV, and flags companies that hit the front page for the second time.

**🧠 Monitor engineering trends.**

A developer advocate scrapes the top stories daily, groups titles by keyword, and reports which frameworks and languages are dominating the conversation.

**💼 Collect job listings.**

A job seeker pulls the latest 200 items from the Jobs list, filters by title keywords, and applies to the five most relevant YC-backed roles.

**📰 Curate a newsletter.**

A newsletter author scrapes the best stories every morning, picks the three with the highest comment-to-score ratio, and writes a daily briefing.

### Why choose this scraper

| | What you get |
|---|---|
| **No API key** | Reads the public feeds directly. No registration, no OAuth, no rate-limit headaches. |
| **Fixed schema** | Every item lands as a flat row with the same columns, ready for analysis. |
| **Any list** | Top, new, best, ask, show, or job. One Actor covers every official feed. |
| **Scalable** | Grab ten items for a quick check or a million for a full archive. |

### How it compares

No other Store actor targets Hacker News the same way, so the honest comparison is with the alternatives teams actually weigh.

| | Hacker News Items Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Hacker News changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor by picking one story list and a maximum item count, and it stops the moment that many items are collected. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10
}
```

A larger pull:

```json
{
 "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Hacker News Items Scraper](https://apify.com/parseforge/hackernews-items-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Hacker News through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/hackernews-items-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting fewer items than my maxItems setting?**

The story list may have fewer items available at that moment. For example, the Jobs list rarely has thousands of items. Try a larger list like Top or New, or lower your maxItems to match the available count.

**The run failed with a timeout error.**

Requesting a very large number of items can take time. Increase the Actor's timeout in the run settings, or lower your maxItems and run it more frequently on a schedule.

**Some fields are empty in my dataset.**

Not every Hacker News item has every field. Job posts often lack a score, and some items may not have a URL if they are text-only Ask HN posts. Empty fields are expected for those item types.

**I need the full text of the linked articles, not the titles.**

This Actor returns the Hacker News item metadata only. To get the full article text behind each URL, pair this Actor with a Web Scraper or use a dedicated article extractor on the resulting URL column.

**The data looks different from what I see on the Hacker News website.**

The Actor reads the same public feed that powers the website. Differences usually come from caching. Run the Actor again after a minute to get the latest state of the list.

### FAQ

| Question | Answer |
|---|---|
| Do I need a Hacker News account or API key? | No. The Actor reads the public story feeds that anyone can visit in a browser. No login, no API key, and no OAuth flow. |
| What is the difference between the story lists? | Top stories are ranked by a points-and-recency algorithm. New stories are the latest submissions in chronological order. Best stories are the highest-scoring items. Ask HN, Show HN, and Jobs are filtered to those specific post types. |
| How many items can I scrape in one run? | You set the limit. The minimum is 1 and the maximum is 1,000,000 items per run. The Actor stops as soon as it reaches your number. |
| Does this Actor get comments too? | No. This Actor returns the story-level item: title, URL, score, author, and comment count. It does not fetch the comment threads themselves. |
| Can I scrape a specific user's submissions? | Not with this Actor. It is built for the six official story lists. To scrape a user's posts you would need a different approach. |
| What format does the data come in? | You can export your dataset as CSV, JSON, Excel, or XML from the Apify platform. The schema is the same flat row regardless of format. |
| How often does the data refresh? | The Actor reads the live feed at the moment you run it. Schedule it to run every hour, every day, or on any cron schedule you choose. |
| Is this faster than the official Firebase API? | It avoids the Firebase API's rate limits and nested JSON. For bulk reads of story lists it is simpler and you control the throughput. |
| Can I filter by score or author? | The Actor pulls items in the order the list provides them. It does not filter by score or author mid-run. You can filter the exported dataset afterward. |
| What happens if Hacker News is down? | The Actor will retry automatically and then fail the run with an error message. You can set up automatic retries in the Apify scheduler. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Y Combinator. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

How many Hacker News items to collect per run.

## `listing` (type: `string`):

Which Hacker News list to export.

## Actor input object example

```json
{
  "maxItems": 10,
  "listing": "top"
}
```

# Actor output Schema

## `results` (type: `string`):

Records produced by Hacker News Items Scraper.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/hackernews-items-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "maxItems": 10 }

# Run the Actor and wait for it to finish
run = client.actor("parseforge/hackernews-items-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10
}' |
apify call parseforge/hackernews-items-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/hackernews-items-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/qf5zDffvmQb4ZTwHT/builds/3bcnIyc3SMUF3okSJ/openapi.json