# Hacker News Scraper — Stories, Comments, Ask HN, Show HN (`foxlabs/hackernews-intelligence`) Actor

Scrape Hacker News stories and full comment threads via the official Firebase API. Top stories, Ask HN, Show HN, jobs, new. Built for founder/VC sentiment monitoring, topic clustering, trend tracking. Clean JSON, comment tree depth-controlled.

- **URL**: https://apify.com/foxlabs/hackernews-intelligence.md
- **Developed by:** [Berkan Kaplan](https://apify.com/foxlabs) (community)
- **Categories:** Developer tools, News
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Hacker News Intelligence 🟠

🎉 Turn Hacker News into clean, structured intelligence — no login, no API key, one row per item, with the title, author, points, comments and links.
Built for developer-tool marketing, hiring and tech research.

### 🔍 What is the Hacker News Intelligence — and when should you use it?

Give this actor keywords or topics and it returns matching items from public Hacker News posts and comments — as clean, deduplicated rows you can filter, export or feed to an AI agent. Every run queries the source live, so the data is as fresh as the registry itself.

**Use it when you need:** a HN item company list for outreach; formation / status monitoring; or a canonical registry record for KYB and due diligence.

**Use something else when:** you need private analytics — this reads public Hacker News content only.

### 🤖 Use with AI agents

**Already on the Apify MCP server?** Ask for this Actor by name: `foxlabs/hackernews-intelligence`.

**Your agent can pay for its own runs.** This Actor is pay-per-event with agentic payments, so an agent can discover it, run it and settle the bill over **x402 (USDC on Base)** or **Skyfire** — no Apify account or API token of its own. Billing is the same either way: per delivered record, never for errors.

Otherwise paste this into Claude, ChatGPT, Cursor or any MCP-enabled assistant:

```
I want to pull HN item company records using the Apify Actor `foxlabs/hackernews-intelligence`.
Input: `queries` is a list of keywords or topics. `maxResultsPerQuery` caps rows per query.
Start with: {"queries":["undefined"],"maxResultsPerQuery":50}
Ask me what to look up, run the Actor, then summarise the rows as a table.
```

The machine-readable API, MCP config and OpenAPI definition live at `apify.com/foxlabs/hackernews-intelligence.md`.

### 📋 Overview

Everything you need to turn public Hacker News posts and comments into clean, structured data — in one actor, with no login, cookies or API key.

**Why teams pick this actor:**

- ✅ **Whole feed, one call** — name or ID in, matching items out.
- 🧹 **No empty-promise columns** — only fields this registry actually fills; degenerate columns are removed.
- 🔗 **Stable identifiers** — every row carries the source's own IDs, ready to join across runs and to other Fox Labs actors.
- 💰 **Pay only for results** — per-row pricing, empty/failed lookups never billed.
- 🤖 **Agent-ready** — MCP + x402 agentic payments.

### ✨ Features

- 🔍 **Name or ID lookup** — relevance-ranked name search or exact registry-ID lookup.
- 🏢 **Full entity profile** — status, legal form, formation date, address and the registry’s own contact fields.
- 🧹 **Clean schema** — deduplicated camelCase rows, ready for CSV/Excel/JSON.

### 🎬 Quick Start

```bash
curl -X POST "https://api.apify.com/v2/acts/foxlabs~hackernews-intelligence/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries":["undefined"],"maxResultsPerQuery":50}'
```

### 🚀 Getting Started (3 steps)

1. **Choose your targets** — keywords or topics.
2. **Set the cap** — `maxResultsPerQuery` limits rows per query.
3. **Run and export** — get a clean dataset as JSON, CSV or Excel.

### 📥 Input

```json
{"queries":["undefined"],"maxResultsPerQuery":50}
```

| Field | Type | Description |
|---|---|---|
| `queries` | array | Keywords or topics. |
| `maxResultsPerQuery` | integer | Caps rows per query. |
| `maxConcurrency` | integer | How many queries to fetch at once. |
| `includeRaw` | boolean | Attach the source’s untouched record under `raw`. |

### 📤 Output

One row per company, saved to the dataset. Every row also carries `query`, `scrapedAt`, and — when a lookup fails — an `error` explaining why (never silently dropped, never billed).

| Field | Description |
|---|---|
| `type` | Type |
| `threadId` | Thread Id |
| `threadTitle` | Thread Title |
| `company` | Company |
| `headline` | Headline |
| `roleLine` | Role Line |
| `remote` | Remote |
| `urls` | Urls |
| `author` | Author |
| `hnUrl` | Hn Url |
| `createdAt` | Created At |
| `text` | Text |
| `fetchedAt` | Fetched At |

### 💼 Use cases

**1. Launch monitoring** — track HN discussion of a product.
*Input:* product keywords. *Output:* items + points + comments. *Use:* a launch dashboard.

**2. Hiring signals** — read "Who is hiring" threads.
*Input:* keywords. *Output:* items + text. *Use:* a hiring list.

**3. Tech trends** — spot rising topics on HN.
*Input:* topic keywords. *Output:* items over time. *Use:* a trend report.

### 🔗 Integration

**JavaScript / Node.js**

```javascript
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('foxlabs/hackernews-intelligence').call({"queries":["undefined"],"maxResultsPerQuery":50});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0]);
```

**Python**

```python
from apify_client import ApifyClient
client = ApifyClient('YOUR_TOKEN')
run = client.actor('foxlabs/hackernews-intelligence').call(run_input={"queries":["undefined"],"maxResultsPerQuery":50})
for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item)
```

**Automation (n8n / Zapier / Make):** schedule or webhook → HTTP request to the actor API with your `queries` → handle the JSON dataset → push to a sheet, CRM or dashboard.

### 📊 Pricing

Pay-per-event: **per delivered record**. Empty or failed lookups are never billed. [View current pricing.](https://apify.com/foxlabs/hackernews-intelligence)

### ❓ FAQ

**Do I need an account, login or API key?** No. This reads public Hacker News posts and comments.

**What do I search by?** Keywords or topics.

**How current is the data?** Every run queries the source live, so results are as fresh as the registry.

**What does each row cover?** One HN item: title, author, points, comment count, URL and text where present.

**Can I export to CSV / Excel / JSON?** Yes — directly from the Apify dataset.

### 🐛 Troubleshooting

- **Fewer rows than expected** — raise `maxResultsPerQuery`, or refine the name.
- **A name returns an unexpected entity** — it matched a similar registered name; search the exact registry ID.
- **No rows for a name** — try the entity’s exact legal name or its registry ID.

### ⚖️ Is it legal to scrape this data?

This actor reads public Hacker News posts and comments. Results can still contain personal data (e.g. a person’s name); personal data is protected by the GDPR and similar laws, so only process it with a legitimate basis. See Apify’s blog post on the legality of web scraping.

### 🤝 Support & contact

- 🌐 **Website:** [data.foxlabs.com.tr](https://data.foxlabs.com.tr)
- 📧 **Email:** info@foxlabs.com.tr
- 🐛 **Issues:** open a ticket in the Actor’s **Issues** tab
- 🧰 **More clean B2B data actors:** [Fox Labs on Apify](https://apify.com/foxlabs)

### Changelog

#### 0.2 — 2026-09-07

- **Dropped empty-promise columns.** Removed `emails` — public Hacker News posts and comments does not carry them, so they were shipped as always-null columns. Only fields this source actually fills are now emitted.
- **Enabled AI-agent payments (x402) + rebuilt the README to the full standard** (What-is / when, AI-agents + x402 agentic payments + MCP, Overview, Features, Use cases, Integration, FAQ, Troubleshooting, Support & contact).

#### 0.0

- Initial release: data from public Hacker News posts and comments by name or registry ID.

# Actor input Schema

## `mode` (type: `string`):

Search = query the full HN archive (stories & comments) with points/date filters — unlimited depth, no 1,000-result wall. Who is hiring = parse the monthly 'Ask HN: Who is hiring?' thread into structured job posts (company, role, remote, e-mails). Feed = live front page / new / best / ask / show / jobs.

## `query` (type: `string`):

Search mode. What to search for — a brand, product, topic (e.g. "postgres", "Supabase", "rust async"). Leave empty to browse everything in the date window.

## `searchType` (type: `string`):

Search mode. Stories = titles/links (launches, articles). Comments = the discussion text (where brand mentions and opinions actually live). Both = the two merged.

## `sortBy` (type: `string`):

Date (default) streams newest-first with UNLIMITED depth. Relevance uses the API's ranked order but is hard-capped at 1,000 results by the API.

## `minPoints` (type: `integer`):

Search mode, stories only. Keep only stories with at least this many upvotes (e.g. 100 = front-page material). Leave empty for all.

## `minComments` (type: `integer`):

Search mode, stories only. Keep only stories with at least this many comments. Leave empty for all.

## `author` (type: `string`):

Search mode. Only items by this HN username (e.g. "patio11"). Leave empty for all authors.

## `threadType` (type: `string`):

Which monthly thread to parse: companies hiring (default), people looking for work, or the freelancer thread.

## `month` (type: `string`):

Which month's thread, e.g. 2026-07. Leave empty for the latest thread.

## `datePreset` (type: `string`):

Search mode: how far back to search. The archive goes back to 2007 — 'All time' really is all time.

## `dateFrom` (type: `string`):

YYYY-MM-DD. Only used when Date range = Custom.

## `dateTo` (type: `string`):

YYYY-MM-DD. Only used when Date range = Custom. Leave empty for 'today'.

## `feed` (type: `string`):

Feed mode: which live list to pull.

## `includeComments` (type: `boolean`):

Feed mode: also walk each story's comment tree into the row (slower).

## `commentDepth` (type: `integer`):

Feed mode, with comments on: how deep to walk each thread (1-5).

## `maxCommentsPerStory` (type: `integer`):

Feed mode, with comments on: cap per story (1-500).

## `maxResults` (type: `integer`):

Hard cap on dataset rows. Set 0 for unlimited (the sliding-window search really can walk the whole archive).

## Actor input object example

```json
{
  "mode": "who_is_hiring",
  "searchType": "stories",
  "sortBy": "date",
  "threadType": "hiring",
  "datePreset": "all_time",
  "feed": "top",
  "includeComments": false,
  "commentDepth": 2,
  "maxCommentsPerStory": 50,
  "maxResults": 1000
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "who_is_hiring",
    "query": "",
    "threadType": "hiring",
    "maxResults": 1000
};

// Run the Actor and wait for it to finish
const run = await client.actor("foxlabs/hackernews-intelligence").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "who_is_hiring",
    "query": "",
    "threadType": "hiring",
    "maxResults": 1000,
}

# Run the Actor and wait for it to finish
run = client.actor("foxlabs/hackernews-intelligence").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "who_is_hiring",
  "query": "",
  "threadType": "hiring",
  "maxResults": 1000
}' |
apify call foxlabs/hackernews-intelligence --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,foxlabs/hackernews-intelligence"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/IzFuDRcqNz0rwi51I/builds/gzEYarpAsj7KzjJtN/openapi.json