Hacker News Scraper — Top, New, Best, Ask, Show Stories

What does Hacker News Scraper do?

Hacker News Scraper is an Apify Actor that extracts structured story data from Hacker News using the official HN Firebase API — no API key or login required. It supports all five story feeds (top, new, best, Ask HN, Show HN), filters by minimum score, optionally fetches top-level comments, and enriches every story with 13 analytics fields including AI/crypto relevance, developer sentiment, innovation signal, and ecosystem category. This HN posts API turns raw Hacker News feeds into analytics-ready data in seconds.

Why use Hacker News Scraper?

As the home of technical discourse and early-adopter opinion, Hacker News is one of the most valuable signals for tech news data extraction. Key use cases:

• Tech trend monitoring — identify emerging technologies before they hit mainstream press. The innovation_signal field flags stories that deviate from baseline community attention.
• Competitive intelligence — monitor mentions of your product, competitors, or key technical topics in real time using the minScore filter to surface only high-traction discussions.
• AI/crypto signal extraction — the built-in ai_crypto_relevance field scores each story's relevance to AI and cryptocurrency topics, saving you a secondary classification step.
• Developer sentiment analysis — the developer_sentiment_score field provides a pre-computed polarity rating per story, ready for downstream NLP pipelines or dashboards.
• Content curation and newsletters — automate sourcing of the day's top engineering and startup stories for internal newsletters or Slack digests.

How to use Hacker News Scraper

1. Open the Actor on Apify Store and click Try for free.
2. Select a feed mode — choose top (front page), new (newest submissions), best (all-time quality), ask (Ask HN threads), or show (Show HN demos).
3. Set the item limit — enter the number of stories to fetch (1–500). Default is 30.
4. Filter by score — set minScore to skip low-traction posts (e.g. 100 returns only stories with 100+ points).
5. Enable comments (optional) — toggle includeComments and set maxComments per story.
6. Run and export — results land in the dataset tab as structured JSON, exportable as JSON, CSV, or XLSX.

Input

Parameter	Type	Default	Description
mode	string	top	Story feed: top, new, best, ask, show
maxItems	integer	30	Max stories to fetch (1–500)
includeComments	boolean	false	Fetch top-level comments per story
maxComments	integer	10	Max top-level comments per story when includeComments is true
minScore	integer	0	Minimum score filter — 0 disables the filter

Example input — top 50 stories with score ≥ 100, including 5 comments each:

{
  "mode": "top",
  "maxItems": 50,
  "includeComments": true,
  "maxComments": 5,
  "minScore": 100
}

Output

Each story produces one dataset record. The 13 analytics fields are appended automatically.

[
  {
    "id": 42817305,
    "type": "story",
    "title": "Show HN: I built a local-first AI code review tool",
    "url": "https://example.com/ai-code-review",
    "hn_url": "https://news.ycombinator.com/item?id=42817305",
    "author": "pg",
    "score": 512,
    "comments_count": 148,
    "created_at": "2026-09-01T10:00:00+00:00",
    "text": null,
    "ai_crypto_relevance": 0.92,
    "developer_sentiment_score": 0.74,
    "innovation_signal": "high",
    "content_category": "tooling",
    "ecosystem_category": "ai/ml",
    "top_comments": [
      {
        "id": 42817306,
        "author": "dang",
        "text": "Really impressive — how does it handle multi-file context?",
        "created_at": "2026-09-01T10:05:00+00:00"
      }
    ]
  }
]

Data fields

Field	Type	Description
id	integer	HN item ID
type	string	Always story for stories
title	string	Story headline
url	string	External link (null for Ask/Show HN text posts)
hn_url	string	Direct Hacker News discussion link
author	string	HN username of the submitter
score	integer	Current upvote count
comments_count	integer	Total comment count
created_at	string	ISO 8601 submission timestamp
text	string	Body text for Ask/Show HN posts (null for link posts)
ai_crypto_relevance	float	0–1 score for AI/crypto topic relevance
developer_sentiment_score	float	Polarity score for developer community sentiment
innovation_signal	string	low, medium, or high innovation signal
content_category	string	Topic category (e.g. tooling, research, business)
ecosystem_category	string	Ecosystem tag (e.g. ai/ml, crypto, web, devops)
top_comments	array	Array of comment objects (when includeComments is true)

Cost estimation

This Actor uses Pay-Per-Event (PPE) pricing — you pay only for stories actually scraped.

Event	Price	Notes
Actor start	$0.005	Charged once per run
Per story scraped	$0.002	Charged per item returned

Typical run	Stories	Estimated cost
30 stories (default)	30	~$0.07
100 top stories	100	~$0.21
500 stories (full feed)	500	~$1.01
Daily monitor (30 stories/day × 30 days)	900	~$1.81/month

FAQ

Does this Actor require a Hacker News account or API key? No. It uses the official HN Firebase REST API, which is fully public and requires no authentication. There are no rate limits per HN guidelines.

How fast does the scraper run? The Actor fetches up to 10 stories concurrently. A run of 100 stories typically completes in under 60 seconds, including comment enrichment.

Can I use this for real-time monitoring? Yes — schedule this Actor on Apify to run every hour or day. Combine with minScore filtering and the ai_crypto_relevance field to build a real-time tech intelligence feed without any manual filtering.

---

Disclaimer: This Actor depends on the Hacker News Firebase API maintained by Y Combinator. Data accuracy and availability are subject to the upstream API. The analytics fields (sentiment, relevance scores) are computed heuristically and should be treated as signals rather than ground truth.

---

Related actors

• Tech News Aggregator — TechCrunch/The Verge/Wired coverage to pair with HN community reaction
• Reddit Thread Analyzer — r/programming, r/technology community sentiment alongside HN developer discussions
• GitHub Trending Tracker — Repos that frequently appear in HN "Show HN" posts and front-page discussions

Feedback

If this actor helps your tech research, a review helps others find it: Leave a review on Apify Store