Reddit Search Scraper

Search Reddit's public Atom feed by keyword and store normalized post records. The Actor uses direct HTTP requests, bounded retries, and optional Apify proxy configuration. It does not launch a browser, spoof fingerprints, open individual posts, or extract comments.

Input

Field	Type	Default	Description
searchQuery	string	required	Keyword or phrase, 1–200 characters
subreddit	string	none	Optional subreddit without r/
sortBy	string	relevance	relevance, new, hot, top, or comments
topTime	string	all	Time window used only with sortBy: "top"
maxItems	integer	25	Maximum unique posts, 1–100
maxPages	integer	3	Maximum feed pages, 1–10
proxyConfiguration	object	none	Optional Apify proxy configuration

Unknown fields and type coercion are rejected.

Output

Each dataset row is a real public feed entry with its Reddit ID, title, permalink, available author/subreddit/text/count metadata, search context, exact source feed URL, HTTP provenance, and explicit truth fields. Optional feed values are omitted when Reddit does not expose them. No placeholder or diagnostic row is written to the dataset.

The OUTPUT key-value-store record contains SUCCESS, PARTIAL, NO_RESULTS, or FAILED, request limits, page and row counts, proxy/browser provenance, and bounded failure messages. A failed run throws after writing this summary.

Limits

Reddit may return rate limits or access denials. The Actor retries retryable responses three times with backoff. Configure an appropriate Apify proxy if the run environment cannot access the public feed. Feed text and counts are limited to what Reddit includes in Atom; this Actor does not claim full post bodies or comment trees.

Local use

npm ci
$env:APIFY_INPUT_FILE = 'test-input.json'
npm start
node validate-datasets.js storage/datasets/default

validate-datasets.js accepts a dataset directory, a JSON file, or JSON/JSONL on standard input.