# HuggingFace Scraper — Models, Datasets & Spaces (`devilscrapes/huggingface-hub-scraper`) Actor

Export models, datasets, and Spaces from the HuggingFace Hub API — filter by task, library, or author, with a trending snapshot mode — to JSON or CSV. Richer schema than incumbents: downloads, likes, tags, license, last-modified. No login.

- **URL**: https://apify.com/devilscrapes/huggingface-hub-scraper.md
- **Developed by:** [DevilScrapes](https://apify.com/devilscrapes) (community)
- **Categories:** AI, Developer tools
- **Stats:** 1 total users, 0 monthly users, 96.6% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

<img src="https://apify.com/.actor/icon.svg" width="160" alt="HuggingFace Hub Scraper" />

## HuggingFace Scraper — Models, Datasets & Spaces

*We do the dirty work so your dataset stays clean.* 😈

**$2.20 / 1,000 rows** — pay only for results that land. No credit card to try.

Export structured metadata for models, datasets, and Spaces from the HuggingFace Hub. Filter by task tag, library, author, or free-text search. Trending snapshot mode included. One Actor handles all three repo types; Pydantic-validated rows land in a dataset you can download as JSON, CSV, Excel, or XML.

### 🎯 What this scrapes

Three repo types, one HuggingFace scraper:

1. **Models** — downloads, likes, pipeline tag, library name, tags, safetensors parameter count, GGUF file detection, and size-category bucketing.
2. **Datasets** — task categories, size categories, and language codes parsed from tag prefixes.
3. **Spaces** — SDK name and runtime stage (RUNNING / SLEEPING / STOPPED) in detail mode.

Five filter modes, mutually exclusive — use at most one per run:

- **Trending snapshot** — leave all filters blank to capture the top-N repos by `downloads`, `likes`, `trending`, `last_modified`, or `created_at`.
- **Tag filter** — pass `filterTags` to restrict to repos carrying specific tags (e.g. `text-generation`).
- **Search query** — pass `searchQuery` for free-text search across repo metadata.
- **Author** — pass `author` for every public repo from one org or user (e.g. `openai`).
- **Single-repo deep fetch** — pass `repoId` in `owner/name` form to call only the detail endpoint and emit one richly-enriched row.

### 🔥 Features

- Three repo types in one Actor: `model`, `dataset`, `space` — pick via the `repoType` selector.
- Five sort fields: `downloads`, `likes`, `trending`, `last_modified`, `created_at` (all descending).
- Five filter modes: tag list, free-text search, author/org, single-repo deep fetch, or no filter (trending snapshot).
- Optional `includeDetails` mode — calls the per-repo detail endpoint to enrich with safetensors parameter counts, GGUF file detection, and Space runtime stage.
- GGUF detection flag and derived `model_size_category` bucket (<1B / 1B-7B / 7B-13B / 13B+) ready for downstream pricing or hardware-fit dashboards.
- Dataset tag prefixes auto-parsed into structured arrays: `size_categories:`, `task_categories:`, `language:`.
- Pydantic v2 input validation — at most one filter may be set; invalid input fails fast with a clear error before any network call.
- Exponential backoff on `429` and `503`; honours `Retry-After`; max 5 attempts per endpoint call.
- Browser fingerprint impersonation via `curl-cffi` — no scraper-detectable headers leave the Actor.
- Companion to `llm-pricing-monitor` as part of the AI Stack Intelligence suite.

### 💡 Use cases

- **AI researcher trend tracking** — pull the trending top-100 models weekly and feed a time-series dashboard tracking which model families dominate the Hub.
- **Investor adoption monitoring** — measure download velocity for specific model families (`pipeline_tag=text-generation` + `library_name=transformers`) to inform AI infrastructure investment theses.
- **Fine-tuner catalog survey** — enumerate every model under a `pipeline_tag` like `image-segmentation` to map the open-weights landscape before choosing a base model.
- **Dataset discovery** — filter datasets by `task_categories` and `language` to find labelled training corpora for a downstream NLP/vision/audio model.
- **Hardware-fit analysis** — use `safetensors_total_params` and `model_size_category` to filter models that fit a target memory budget before benchmarking.
- **GGUF availability tracking** — set `includeDetails=true` and filter on `has_gguf=true` to find quantized inference-ready models for llama.cpp or LM Studio.
- **Space monitoring** — capture which Spaces are `RUNNING` vs `SLEEPING` for a creator or topic, useful for community health dashboards.
- **Content creator coverage** — feed every model from a popular org like `openai` or `meta-llama` into a content pipeline for blog posts or YouTube videos.
- **Track HuggingFace model downloads over time** — schedule recurring runs to build a time-series of download counts and detect fast-movers before they hit mainstream coverage.

### ⚙️ How to use it

1. Open the Actor input form.
2. Pick **Repo type** — `model`, `dataset`, or `space`.
3. Optionally pick a **Sort field** (default `downloads`).
4. Set **at most one** filter: `filterTags`, `searchQuery`, `author`, or `repoId`. Leave all four blank for a trending snapshot.
5. Adjust **Max results** (default 100, maximum 5000). Ignored in `repoId` mode (always 1 row).
6. Toggle **Include detail enrichment** on if you need safetensors parameter counts, GGUF detection, or Space runtime stage. Detail mode is charged at $0.005/row instead of $0.002/row.
7. Click **Start** and watch the run log. Results stream into the default dataset and can be downloaded as JSON, CSV, Excel, or XML via the **Export** button.

#### What we handle for you

We absorb every failure mode you'd otherwise hit running this yourself:

- **We rotate browser fingerprints** — `curl-cffi` impersonates real browser TLS signatures (Chrome / Firefox) so our requests look like genuine browser traffic.
- **We retry with exponential backoff** on `408 / 429 / 503` and honour `Retry-After`. Up to 5 attempts per page before surfacing a partial-success status.
- **We rotate proxies** through Apify Proxy on blocks — fresh session ID, fresh exit IP, back on track.
- **We back off when the target rate-limits** — partial successes surface with a clear status message; we never silently return an empty dataset.
- **We keep your dataset clean** — Pydantic-validated rows, ISO-8601 timestamps, stable repo IDs, `null` for absent optional fields rather than missing keys.
- **You pay only for results that land** — if no data comes back, you're not charged for rows (only the small Actor-start warm-up fee).

#### Quick examples

**Trending top-100 models, list mode:**

```json
{
  "repoType": "model",
  "sort": "downloads",
  "maxResults": 100,
  "includeDetails": false
}
```

**Every transformers text-generation model with safetensors and GGUF detection:**

```json
{
  "repoType": "model",
  "filterTags": ["text-generation", "transformers"],
  "maxResults": 500,
  "includeDetails": true
}
```

**Single-repo deep fetch:**

```json
{
  "repoType": "model",
  "repoId": "openai/whisper-large-v3"
}
```

**HuggingFace dataset export by language:**

```json
{
  "repoType": "dataset",
  "filterTags": ["language:fr"],
  "maxResults": 200,
  "includeDetails": false
}
```

### 📥 Input

| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `repoType` | string | yes | — | `model`, `dataset`, or `space` |
| `sort` | string | no | `downloads` | One of `downloads`, `likes`, `last_modified`, `created_at`, `trending` |
| `filterTags` | array | one-of | — | Tag filter; joined with comma for HF `filter=` param |
| `searchQuery` | string | one-of | — | Free-text search via HF `search=` |
| `author` | string | one-of | — | Org or user slug via HF `author=` |
| `repoId` | string | one-of | — | Single `owner/name` deep-fetch; forces detail mode |
| `maxResults` | integer | no | `100` | Max rows emitted (1 to 5000) |
| `includeDetails` | boolean | no | `false` | Per-row detail enrichment |
| `proxyConfiguration` | object | no | — | Apify Proxy configuration (recommended for high-volume runs) |

At most one of `filterTags`, `searchQuery`, `author`, `repoId` may be non-null. Setting two or more raises a Pydantic validation error before any network call.

### 📤 Output

One row per repo. Type-specific fields are `null` for repo types where they don't apply.

| Field | Type | Description |
|---|---|---|
| `repo_type` | string | One of `model`, `dataset`, `space` |
| `repo_id` | string | HuggingFace repo identifier in `owner/name` form |
| `repo_owner` | string | null | Owning org or user slug |
| `repo_name` | string | Last segment of `repo_id` |
| `repo_url` | string | `https://huggingface.co/{repo_id}` |
| `downloads` | integer | null | 30-day rolling download count (always null for Spaces) |
| `likes` | integer | Like count |
| `created_at` | string | ISO 8601 creation timestamp |
| `last_modified` | string | null | ISO 8601 last-modified timestamp |
| `tags` | array | Repo tags (may be empty) |
| `gated` | boolean | null | `manual` → true, false → false, absent → null |
| `private` | boolean | Always false for public catalog entries |
| `pipeline_tag` | string | null | Model pipeline tag (models only) |
| `library_name` | string | null | Model library name (models only) |
| `safetensors_total_params` | integer | null | Total safetensors parameter count (detail mode, models only) |
| `model_size_category` | string | null | Bucket label `<1B` / `1B-7B` / `7B-13B` / `13B+` |
| `has_gguf` | boolean | null | True if any sibling filename ends `.gguf` (detail mode) |
| `dataset_size_categories` | array | null | Stripped from `size_categories:` tag prefix (datasets only) |
| `dataset_task_categories` | array | null | Stripped from `task_categories:` tag prefix (datasets only) |
| `dataset_languages` | array | null | Stripped from `language:` tag prefix (datasets only) |
| `space_sdk` | string | null | Space SDK (`gradio`, `streamlit`, `docker`, `static`) |
| `space_runtime_stage` | string | null | Space runtime stage (detail mode, Spaces only) |
| `scraped_at` | string | ISO 8601 UTC datetime this row was written |

```json
{
  "repo_type": "model",
  "repo_id": "openai/whisper-large-v3",
  "repo_owner": "openai",
  "repo_name": "whisper-large-v3",
  "repo_url": "https://huggingface.co/openai/whisper-large-v3",
  "downloads": 4932732,
  "likes": 5690,
  "created_at": "2023-11-07T18:41:14.000Z",
  "last_modified": "2024-08-12T10:20:10.000Z",
  "tags": ["transformers", "safetensors", "whisper", "automatic-speech-recognition"],
  "gated": false,
  "private": false,
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "transformers",
  "safetensors_total_params": 1543490560,
  "model_size_category": "1B-7B",
  "has_gguf": false,
  "dataset_size_categories": null,
  "dataset_task_categories": null,
  "dataset_languages": null,
  "space_sdk": null,
  "space_runtime_stage": null,
  "scraped_at": "2026-05-16T12:00:00+00:00"
}
```

Optional fields are emitted as `null` when the API does not return them. Rows are never dropped for missing optional fields.

#### Export formats

After a run completes, click **Export** in the Apify Console to download:

- **JSON** — full fidelity, all fields, newline-delimited
- **CSV** — flat, one row per repo
- **Excel** — `.xlsx` via the Apify dataset converter
- **XML** — structured per-item

All formats are also available via the Apify API: `GET /datasets/{id}/items?format=csv&clean=true`.

### 💰 Pricing

Pay-Per-Event (PPE) — you pay only for what you use:

| Event | Price (USD) | When |
|---|---|---|
| `actor-start` | $0.20 | Once per run, at boot |
| `result-row` | $0.002 | Per repo row written in list mode (`includeDetails=false`) |
| `result-row-detailed` | $0.005 | Per repo row written in detail mode or `repoId` mode |

#### Example costs

| Rows scraped | Mode | Actor starts | Total cost |
|---|---|---|---|
| 100 | list | 1 | $0.25 |
| 500 | list | 1 | $1.05 |
| 1,000 | list | 1 | $2.20 |
| 1,000 | detail | 1 | $5.05 |
| 5,000 | list | 1 | $10.05 |
| 5,000 | detail | 1 | $25.05 |

Honest pricing, no fine print. Consistent with the `llm-pricing-monitor` companion Actor so the AI Stack Intelligence suite bills at a uniform rate.

### 🚧 Limitations

- **Private and gated repos are not accessible.** The unauthenticated public API only returns publicly visible data.
- **Rate limit: 500 requests per 5 minutes** (verified 2026-05-16 via `ratelimit-policy` header). At default page size 100, this allows ~10,000 list rows per 5-min window. Detail mode adds one request per row, halving throughput to ~250 enriched rows per minute.
- **Spaces never have a `downloads` metric.** The field is always `null` for `repo_type=space` — verified on both list and detail endpoints.
- **Sparse Spaces list:** `repo_owner`, `last_modified`, and `space_runtime_stage` require detail mode for Spaces.
- **Safetensors and GGUF fields need detail mode.** `safetensors_total_params`, `model_size_category`, and `has_gguf` are only populated when `includeDetails=true`.
- **No cross-run deduplication.** Re-running the same input returns the same repos with refreshed metadata. Use a downstream dedupe pass if you need uniqueness across runs.
- **No model card or dataset card markdown content.** Only structured metadata fields; the README body is excluded as too noisy for a structured dataset.
- **No HuggingFace Inference API calls or model benchmarking.** This Actor only scrapes catalog metadata, not model outputs.
- **The Apify FREE tier retains run-scoped storage for 7 days only.** For longer retention, export your dataset immediately or upgrade to a paid Apify plan.

#### Tips for best results

- **Use a trending snapshot weekly** to track the rapidly-evolving model leaderboard. Set up an Apify Schedule for a recurring run.
- **Cap `maxResults` to what you actually need.** The HF Hub has 1M+ models; setting a sensible cap keeps cost and runtime predictable.
- **Use detail mode sparingly.** It is 2.5x the per-row cost and roughly 4x the per-row latency. Prefer list mode for catalog snapshots; flip to detail mode only when you need safetensors, GGUF, or runtime stage.
- **Combine with `llm-pricing-monitor`** to correlate open-weights releases on the Hub with hosted-API price moves.

### ❓ FAQ

**Do I need a HuggingFace account or API token?**

No account and no API token are required to run this scraper. The HuggingFace Hub exposes read access to public repo metadata via a public REST API. This Actor uses that interface only — it never accesses gated content, never submits content, and never touches private repos.

**What is the difference between list mode and detail mode?**

List mode (`includeDetails=false`, $0.002/row) makes one API request per page of 100 rows and returns `repo_id`, owner, downloads, likes, tags, `pipeline_tag`, `library_name`, gated flag, and timestamps. Detail mode (`includeDetails=true`, $0.005/row) additionally calls the per-repo endpoint to fetch safetensors parameter counts, GGUF file detection, Space runtime stage, and dataset card data. Use detail mode when you need those enriched fields; otherwise list mode gives you 2.5x cheaper rows and faster throughput.

**How do I get trending models — huggingface trending models export?**

Leave all four filter fields (`filterTags`, `searchQuery`, `author`, `repoId`) blank, keep the default `sort=downloads`, and set `maxResults` to the top-N you want (e.g. 100). To use the HF native "trending" score instead of download count, set `sort=trending`.

**Can I do a HuggingFace dataset export or scrape Spaces too?**

Yes. Switch the `repoType` input to `dataset` or `space`. The same filtering, pagination, and detail-mode features apply across all three repo types. Note that Spaces never carry a `downloads` count and need detail mode for `repo_owner` / `last_modified` / `runtime_stage`.

**Why are some fields null on my rows?**

HuggingFace's list endpoint returns different field sets for different repo types. Space list rows lack `repo_owner`, `last_modified`, and `runtime` entirely — detail mode populates them. The `null` values are accurate, not bugs.

**What is GGUF and why detect it?**

GGUF (GPT-Generated Unified Format) is the quantized model file format used by llama.cpp, LM Studio, and Ollama for local CPU inference. When a model repo includes a `.gguf` sibling file, it can run on consumer hardware without a GPU. Set `includeDetails=true` and filter your dataset on `has_gguf=true` to list all inference-ready open-weights models.

**Is this an HuggingFace API wrapper — how does it differ from `huggingface_hub`?**

The official `huggingface_hub` Python library is excellent for one-off queries inside your own code. This Actor is designed for the *batched, scheduled, cross-author snapshot* use case: large paginated exports, recurring runs on Apify Schedules, clean CSV/JSON for BI tools and dashboards — without writing or maintaining any scraping infrastructure yourself.

**Is scraping the HuggingFace Hub legal?**

This Actor only reads publicly visible repository metadata via the documented public API. It never bypasses authentication, never accesses gated content, and never submits content. Always verify the current Terms of Service at `huggingface.co/terms-of-service` and your local data-protection rules before using scraped data commercially.

### 💬 Your feedback

Found a bug, hit a rate limit, or need a new field on the output row? Open an issue on the Actor's Apify Store page or contact the Devil Scrapes team at [apify.com/DevilScrapes](https://apify.com/DevilScrapes). We ship updates within days of validated reports.

# Actor input Schema

## `repoType` (type: `string`):

Which HuggingFace Hub repo type to scrape: <code>model</code>, <code>dataset</code>, or <code>space</code>.

## `sort` (type: `string`):

Sort the list endpoint by this field. <code>trending</code> maps to the HF <code>trendingScore</code> param; <code>last\_modified</code> → <code>lastModified</code>; <code>created\_at</code> → <code>createdAt</code>.

## `filterTags` (type: `array`):

Optional tag filter — joined with comma for the HF <code>filter=</code> query param. Example: <code>\["text-generation", "transformers"]</code>. Mutually exclusive with the other three filter inputs.

## `searchQuery` (type: `string`):

Optional free-text search via the HF <code>search=</code> query param. Mutually exclusive with the other three filter inputs.

## `author` (type: `string`):

Optional org or user slug via the HF <code>author=</code> query param (e.g. <code>openai</code>). Mutually exclusive with the other three filter inputs.

## `repoId` (type: `string`):

Optional single-repo deep-fetch in <code>owner/name</code> format. Skips the list endpoint and calls the per-repo detail endpoint directly. Forces detail mode. Mutually exclusive with the other three filter inputs.

## `maxResults` (type: `integer`):

Maximum number of rows emitted. Ignored in <code>repoId</code> mode (always 1 row).

## `includeDetails` (type: `boolean`):

If enabled, calls the per-repo detail endpoint for each row to enrich safetensors / siblings / runtime fields. Charged as <code>result-row-detailed</code> ($0.005/row).

## `useProxy` (type: `boolean`):

Route requests through Apify Proxy (<code>BUYPROXIES94952</code>). The HuggingFace Hub API does not block datacenter IPs — leave disabled unless you are behind a restrictive ISP.

## Actor input object example

```json
{
  "repoType": "model",
  "sort": "downloads",
  "maxResults": 100,
  "includeDetails": false,
  "useProxy": false
}
```

# Actor output Schema

## `datasetItems` (type: `string`):

All dataset items as JSON.

## `datasetItemsCsv` (type: `string`):

Same data exported to CSV.

## `datasetView` (type: `string`):

Open the run dataset in the Console.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "repoType": "model"
};

// Run the Actor and wait for it to finish
const run = await client.actor("devilscrapes/huggingface-hub-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "repoType": "model" }

# Run the Actor and wait for it to finish
run = client.actor("devilscrapes/huggingface-hub-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "repoType": "model"
}' |
apify call devilscrapes/huggingface-hub-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,devilscrapes/huggingface-hub-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/85rHzJCcMcSomrt03/builds/Z1eOC0VSVMHYSvp69/openapi.json