# Hex.pm Scraper (`parseforge/hex-pm-scraper`) Actor

Scrape Elixir and Erlang package metadata from Hex.pm by keyword. Returns downloads, version, license, and repository links as flat rows. Export to JSON, CSV, Excel, or XML.

- **URL**: https://apify.com/parseforge/hex-pm-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Hex.pm Scraper

**Scrape Elixir and Erlang package metadata from Hex.pm by keyword, sorted by downloads or recency, up to a million per run.** Every package row includes its downloads, version, license, and repository links. No API key or authentication. Export to JSON, CSV, Excel, or XML.

Hex.pm is the central package registry for the Elixir and Erlang ecosystems, but browsing it manually or through its API limits how many packages you can review at once. This Actor reads the public package listing directly, filters by a search keyword, and returns each match in one consistent flat schema. You can pull the most downloaded packages, the newest releases, or everything matching a library name like Phoenix or Ecto.

| Who uses it | What they scrape Hex.pm for |
|---|---|
| Elixir developers | Auditing dependency popularity and recent updates before adding a library to a project. |
| DevOps engineers | Tracking version histories and repository links across an organization's allowed package list. |
| Security researchers | Checking license types and latest versions for compliance across the Erlang/Elixir supply chain. |
| Market analysts | Measuring package download trends to gauge ecosystem growth and library adoption. |

### What it does

This Actor collects Hex.pm package listings by keyword and returns each package as a flat row with its downloads, version, license, and links.

- 🔍 **Keyword search:** Filter packages by name or term, like 'phoenix' or 'ecto', or leave empty to fetch the global list.
- 📊 **Flexible sorting:** Order results by total downloads, recent downloads, alphabetical name, or newest first insertion date.
- 📦 **Bulk collection:** Set a maximum from 1 up to 1,000,000 packages per run to fit a quick check or a full registry snapshot.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Hex.pm data

**📈 Monitor library adoption.**

A developer pulls the top 100 packages sorted by recent downloads every week to spot which new libraries the Elixir community is adopting.

**🔐 Audit license compliance.**

A security engineer scrapes all packages matching an internal allowlist by name, then reviews the license field in the exported CSV before a product release.

**🧪 Research package metadata.**

A data analyst collects the full list of packages sorted by insertion date to study how the Hex.pm registry has grown over time.

**🔗 Gather repository links.**

A platform team exports package rows to get the source code repository URL for every dependency in their monorepo.

### Why choose this scraper

| | What you get |
|---|---|
| **No API key required** | Reads the public Hex.pm pages directly with zero authentication or rate-limit setup. |
| **Fixed flat schema** | Every package row has the same columns, so you can append runs without reshaping. |
| **Sort by what matters** | Rank by total downloads for popularity, recent downloads for momentum, or newest first for fresh releases. |

### How it compares

Three other Hex.pm scrapers exist on Apify. This Actor focuses on keyword search with flexible sorting and a high item ceiling, matching the core listing data the competitors offer.

| Feature | ParseForge | Hexpm Scraper | Hex.pm Package Scraper | Hex.pm Scraper - Elixir & Erlang Packages |
|---|---|---|---|---|
| Keyword search filter | Yes | Not listed | Yes | Not listed |
| Sort by total downloads | Yes | Not listed | Not listed | Not listed |
| Sort by newest first | Yes | Not listed | Not listed | Not listed |
| Up to 1,000,000 packages per run | Yes | Not listed | Not listed | Not listed |
| License field | Yes | Not listed | Yes | Yes |
| Repository links | Yes | Not listed | Yes | Yes |

### Configure the run

Drive the Actor with a package name or keyword, choose a sort order, and set a maximum item count so only the packages you need land in your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10
}
```

A larger pull:

```json
{
 "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Hex.pm Scraper](https://apify.com/parseforge/hex-pm-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Hex.pm through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/hex-pm-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check your searchQuery spelling. Hex.pm search matches package names and descriptions. Try a broader term or leave the query empty to confirm the listing pages are reachable.

**The run stopped before reaching my maxItems.**

The Actor stops when it has exhausted the available listing pages for your query. If fewer packages exist than your maxItems value, the run ends early with whatever was found.

**Some fields are empty in my dataset.**

Not every Hex.pm package fills in all metadata. A missing license or repository link means the package author did not provide that information on Hex.pm.

**The sort order does not match what I see on the Hex.pm website.**

The Actor uses the same sort parameters that Hex.pm exposes on its package listing pages. Ensure you selected the correct enum value: downloads for total, recent for recent downloads, name for alphabetical, or inserted\_at for newest first.

**My run is taking a long time.**

Large maxItems values mean more pages to fetch. Reduce maxItems to a smaller number for a quicker test, then scale up when you are ready for a full collection.

### FAQ

| Question | Answer |
|---|---|
| Do I need a Hex.pm account or API key to use this scraper? | No. The Actor reads the public Hex.pm package listing pages directly. You do not need to register an account, create an API key, or handle OAuth. |
| What data does each package row contain? | Each row returns the package name, description, total downloads, recent downloads, latest version number, license, and repository links as they appear on Hex.pm. |
| Can I scrape all packages on Hex.pm? | Yes. Leave the search query empty and set the maximum items high enough to cover the registry. The Actor will paginate through the public listing until it hits your limit. |
| How do I get only the newest packages? | Set the sort order to 'Newest First' (inserted\_at). The Actor will return packages in reverse chronological order of when they were added to Hex.pm. |
| What export formats are supported? | You can export your dataset to JSON, CSV, Excel, or XML from the Apify platform after the run completes. |
| Can I search for multiple keywords in one run? | A single run accepts one search query. To collect packages for several keywords, run the Actor multiple times with different searchQuery values, or leave the query empty and filter the full dataset afterward. |
| Does this scraper get version history for each package? | The Actor returns the latest version number as shown on the package listing page. It does not fetch the full version history or changelog for each package. |
| Is there a rate limit? | The Actor respects Hex.pm's public pages. Apify's platform handles request pacing, but you should set a reasonable maxItems for your use case. |
| Can I filter by license type directly? | The input schema does not include a license filter. The Actor returns the license field for each package, and you can filter the exported dataset by license in your own tooling. |
| What is the difference between total downloads and recent downloads? | Total downloads is the all-time count for a package. Recent downloads reflects a shorter rolling window as reported by Hex.pm, useful for gauging current momentum. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Hex.pm. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `searchQuery` (type: `string`):

Filter packages by keyword (e.g. 'phoenix', 'ecto'). Leave empty to get all packages.

## `sortBy` (type: `string`):

How to sort the results

## `maxItems` (type: `integer`):

How many packages to collect per run.

## Actor input object example

```json
{
  "sortBy": "downloads",
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/hex-pm-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "maxItems": 10 }

# Run the Actor and wait for it to finish
run = client.actor("parseforge/hex-pm-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10
}' |
apify call parseforge/hex-pm-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/hex-pm-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Y23H8MVIpc654E9xo/builds/wgaD2w2Gesgtd08xj/openapi.json