# OPAM OCaml Packages Scraper (`parseforge/opam-ocaml-scraper`) Actor

Scrapes OCaml package metadata from the OPAM repository. Filter by package name or keyword, or collect the full index. Each package is returned as a flat row with its version, dependencies, license, synopsis, and maintainer.

- **URL**: https://apify.com/parseforge/opam-ocaml-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### OPAM OCaml Packages Scraper

**Scrape OCaml package metadata from the OPAM repository by keyword or from the full index, up to a million per run.** Every package comes with its version, dependencies, license, synopsis, and maintainer. Export to CSV, JSON, Excel, or XML.

Finding the right OCaml library means browsing the OPAM package index by hand or writing one-off scripts against the repository. This reads the public package listing directly, filters by package name or keyword, and returns each match in one fixed schema. No API key, no rate-limit headaches.

| Who uses it | What they scrape OPAM for |
|---|---|
| OCaml developers | Audit the dependency tree and license of every package a project pulls in. |
| DevOps engineers | Build a private mirror of allowed OPAM packages with their checksums and URLs. |
| Security researchers | Scan the OPAM ecosystem for packages with outdated dependencies or missing maintainers. |
| Data analysts | Chart the growth of the OCaml package ecosystem over time by version count and author. |

### What it does

This Actor collects OCaml package records from the OPAM repository by keyword search or from the full index, and returns each one as a flat row.

- 🔎 **Keyword filter:** supply a package name or keyword like lwt, dune, or async and collect only matching packages.
- 📦 **Full-index mode:** leave the search field empty to walk the entire OPAM package listing from the top.
- 🧮 **Volume control:** set a ceiling from 1 to 1,000,000 packages so a quick spot-check does not turn into a full mirror.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with OPAM data

**📋 Audit project dependencies.**

A lead developer searches for every package a codebase depends on, collects the license and maintainer fields, and decides which ones need a legal review before the next release.

**🔐 Build an allowed-package mirror.**

A platform team scrapes the full OPAM index, filters by an internal allowlist of licenses, and publishes only vetted packages to a private opam-repository.

**📊 Track ecosystem growth.**

A researcher runs the Actor weekly with no keyword, collects the version count and author list, and plots the month-over-month growth of the OCaml package set.

**⚠️ Find unmaintained packages.**

A security engineer scrapes the index, filters for packages whose latest version is older than two years, and flags them for replacement or forking.

### Why choose this scraper

| | What you get |
|---|---|
| **Flat row per package** | One package equals one row with its name, version, synopsis, license, dependencies, and maintainer. |
| **No API registration** | Reads the public OPAM index directly. No app registration, no OAuth, no token refresh. |
| **Fixed schema** | Every run returns the same columns, so your downstream pipeline never breaks on a schema change. |

### How it compares

No other Store actor targets OPAM the same way, so the honest comparison is with the alternatives teams actually weigh.

| | OPAM OCaml Packages Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When OPAM changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor from a package-name keyword or leave it empty for the full index, and the filter runs as each package is read so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10
}
```

A larger pull:

```json
{
 "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [OPAM OCaml Packages Scraper](https://apify.com/parseforge/opam-ocaml-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to OPAM through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/opam-ocaml-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check the search keyword. It must match part of a package name. Try a broader term or leave the field empty to scrape the full index. Also confirm that maxItems is set to a number greater than zero.

**The run stopped before collecting all packages I wanted.**

Increase the maxItems value. The Actor stops when it hits that ceiling. If you need the full index, set maxItems to a number larger than the total package count.

**Some fields are empty in the output.**

Not every OPAM package declares every metadata field. A missing maintainer or license in the output means the package did not publish that information upstream.

**The run timed out.**

A full-index scrape with a very high maxItems value can hit the default timeout. Increase the run timeout in the Actor's advanced settings, or raise the memory allocation.

**I got an error about the website being unreachable.**

The OPAM repository may have been temporarily down. Retry the run after a few minutes. If the error persists, check the OPAM status page or contact support.

### FAQ

| Question | Answer |
|---|---|
| Do I need an OPAM account or API key? | No. The Actor reads the public OPAM package index. You do not need an account, an API key, or any authentication. |
| What does one row of output look like? | Each row is a flat JSON object with the package name, version, synopsis, license, dependencies, maintainer, and other metadata fields. The exact field list is shown in the sample output on the Actor page. |
| Can I scrape the entire OPAM repository? | Yes. Leave the search field empty and set the maximum packages high enough. The Actor will walk the full index and return every package it finds. |
| How do I filter for a specific package? | Type a package name or keyword into the search field. The Actor will return only packages whose name contains that text. |
| Does this scrape package source code or tarballs? | No. It collects the metadata record for each package: name, version, dependencies, license, synopsis, and maintainer. It does not download source archives. |
| What export formats are supported? | You can export the dataset to CSV, JSON, Excel, or XML from the Apify dataset tab after the run finishes. |
| How long does a full-index scrape take? | It depends on the number of packages and the Actor's memory allocation. A few thousand packages finish in minutes. A full mirror of the entire OPAM index may take longer. |
| Can I run this on a schedule? | Yes. Use Apify's scheduler to run the Actor daily or weekly and track new packages or version bumps automatically. |
| What happens if the OPAM website changes its layout? | The Actor is maintained to track the OPAM repository structure. If a change breaks the scraper, report it and we will update the parser. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by OCaml.org. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `search` (type: `string`):

Optional. Keep only packages whose name contains this text (for example lwt, async, dune). Leave empty to scrape from the top of the full index.

## `maxItems` (type: `integer`):

How many packages to collect per run.

## Actor input object example

```json
{
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "search": "",
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/opam-ocaml-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "search": "",
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/opam-ocaml-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "search": "",
  "maxItems": 10
}' |
apify call parseforge/opam-ocaml-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/opam-ocaml-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/3ZFGbCiHRilGpPIMa/builds/7WKsVO70a01Y84IDU/openapi.json