# Puppet Forge Modules Scraper (`parseforge/puppet-forge-modules-scraper`) Actor

Scrapes Puppet Forge modules by keyword and returns each module as a flat row with name, summary, tags, and release metadata. Export to CSV, JSON, Excel, or XML.

- **URL**: https://apify.com/parseforge/puppet-forge-modules-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Puppet Forge Modules Scraper

**Scrape Puppet Forge modules by keyword, up to a million per run.** Each module comes with its name, summary, tags, and release metadata. No login or API key. Export to CSV, JSON, Excel, or XML.

Puppet Forge's official API needs an account and rate-limits you. This reads the public module list directly, filtered by keyword, and returns each match in one fixed schema.

| Who uses it | What they scrape Puppet Forge for |
|---|---|
| DevOps engineers | Which Puppet modules exist for a tool they need to manage |
| IT automation consultants | A list of modules to recommend for a client's infrastructure |
| Security researchers | Which modules are published for a specific software package |
| Market researchers | How many modules exist for a technology category |

### What it does

This Actor collects Puppet Forge modules by keyword and returns each one as a flat row.

- 🔍 **Keyword search:** find modules by name, summary, or tags, for example nginx, apache, mysql.
- 📦 **Full catalog:** leave the keyword empty to browse the complete module list.
- ⚙️ **Max items control:** set a hard cap from 1 to 1,000,000 modules per run.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Puppet Forge data

**🧰 Build a module inventory.**

A DevOps engineer searches for 'nginx' and exports every matching module to CSV to compare options.

**📊 Track module popularity.**

A market researcher collects all modules for a category and sorts by release date to spot trends.

**🔎 Audit available automation.**

A security researcher searches for a specific software name to see which modules exist before choosing one.

**📋 Recommend modules to clients.**

An IT consultant runs a keyword search and shares the JSON export with a client as a shortlist.

### Why choose this scraper

| | What you get |
|---|---|
| **No API key** | Reads the public module list directly, no registration or OAuth. |
| **One fixed schema** | Every module returns the same flat row, ready for analysis. |
| **Any volume** | Collect one module or a million, the cap is yours. |

### How it compares

This Actor focuses on keyword search and a simple flat schema, while the competitors below offer different metadata or export options.

| Feature | ParseForge | Puppet Forge Modules Scraper | Puppet Forge Scraper |
|---|---|---|---|
| Keyword search | Yes | Not listed | Not listed |
| Full catalog browsing | Yes | Not listed | Not listed |
| Max items control | Yes | Not listed | Not listed |
| Module name and summary | Yes | Yes | Not listed |
| Release metadata | Yes | Yes | Not listed |
| Versions and downloads | Not listed | Not listed | Yes |
| OS compatibility and dependencies | Not listed | Not listed | Yes |

### Configure the run

Drive the Actor from a keyword search, and the max items cap runs as modules are read so only the first N matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10
}
```

A larger pull:

```json
{
 "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Puppet Forge Modules Scraper](https://apify.com/parseforge/puppet-forge-modules-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Puppet Forge through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/puppet-forge-modules-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Your keyword may be too specific or misspelled. Try a broader term or leave the field empty to browse the full list.

**Why did the run stop before reaching my max items?**

The Actor stops when there are no more matching modules. If you expected more, try a different keyword or remove the keyword to get the full catalog.

**Why are some module fields empty?**

Puppet Forge does not always provide every field for every module. Empty values mean the data was not available on the public page.

**Can I get more metadata like downloads or dependencies?**

This Actor collects the public module list fields. For deeper metadata, you would need a scraper that calls the Puppet Forge API endpoints directly.

### FAQ

| Question | Answer |
|---|---|
| Do I need a Puppet Forge account or API key? | No. This Actor reads the public module list directly, so no registration or authentication is required. |
| What does each module record include? | Each record includes the module name, summary, tags, and release metadata as returned by Puppet Forge. The exact fields are shown in the sample output. |
| Can I scrape the entire Puppet Forge catalog? | Yes. Leave the keyword field empty and set a high max items value to collect the full list, up to one million modules per run. |
| How do I search for a specific module? | Enter a keyword in the search query field. The Actor matches against module name, summary, and tags. |
| What export formats are supported? | You can export the results as CSV, JSON, Excel, or XML from the Apify dataset. |
| Is there a limit on how many modules I can collect? | You set the limit yourself with the max items field, from 1 to 1,000,000 modules per run. |
| Does this Actor get module download counts or dependencies? | The Actor returns the fields available on the public module list. For deeper metadata like downloads or dependencies, consider a dedicated Puppet Forge API scraper. |
| Can I schedule this Actor to run regularly? | Yes, you can schedule it in Apify to run daily, weekly, or at any interval you need. |
| What if my keyword returns no results? | Check the spelling and try a broader term. The search matches name, summary, and tags, so a more general keyword may help. |
| Is this Actor free to use? | You can run it on the Apify free plan with a limited number of results. For larger runs, you need a paid plan. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Puppet, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `searchQuery` (type: `string`):

Optional. Keyword to search Puppet Forge modules by name, summary, or tags (for example nginx, apache, mysql). Leave empty to browse the full module list.

## `maxItems` (type: `integer`):

How many modules to collect per run.

## Actor input object example

```json
{
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "",
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/puppet-forge-modules-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQuery": "",
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/puppet-forge-modules-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": "",
  "maxItems": 10
}' |
apify call parseforge/puppet-forge-modules-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/puppet-forge-modules-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/tA8dAlBvaHANUFI4I/builds/QKdMpUXhoaTRfA3IE/openapi.json