# PyPI Scraper (`parseforge/pypi-scraper`) Actor

Searches PyPI by keyword and returns each Python package as a flat row with metadata, download statistics, classifiers, and release history.

- **URL**: https://apify.com/parseforge/pypi-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $19.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### PyPI Scraper

**Search Python packages on PyPI and get metadata, download stats, classifiers, and release history for each one.** Feed it a keyword like 'web scraping' or 'django' and collect up to a million packages per run. Export to CSV, JSON, Excel, or XML.

PyPI's JSON API is rate-limited and returns one package at a time. This Actor searches the Python Package Index by keyword, reads the public package pages, and returns every match in a flat row. No API key, no registration, no per-package lookups.

| Who uses it | What they scrape PyPI for |
|---|---|
| Python developers | Find the most popular and well-maintained packages for a specific task before adding a dependency. |
| DevOps engineers | Audit package download trends and release frequency to decide which libraries to standardize on. |
| Security researchers | Collect package metadata and release histories to spot abandoned or rarely-updated dependencies in a supply chain. |
| Data analysts | Gather download counts and classifier distributions to report on Python ecosystem trends. |

### What it does

This Actor searches PyPI for Python packages by keyword and returns each package as a flat row with its metadata, download statistics, classifiers, and release information.

- 🔍 **Keyword search:** enter any term, from 'web scraping' to 'pytorch', and collect every matching package.
- 📊 **Download statistics:** recent download counts are included so you can gauge real-world usage.
- 🏷️ **Classifiers and metadata:** each row carries the full set of PyPI trove classifiers, license, author, and maintainer info.
- 📦 **Release history:** version strings and release dates let you see how actively a package is maintained.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with PyPI data

**📈 Compare package popularity.**

A data analyst searches 'data visualization' and collects download counts for the top 200 packages to recommend the most-used library to their team.

**🔒 Audit dependency health.**

A security engineer searches for packages their company uses, collects release dates, and flags any that have not been updated in over two years.

**🏷️ Map the Python ecosystem.**

A researcher searches broad terms like 'machine learning' and 'web framework', collects classifiers, and maps which categories are growing fastest.

**📦 Find the right tool for a job.**

A developer searches 'pdf generation', collects the top 50 results with their descriptions and Python version classifiers, and picks the best fit.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API key required** | Reads public PyPI pages directly, no registration or token management. |
| **Flat row per package** | Every package lands in your dataset as one row, ready for CSV, JSON, Excel, or XML export. |
| **Up to a million packages** | The maxItems limit lets you collect a handful of results or a full category sweep in one run. |

### How it compares

No other Store actor targets PyPI the same way, so the honest comparison is with the alternatives teams actually weigh.

| | PyPI Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When PyPI changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor with a single search keyword and cap the number of packages returned with the maxItems limit. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "search": "web scraping",
  "maxItems": 10
}
```

A larger pull:

```json
{
  "search": "web scraping",
  "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.021 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $2.10 |
| 1,000 results | $21.00 |
| 10,000 results | $210.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [PyPI Scraper](https://apify.com/parseforge/pypi-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to PyPI through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/pypi-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check your search keyword for typos. Try a broader term. PyPI search matches package names, summaries, and descriptions, so a very specific phrase may return nothing.

**Why did the run stop before reaching my maxItems limit?**

PyPI returned fewer total matches than your limit. The Actor collects every package that matches your keyword. If only 47 packages match, the run stops at 47.

**Why are download counts different from what I see on pypi.org?**

Download counts are cached and updated periodically. A small discrepancy between the Actor's result and the live page is normal and resolves within a day.

**Why is the run taking a long time?**

Collecting tens of thousands of packages requires many page reads. Increase your Actor's memory or reduce maxItems if you need faster results.

**Why are some fields empty for certain packages?**

Not every package maintainer fills in all metadata. Fields like author email or homepage are optional on PyPI and will be blank when the maintainer left them empty.

### FAQ

| Question | Answer |
|---|---|
| Can I scrape a specific package by name instead of searching? | Yes. Enter the exact package name as the search keyword. The Actor will return that package if it exists on PyPI. |
| What download stats are included? | The Actor collects recent download counts as displayed on the public PyPI package pages. These are the same numbers you see on pypi.org. |
| Does this Actor need a PyPI account or API token? | No. It reads the public package index pages directly. You do not need a PyPI account, an API key, or any registration. |
| How many packages can I collect in one run? | You set the limit with the maxItems field, up to one million packages per run. The Actor stops when it reaches that number or when PyPI has no more matches. |
| What output formats are supported? | Your dataset can be exported to CSV, JSON, Excel, or XML from the Apify platform. |
| Can I filter by Python version or license? | The Actor returns the full set of classifiers for each package. You can filter the resulting dataset in your own code or spreadsheet by the classifier fields. |
| Does it return release history? | Yes. Each row includes version strings and release dates so you can see the full release timeline for a package. |
| Is the search case-sensitive? | No. PyPI search is case-insensitive. Searching 'Django' and 'django' return the same results. |
| Can I schedule this Actor to run weekly? | Yes. Use Apify's scheduler to run the Actor on a recurring basis and track how package stats change over time. |
| What happens if my search keyword returns no results? | The run completes with an empty dataset. Try a broader or different keyword. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Python Software Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `search` (type: `string`):

Search term to find Python packages on PyPI (e.g. 'web scraping', 'machine learning', 'django').

## `maxItems` (type: `integer`):

How many packages to collect per run.

## Actor input object example

```json
{
  "search": "web scraping",
  "maxItems": 10
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "search": "web scraping",
    "maxItems": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/pypi-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "search": "web scraping",
    "maxItems": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/pypi-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "search": "web scraping",
  "maxItems": 10
}' |
apify call parseforge/pypi-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/pypi-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/gggPVFgjo1lbnVVEh/builds/MevNJTZ8w02QhObt1/openapi.json