# PyPI Packages Scraper (`parseforge/pypi-packages-scraper`) Actor

Scrapes PyPI package metadata by exact name or keyword search. Returns each package as a flat row with name, version, summary, author, license, and more.

- **URL**: https://apify.com/parseforge/pypi-packages-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Automation, Education
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $12.00 / 1,000 result items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### PyPI Packages Scraper

**Scrape PyPI package metadata for any Python package or keyword, up to a million per run.** Get name, version, summary, author, license, and more in one flat row. No API key or login. Export to CSV, JSON, Excel, or XML.

PyPI's official JSON API is fine for one package at a time, but bulk lookups and keyword discovery mean writing your own client and handling rate limits. This Actor reads the public PyPI index directly, in lookup or search mode, and returns each matching package as a clean row. Filter by exact name or keyword, and cap the run at any number up to a million.

| Who uses it | What they scrape PyPI for |
|---|---|
| Python developers | Audit the dependencies of a project before upgrading |
| Security researchers | Find packages with suspicious names or metadata |
| Data analysts | Build a dataset of package popularity and release activity |
| Open source maintainers | Monitor packages that depend on their own libraries |

### What it does

This Actor collects PyPI package metadata by exact name or keyword search, and returns each package as one flat row.

- 🔍 **Two modes:** lookup fetches exact package names you already know, search finds packages by keyword across the full PyPI index.
- 📦 **Bulk ready:** collect up to 1,000,000 packages per run, with one row per package.
- 🧹 **Clean output:** every package returns the same flat schema, so you can export to CSV, JSON, Excel, or XML without cleanup.
- ⚡ **No API key:** reads the public PyPI index directly, no registration or rate-limit headaches.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with PyPI data

**📈 Track package popularity.**

A data analyst runs the Actor in search mode for a keyword like 'django' to collect all matching packages and their download counts, then charts which extensions are gaining traction.

**🔒 Audit dependencies.**

A developer feeds a list of package names from requirements.txt into lookup mode and gets the latest version, license, and author for each, flagging any that are outdated or abandoned.

**🕵️ Spot typosquatting.**

A security researcher searches for a popular package name and reviews the results for lookalike names that might be malicious, using the metadata to decide which to investigate.

**📊 Build a package dataset.**

An analyst runs the Actor in search mode for several keywords, exports the results to CSV, and joins them with other data to analyze the Python ecosystem.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API key** | Reads the public PyPI index directly, no registration or OAuth. |
| **Bulk lookups** | Fetch thousands of exact package names in one run. |
| **Keyword discovery** | Search by keyword and get packages ranked by relevance. |
| **Flat schema** | Every package returns the same fields, ready for export. |

### How it compares

No other Store actor targets PyPI the same way, so the honest comparison is with the alternatives teams actually weigh.

| | PyPI Packages Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When PyPI changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor from exact package names or a keyword query, and set a maximum number of packages to collect per run. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "maxItems": 10,
  "mode": "lookup",
  "names": "requests\nnumpy\npandas\nflask\nfastapi\ndjango\npydantic\nblack",
  "query": "django"
}
```

A larger pull:

```json
{
  "maxItems": 200,
  "mode": "lookup",
  "names": "requests\nnumpy\npandas\nflask\nfastapi\ndjango\npydantic\nblack",
  "query": "django"
}
```

### Pricing

Pay-per-result: **$0.016 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $1.60 |
| 1,000 results | $16.00 |
| 10,000 results | $160.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [PyPI Packages Scraper](https://apify.com/parseforge/pypi-packages-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to PyPI through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/pypi-packages-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that you selected the correct mode. In lookup mode, make sure each package name is on its own line and spelled correctly. In search mode, try a broader keyword.

**Why did the run stop before reaching my maximum packages?**

The Actor stops when there are no more matching packages. If you are in search mode, try a more common keyword. If you are in lookup mode, verify that all names exist on PyPI.

**Can I get more than 1,000,000 packages?**

The maximum is 1,000,000 per run. If you need more, split your list of names or keywords into multiple runs and merge the datasets.

**Why are some fields empty in the output?**

Not all packages on PyPI provide every metadata field. If a package does not have a license or author listed, that field will be empty in the row.

### FAQ

| Question | Answer |
|---|---|
| What is PyPI? | PyPI, the Python Package Index, is the official repository of third-party software for the Python programming language. It hosts over 500,000 packages that developers can install with pip. |
| Do I need an API key or login? | No. This Actor reads the public PyPI index directly, so you do not need to register an application or manage credentials. |
| What is the difference between lookup and search mode? | Lookup mode fetches exact package names you already know, one per line. Search mode takes a keyword and returns matching packages ranked by exact match, then prefix match, then substring match, then most recently active. |
| How many packages can I collect in one run? | You can set the maximum packages field to any number from 1 to 1,000,000. The Actor stops when it reaches that limit or when there are no more matches. |
| What fields are returned for each package? | Each package returns a flat row with metadata such as name, version, summary, author, license, and more. The exact fields are shown in the sample output below. |
| Can I export the results? | Yes. The Actor outputs a dataset that you can export to CSV, JSON, Excel, or XML from the Apify platform. |
| Is this the official PyPI API? | No, this Actor scrapes the public PyPI website and index directly. It does not use the official JSON API, so it is not subject to its rate limits. |
| Can I search for packages by author or license? | The current input schema only supports lookup by exact package name or search by keyword. To filter by author or license, you can export the results and filter them in your own tooling. |
| What if a package name does not exist? | In lookup mode, the Actor skips names that are not found and continues with the rest. In search mode, it returns an empty dataset if no packages match the keyword. |
| How fresh is the data? | The Actor reads live data from PyPI at the time of the run, so you get the latest published metadata for each package. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Python Software Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

How many packages to collect per run.

## `mode` (type: `string`):

Choose 'lookup' to fetch exact package names you already know, or 'search' to find packages by keyword.

## `names` (type: `string`):

Lookup mode only. Python package names, one per line, comma- or semicolon-separated.

## `query` (type: `string`):

Search mode only. Returns matching packages ranked by exact match, then prefix match, then substring match, then most-recently-active. Example: 'django' returns django itself + django-cms, django-rest-framework, etc.

## Actor input object example

```json
{
  "maxItems": 10,
  "mode": "lookup",
  "names": "requests\nnumpy\npandas\nflask\nfastapi\ndjango\npydantic\nblack",
  "query": "django"
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "mode": "lookup",
    "names": `requests
numpy
pandas
flask
fastapi
django
pydantic
black`,
    "query": "django"
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/pypi-packages-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "mode": "lookup",
    "names": """requests
numpy
pandas
flask
fastapi
django
pydantic
black""",
    "query": "django",
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/pypi-packages-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "mode": "lookup",
  "names": "requests\\nnumpy\\npandas\\nflask\\nfastapi\\ndjango\\npydantic\\nblack",
  "query": "django"
}' |
apify call parseforge/pypi-packages-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/pypi-packages-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/55bzPDP7pKcaSAMh2/builds/Gd09lG5AdqsatiSL8/openapi.json