# RapidAPI Scraper (`yourapiservice/rapidapi-scraper`) Actor

Scrape and download API listings by category and collection from RapidAPI. Extract API names, descriptions, and other metadata using category-specific search queries. Export scraped data, run the scraper via API, schedule and monitor runs, and integrate with other tools seamlessly.

- **URL**: https://apify.com/yourapiservice/rapidapi-scraper.md
- **Developed by:** [Your API Service](https://apify.com/yourapiservice) (community)
- **Categories:** Developer tools, Integrations
- **Stats:** 101 total users, 1 monthly users, 100.0% runs succeeded, 8 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### What does RapidAPI Scraper do?

This RapidAPI Scraper allows you to scrape the list of APIs from RapidAPI, including each category and supports scraping
without category selection. It scrapes all categories first, then scrapes the data. The data is provided in structured
formats such as Excel, XML, HTML, JSON, and CSV, which you can use in your own reports, spreadsheets, and applications.
RapidAPI Scraper allows you to scrape:

- Results from a RapidAPI category.
- Numbers of total builds, runs, and users.
- Detailed information about each API, including title, description, categories, and pricing model.

### Why use RapidAPI Scraper?

Because RapidAPI does not provide a comprehensive API list. With RapidAPI Scraper, you can build your own products and
applications on top of the RapidAPI platform.

### How to use scraped RapidAPI data

- Building new applications that use the API list.
- Example: https://yourapiservice.com, etc.
- Creating a comprehensive API directory for developers.

### Input parameters

If this actor is run on the Apify platform, our simple interface will help you configure all the necessary and optional
parameters of this scraper before running it. This scraper recognizes the following input parameters:

- **includeCategories** - If set to true, the scraper will scrape the categories first and then scrape the data. If set
  to false, the scraper will not scrape with category (You can use Collections instead).
- **categoryNames** - Scrapes RapidAPI for specified categories. You can duplicate this parameter to scrape several
  categories simultaneously.
- **maxItemsPerCategory** - Sets the maximum limit for the amount of scraped results per category/collection.
- **includeCollections** - If set to true, the scraper will scrape the collections first and then scrape the data. If
  set to false, the scraper will not scrape with collection (You can use Categories instead).
- **collectionNames** - Scrapes RapidAPI for specified collections. You can duplicate this parameter to scrape several
  collections simultaneously.
- **sortBy** - Sorts the search query results by the specified parameter.
- **scrapeDetails** - Enables scraping of detailed information about the APIs, such as endpoints, parameters, headers,
  and more. (May take a long time to scrape all details, especially with proxies.)
- **scrapeEndpoints** - Enables scraping of API Endpoints, Playground URLs, Methods, and Descriptions. (Not recommended
  for large collections, as it may take a long time.)
- **scrapeAllDetails** - Enables scraping of all details including Parameters, Headers, etc. (Not recommended for large
  collections, as it may take a long time.)
- **proxyConfiguration** - Allows you to set a proxy configuration for the scraper. You can use Apify Proxy, your own
  proxy, or no proxy at all. If you want to use Apify Proxy, set this parameter to `APIFY_PROXY_CONFIGURATION`. If you
  want to use your own proxy, set this parameter to `YOUR_PROXY_CONFIGURATION`. If you don't want to use a proxy, set
  this parameter to `NO_PROXY`.

Example:

```json
{
    "includeCategories": true,
    "categoryNames": [
        "Finance",
        "Data"
    ],
    "includeCollections": true,
    "collectionNames": [
        "recommended-apis",
        "popular-apis",
        "list-of-free-apis"
    ],
    "maxItemsPerCategory": 100,
    "sortBy": "ByTrending",
    "scrapeDetails": false,
    "scrapeEndpoints": false,
    "scrapeAllDetails": false
}
```

### Output

The output from RapidAPI Scraper is stored in a dataset. Each item contains information about an API. Here's an example
of some of the output you would get if you used the input parameters above to scrape the categories "Finance":

```json
{
    "source": "RapidAPI",
    "title": "Seeking Alpha",
    "name": "seeking-alpha",
    "username": "apidojo",
    "description": "Query for news, market moving, price quotes, chart, indices, analysis from investors and experts, etc...",
    "categories": [
        "Finance"
    ],
    "pictureUrl": "https://rapidapi-prod-apis.s3.amazonaws.com/be878f94-ea6d-4a87-aae2-ea43c69f155f.jpg",
    "stats": {
        "totalBuilds": 0,
        "totalRuns": 0,
        "totalUsers": 0,
        "totalUsers7Days": 0,
        "totalUsers30Days": 0,
        "totalUsers90Days": 0,
        "lastRunStartedAt": "2024-06-27T05:30:22.633Z",
        "totalMetamorphs": 0,
        "publicActorRunStats30Days": {
            "ABORTED": 0,
            "FAILED": 0,
            "SUCCEEDED": 0,
            "TIMED_OUT": 0,
            "TOTAL": 0
        }
    },
    "notice": "NONE",
    "userPictureUrl": "",
    "userFullName": "Api Dojo",
    "currentPricingInfo": {
        "pricingModel": "FREEMIUM"
    },
    "objectID": "api_ba97c871-0271-454d-9e7e-983860c54f65",
    "updatedAt": "2024-06-27T05:30:22.633Z",
    "score": {
        "popularityScore": 9.9,
        "avgLatency": 2269,
        "avgServiceLevel": 100,
        "avgSuccessRate": 100
    }
}
```

### Example Output for Scrape Details

If you set the `scrapeDetails` and `scrapeEndpoints`  parameters to true, the output will include detailed information
about each API, such as endpoints, playgroundUrls, methods and titles. Here's an example of what the output might look
like:

```json
{
    "endpoints": [
        {
            "playgroundUrl": "https://rapidapi.com/apidojo/api/realty-in-us/playground/apiendpoint_cd6f72a3-8077-47f1-b2aa-ad74b4b383ee",
            "title": "locations/v2/auto-complete",
            "method": "GET",
            "url": "https://realty-in-us.p.rapidapi.com/locations/v2/auto-complete"
        }
    ]
}
```

### How much will scraping RapidAPI cost you?

When it comes to scraping, it can be challenging to estimate the resources needed to extract data as use cases may vary
significantly. That's why the best course of action is to run a test scrape with a small sample of input data and
limited output. You’ll get your price per scrape, which you’ll then multiply by the number of scrapes you intend to do.

### Integrations and RapidAPI Scraper

Last but not least, RapidAPI Scraper can be connected with almost any cloud service or web app thanks to integrations on
the Apify platform. You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more.
Or you can use webhooks to carry out an action whenever an event occurs, e.g., get a notification whenever Instagram API
Scraper successfully finishes a run.

### Using RapidAPI Scraper with the Apify API

The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints
that enable you to manage, schedule, and run Apify actors. The API also lets you access any datasets, monitor actor
performance, fetch results, create and update versions, and more.

To access the API using Node.js, use the apify-client NPM package. To access the API using Python, use the apify-client
PyPI package.

Check out the Apify API reference docs for full details or click on the API tab for code examples.

### Personal data

You should be aware that your results might contain personal data. Personal data is protected by GDPR in the European
Union and other laws and regulations around the world. You should not scrape personal data unless you have a legitimate
reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. You can read the basics of
ethical web scraping in our blog post on the legality of web scraping.

### Your feedback

We’re always working on improving the performance of our Actors. So if you’ve got any technical feedback for RapidAPI
Scraper or simply found a bug, please create an issue on the Actor’s Issues tab in Apify Console.

# Actor input Schema

## `includeCategories` (type: `boolean`):

Include categories in the search results

## `categoryNames` (type: `array`):

List of Names to scrape. If empty, all categories will be scraped. (NOTE: It will takes a long time!)

## `sortBy` (type: `string`):

Sort by for the search results Default: Relevance

## `maxItemsPerCategory` (type: `integer`):

Maximum number of items to scrape. Currently max 1000 items per category.

## `includeCollections` (type: `boolean`):

Include collections in the search results

## `collectionNames` (type: `array`):

List of Names to scrape. If empty, all Collections will be scraped. (NOTE: It will takes a long time!)

## `scrapeDetails` (type: `boolean`):

It enables the scraping of detailed information about the APIs, such as endpoints, parameters, headers, and more. (It may take a long time to scrape all details also Proxies!.)

## `scrapeEndpoints` (type: `boolean`):

Enable scraping of API Endpoints, Playground Urls, Methods and Descriptions. (Not recommended for large collections, it may take a long time to scrape all details also Proxies!.)

## `scrapeAllDetails` (type: `boolean`):

Enable scraping of all details including Parameters, Headers, etc. (Not recommended for large collections, it may take a long time to scrape all details also Proxies!.)

## `proxy` (type: `object`):

Select proxies to be used.

## Actor input object example

```json
{
  "includeCategories": true,
  "categoryNames": [
    "Artificial Intelligence/Machine Learning",
    "Data"
  ],
  "sortBy": "ByRelevance",
  "maxItemsPerCategory": 100,
  "includeCollections": false,
  "collectionNames": [
    "recommended-apis",
    "popular-apis"
  ],
  "scrapeDetails": false,
  "scrapeEndpoints": true,
  "scrapeAllDetails": false,
  "proxy": {
    "useApifyProxy": true
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "includeCategories": true,
    "categoryNames": [
        "Artificial Intelligence/Machine Learning",
        "Data"
    ],
    "sortBy": "ByRelevance",
    "includeCollections": false,
    "collectionNames": [
        "recommended-apis",
        "popular-apis"
    ],
    "proxy": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("yourapiservice/rapidapi-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "includeCategories": True,
    "categoryNames": [
        "Artificial Intelligence/Machine Learning",
        "Data",
    ],
    "sortBy": "ByRelevance",
    "includeCollections": False,
    "collectionNames": [
        "recommended-apis",
        "popular-apis",
    ],
    "proxy": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("yourapiservice/rapidapi-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "includeCategories": true,
  "categoryNames": [
    "Artificial Intelligence/Machine Learning",
    "Data"
  ],
  "sortBy": "ByRelevance",
  "includeCollections": false,
  "collectionNames": [
    "recommended-apis",
    "popular-apis"
  ],
  "proxy": {
    "useApifyProxy": true
  }
}' |
apify call yourapiservice/rapidapi-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,yourapiservice/rapidapi-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OuGwCW6c1ZMftAVn6/builds/RXIX0eEp3TkoQVFcH/openapi.json