# AI Valley Scraper (`bitdoze/ai-valley-scraper`) Actor

Scrape AI tool data (names, categories, descriptions, images, URLs) from the AI Valley directory of 1,700+ AI tools.

- **URL**: https://apify.com/bitdoze/ai-valley-scraper.md
- **Developed by:** [Dragos Mihai Balota](https://apify.com/bitdoze) (community)
- **Categories:** AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### What does AI Valley Scraper do?

**AI Valley Scraper** extracts structured data from [AI Valley](https://aivalley.ai/), a curated directory of **1,700+ AI tools** across dozens of categories. It pulls **tool names, categories, descriptions, images, and URLs** for every tool listed, and returns the results as a clean, structured dataset.

AI Valley is one of the most comprehensive AI tool directories on the web, covering categories like AI assistants, code tools, design, data analysis, chatbots, content creation, and more. The site uses WordPress with server-rendered HTML, so this Actor uses **Cheerio** (HTTP-based parsing) for maximum speed and low run cost.

You get all the benefits of the Apify platform: API access, scheduling, proxy rotation, monitoring, and one-click exports to JSON, CSV, Excel, and more.

### Why use AI Valley Scraper?

- **Market intelligence**: Track the AI tools landscape - new launches, categories, and trends.
- **Competitor research**: Monitor which AI tools are gaining traction in your category.
- **Lead generation**: Build lists of AI tool companies for outreach, partnerships, or investment research.
- **Content creation**: Curate AI tool roundups, newsletters, or comparison articles.
- **SEO & directories**: Build your own AI tool directory with structured, up-to-date data.

### How to use AI Valley Scraper

1. Click **Try for free** or **Try Actor** on the Apify Store.
2. In the **Input** tab, optionally add category URLs or category slugs to filter (e.g., `design`, `code-assistant`). Leave empty to scrape all tools.
3. Set **Max items total** to limit results, or leave at 0 for all tools.
4. Click **Start** and wait for the run to finish.
5. Open the **Output** tab to view, filter, and export your dataset.

### Input

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `startUrls` | Array of URLs | No | AI Valley category or listing URLs to start from. Defaults to all AI tools. |
| `categories` | Array of strings | No | Category slugs to scrape (e.g., `design`, `chatbots`). |
| `maxItems` | Integer | No | Maximum total tools to collect. `0` = no limit. |
| `maxRequestsPerCrawl` | Integer | No | Maximum pages to crawl. Default: 50. |
| `scrapeDetails` | Boolean | No | Visit each tool's detail page for pricing and full description (slower). |

#### Example input

```json
{
    "startUrls": [
        { "url": "https://aivalley.ai/category/aitools/" }
    ],
    "maxItems": 0,
    "maxRequestsPerCrawl": 50
}
```

### Output

Each result is an AI tool object. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

```json
[
    {
        "toolName": "Tables",
        "category": "data-analyst",
        "toolUrl": "https://aivalley.ai/tables/",
        "imageUrl": "https://aivalley.ai/wp-content/uploads/2024/12/Screenshot-2024-12-07.png",
        "excerpt": "Instantly transform unstructured data into actionable tables",
        "pricingModel": null,
        "scrapedAt": "2026-07-07T14:07:00.000Z"
    }
]
```

#### Output fields

| Field | Description |
|-------|-------------|
| `toolName` | Name of the AI tool. |
| `category` | AI Valley category slug (e.g., `design`, `code-assistant`). |
| `toolUrl` | AI Valley page URL for the tool. |
| `imageUrl` | Tool thumbnail/screenshot image URL. |
| `excerpt` | Short description from the listing. |
| `pricingModel` | Pricing model if available (`free`, `freemium`, `paid`). |
| `price` | Numeric price if available. |
| `currency` | Currency code if price is available. |
| `scrapedAt` | ISO timestamp of when the item was collected. |

### Pricing / Cost estimation

This Actor uses lightweight HTTP parsing (no browser), so runs are very inexpensive. Scraping all 1,700+ tools across 18+ pages completes in under a minute. You only pay for the Apify platform usage, and you can try it for free with the platform's trial credits.

### Tips

- **Filter by category**: Pass category slugs in the `categories` field to only scrape specific tool types.
- **Limit results**: Use `maxItems` to cap collection and keep costs predictable.
- **Bulk scraping**: The default settings scrape all AI tools. Increase `maxRequestsPerCrawl` to scrape more pages.

### FAQ, disclaimers, and support

- **Is this legal?** This Actor only reads publicly available directory pages. You are responsible for complying with AI Valley's Terms of Service and applicable laws when using the data.
- **Does it need login or API keys?** No. It scrapes public HTML.
- **How often is the data updated?** AI Valley adds new tools regularly. Schedule periodic runs to keep your dataset fresh.
- **Need a custom solution?** Open an issue on the Actor's **Issues** tab for feedback or custom requests.

### Getting started (local development)

```bash
npm install
npm test        # runs unit + integration tests
npm run build   # compiles TypeScript
apify run       # run locally with simulated Apify environment
```

### Deploy to Apify

```bash
apify login    # provide your Apify API token
apify push     # build & deploy the Actor to the Apify platform
```

# Actor input Schema

## `startUrls` (type: `array`):

AI Valley category or listing URLs to start scraping from. Leave empty to scrape all AI tools.

## `categories` (type: `array`):

AI Valley category slugs to scrape (e.g., 'design', 'code-assistant', 'chatbots'). Leave empty to scrape all tools.

## `maxItems` (type: `integer`):

Maximum total tools to collect across all pages. 0 = no limit.

## `maxRequestsPerCrawl` (type: `integer`):

Maximum number of pages the crawler may load.

## `scrapeDetails` (type: `boolean`):

If true, visit each tool's detail page for pricing and full description (slower).

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://aivalley.ai/category/aitools/"
    }
  ],
  "categories": [],
  "maxItems": 0,
  "maxRequestsPerCrawl": 50,
  "scrapeDetails": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://aivalley.ai/category/aitools/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("bitdoze/ai-valley-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://aivalley.ai/category/aitools/" }] }

# Run the Actor and wait for it to finish
run = client.actor("bitdoze/ai-valley-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://aivalley.ai/category/aitools/"
    }
  ]
}' |
apify call bitdoze/ai-valley-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bitdoze/ai-valley-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/JApzavEcYRY3uifhg/builds/TlyjFp1EA0yvU5Mg6/openapi.json