# Similarweb Scraper (`morph_coder/similarweb-scraper`) Actor

Extract website traffic data, traffic source breakdowns, monthly visits, similar sites, WHOIS info, and on-page keyword density for any domain.

- **URL**: https://apify.com/morph\_coder/similarweb-scraper.md
- **Developed by:** [Morph Coder](https://apify.com/morph_coder) (community)
- **Categories:** SEO tools, Lead generation, E-commerce
- **Stats:** 5 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.00 / 1,000 result items

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## SimilarWeb Traffic Scraper

**Actor ID:** `morph_coder/similarweb-scraper`

SimilarWeb Traffic Scraper is a powerful, production-ready Apify Actor designed to extract website traffic intelligence, competitor data, domain WHOIS records, and on-page SEO keyword density in bulk.

This Actor is highly optimized to avoid detection using rotated residential proxies and custom TLS/HTTP2 fingerprints.

> This Actor is **not affiliated** with SimilarWeb, SimilarSites, or AITDK. Users must comply with applicable terms of service and local laws.

***

### ❶ Key Features

- 🌍 **Comprehensive Website Analytics (SimilarWeb Mode):** Extract global rank, country rank, category rank, total monthly visits, 3-month historical visits, and traffic source breakdowns (direct, search, referrals, social, mail, display ads).
- 🖼️ **Asset Reconstruction:** Reconstruct website icons, desktop previews, and mobile previews directly from metadata, eliminating the need for heavy browser instances.
- 🚦 **Geographic Breakdown:** Get the top 5 traffic-driving countries with percentage shares resolved to full country names.
- 🤝 **Similar Sites Discovery (Similar Sites Mode):** Retrieve up to 20 similar competitor websites from Similarsites.com, including site descriptions and total visits.
- 🔍 **SEO & Domain Intelligence (AITDK Mode):** Perform structured domain WHOIS lookups (via RDAP) and homepage n-gram keyword density analysis (1-to-5 word phrases) with multi-language stop-word filtering.
- ⚡ **Concurrency & Rotated Proxies:** Supports bulk lists processed in parallel. Utilizes unique proxy sessions per request to ensure maximum bypass success rates.

***

### ❷ Scraper Modes

The Actor operates in three distinct search modes selected via the `searchType` input:

1. **`similarweb` (Traffic Data - Default):** Retrieves core website metrics, rank history, monthly visits, and traffic sources.
2. **`similar_sites` (Competitors):** Retrieves a list of related/alternative websites with similarity scores.
3. **`aitdk` (SEO & WHOIS):** Returns WHOIS registry details (creation date, expiration date, registrar, nameservers) and visible page keyword density.

***

### ❸ Input Parameters

The Actor accepts the following JSON properties:

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `urls` | `array` | No | `["google.com"]` | List of webpage URLs, domains, or websites to analyze. |
| `domains` | `array` | No | — | Alternative input list of domains (merged with `urls`). |
| `websites` | `array` | No | — | Alternative input list of website domains (merged with `urls`). |
| `searchType` | `string` | No | `"similarweb"` | Mode of scraping: `"similarweb"`, `"similar_sites"`, or `"aitdk"`. |
| `include_base_data` | `boolean` | No | `true` | When enabled, returns base SimilarWeb traffic and ranking data. |
| `include_similar_sites` | `boolean` | No | `false` | When enabled, includes SimilarSites competitor lists. |
| `include_indepth_data` | `boolean` | No | `false` | Includes additional historical trends and company details if available. |
| `output_mode` | `string` | No | `"aggregated"` | Format for output items: `"aggregated"` or `"individual"`. |
| `proxyConfiguration` | `object` | No | — | Proxy configuration. Residential proxy rotation is highly recommended. |

#### Sample Input JSON

```json
{
    "urls": ["sportsdigest.in"],
    "searchType": "similarweb",
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"]
    }
}
```

***

### ❹ Output Examples

#### SimilarWeb Mode (Traffic & Ranks)

```json
{
  "url": "https://similarweb.com/website/sportsdigest.in",
  "name": "sportsdigest.in",
  "title": "Football News: Live Scores, Results & Match Updates",
  "description": "Football news, live scores, match results...",
  "category": "sports/sports",
  "icon": "https://site-images.similarcdn.com/image?url=sportsdigest.in&t=2&s=1&h=47c9aefee98a6b...",
  "previewDesktop": "https://site-images.similarcdn.com/image?url=sportsdigest.in&t=1&s=1&h=47c9aefee98a6b...",
  "previewMobile": "https://site-images.similarcdn.com/image?url=sportsdigest.in&t=4&s=1&h=47c9aefee98a6b...",
  "globalRank": { "rank": 441621 },
  "countryRank": { "countryCode": "IN", "rank": 36173 },
  "categoryRank": { "category": "Sports/Sports", "rank": 135 },
  "engagements": {
    "visits": 49385,
    "timeOnSite": 371.7,
    "pagePerVisit": 9.62,
    "bounceRate": 0.33
  },
  "trafficSources": {
    "direct": 0.58,
    "referrals": 0.08,
    "mail": 0.01,
    "social": 0.03,
    "search": 0.29,
    "paidReferrals": 0
  },
  "topKeywords": [
    { "name": "hardik pandya trade to csk", "estimatedValue": 970, "volume": 980 }
  ],
  "topCountries": [
    { "countryCode": "IN", "countryName": "India", "countryUrlCode": "india", "visitsShare": 0.85 }
  ],
  "estimatedMonthlyVisits": {
    "2026-03-01": 61051,
    "2026-04-01": 54721,
    "2026-05-01": 49385
  },
  "scrapedAt": "2026-06-21T13:38:31.361Z",
  "snapshotDate": "2026-05-01T00:00:00+00:00"
}
```

#### Similar Sites Mode (Competitors)

```json
{
  "SiteName": "sportsdigest.in",
  "SimilarSites": [
    {
      "Site": "iplt20.com",
      "TotalVisits": 1353414.01,
      "CategoryRank": null,
      "TopCountryRank": 5418,
      "Description": "The official IPL website with live scores, match updates..."
    }
  ]
}
```

#### AITDK Mode (WHOIS + Keyword Density)

```json
{
  "SiteName": "sportsdigest.in",
  "Whois": {
    "registrationDate": "2021-03-07T03:46:41.533Z",
    "expirationDate": "2028-03-07T03:46:41.533Z",
    "registrar": "GoDaddy",
    "registrantCountry": "IN",
    "nameservers": ["samara.ns.cloudflare.com", "porter.ns.cloudflare.com"],
    "status": ["client update prohibited", "client delete prohibited"]
  },
  "KeywordDensity": {
    "1word": [
      { "keyword": "sports", "count": 17, "total": 803, "density": 2.12 }
    ],
    "2words": [
      { "keyword": "world cup", "count": 10, "total": 1013, "density": 0.99 }
    ]
  }
}
```

***

### ❺ Pricing & Monetization

This Actor is configured with custom pay-per-event pricing on the Apify Store:

- **Custom Event (`processed-item`):** Charged once per successfully scraped domain.
- **Compute & Data Transfer:** Billed in standard platform usage rates.
- **Apify Proxy:** Residential proxy data transmission usage is billed separately per gigabyte.

***

# Actor input Schema

## `urls` (type: `array`):

List of domains, websites, or webpage URLs to analyze (e.g. google.com, https://facebook.com).

## `domains` (type: `array`):

Alternative array for domain list. Will be merged with URLs.

## `websites` (type: `array`):

Alternative array for website list. Will be merged with URLs.

## `searchType` (type: `string`):

Choose between SimilarWeb Traffic, Similar Sites (competitors), or AITDK (WHOIS + Keyword Density).

## `include_base_data` (type: `boolean`):

Include standard traffic statistics, global and category ranks.

## `include_similar_sites` (type: `boolean`):

Include competitor sites from SimilarSites.com.

## `include_indepth_data` (type: `boolean`):

Include additional in-depth details such as company information and historical traffic trends.

## `output_mode` (type: `string`):

Aggregated format (combine data in single item) or Individual (one item per record).

## `proxyConfiguration` (type: `object`):

Proxy settings. Residential proxies are highly recommended to prevent blocks.

## Actor input object example

```json
{
  "urls": [
    "google.com"
  ],
  "searchType": "similarweb",
  "include_base_data": true,
  "include_similar_sites": false,
  "include_indepth_data": false,
  "output_mode": "aggregated"
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset items from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "google.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("morph_coder/similarweb-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["google.com"] }

# Run the Actor and wait for it to finish
run = client.actor("morph_coder/similarweb-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "google.com"
  ]
}' |
apify call morph_coder/similarweb-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,morph_coder/similarweb-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/abb2fQDhfSAUvHbMT/builds/ChVhNOxLvBdclPU2Y/openapi.json