# SEMrush Scraper - Domain Authority & SEO Metrics (`devnaz/semrush-scraper`) Actor

Extract 20+ SEO metrics from any domain: authority score, monthly traffic, bounce rate, backlinks, organic keywords, competitors & more. No SEMrush subscription needed. Perfect for competitor analysis, SEO monitoring & market research.
Fast, reliable data extraction in 40 seconds.

- **URL**: https://apify.com/devnaz/semrush-scraper.md
- **Developed by:** [DevnaZ](https://apify.com/devnaz) (community)
- **Categories:** SEO tools, Automation, Developer tools
- **Stats:** 269 total users, 7 monthly users, 100.0% runs succeeded, 6 bookmarks
- **User rating**: 1.00 out of 5 stars

## Pricing

$25.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## SEMrush Scraper - Domain Authority, Traffic & SEO Data

Fast, reliable extraction of **domain authority, traffic data, rankings, and competitor insights** from SEMrush. **No SEMrush account required**. Perfect for SEO tracking, competitor monitoring, and market research.

### Features

Extract **20+ SEO metrics** from any domain in under 20 seconds:

#### Traffic Metrics

- **Monthly visits** - Total website traffic
- **Bounce rate** - Visitor engagement percentage
- **Pages per visit** - Average pages viewed per session
- **Visit duration** - Time spent on site (in seconds)
- **Traffic change** - Month-over-month growth/decline percentage

#### Traffic Sources

- **Organic search visits** - Traffic from search engines
- **Paid search visits** - Traffic from paid ads
- **Direct traffic %** - Percentage of direct visitors
- **Desktop traffic %** - Desktop vs mobile breakdown
- **Mobile traffic %** - Mobile device traffic share

#### Rankings & Authority

- **Domain authority** - SEMrush authority score (0-100)
- **Global rank** - Worldwide ranking
- **Country rank** - Ranking in primary country
- **Industry category** - Business category classification

#### Backlinks & SEO

- **Total backlinks** - Number of incoming links
- **Referring domains** - Unique domains linking to site

#### Competitive Intelligence

- **Top 5 competitors** - Similar websites by audience
- **Top 5 organic keywords** - Best performing search terms with position & volume
- **Top 5 countries** - Geographic traffic distribution with percentages

### Input

```json
{
  "domains": [
    "amazon.com",
    "apple.com",
    "make.com"
  ],
  "maxConcurrency": 3,
  "navigationTimeout": 60,
  "pageLoadDelay": 5000,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["DATACENTER"]
  }
}
```

#### Parameters

- **`domains`** (required) - Array of domain names to analyze (e.g., `["example.com"]`)
- **`maxConcurrency`** (optional) - Number of pages to scrape simultaneously (default: 3, max: 10)
- **`navigationTimeout`** (optional) - Maximum time to wait for page navigation in seconds (default: 60, max: 300). Increase if you get timeout errors.
- **`pageLoadDelay`** (optional) - Additional wait time for dynamic content in milliseconds (default: 5000, max: 15000)
- **`proxyConfiguration`** (optional) - Proxy settings (free Apify Datacenter proxies included by default)

### Output

```json
{
  "domain": "make.com",
  "scrapedAt": "2025-11-10T20:43:27.750Z",
  "domain_authority": 55,
  "monthly_visits": 8300000,
  "bounce_rate": 37.22,
  "pages_per_visit": 6.68,
  "avg_visit_duration_seconds": 770,
  "traffic_change_percent": -2.91,
  "organic_search_visits": 657960,
  "paid_search_visits": 123320,
  "direct_visits_percent": 79.72,
  "desktop_traffic_percent": 76.92,
  "mobile_traffic_percent": 23.08,
  "global_rank": 6757,
  "country_rank": 7264,
  "country": "United States",
  "industry_category": "Computer Software and Development",
  "total_backlinks": 2370000,
  "referring_domains": 26590,
  "competitors": [
    {"domain": "n8n.io"},
    {"domain": "zapier.com"},
    {"domain": "pipedream.com"}
  ],
  "organic_keywords": [
    {
      "keyword": "make",
      "position": 1,
      "search_volume": 74000
    }
  ],
  "top_countries": [
    {
      "country": "United States",
      "traffic_percent": 16.96,
      "visits": 1410000
    }
  ]
}
```

### Use Cases

#### 1. Competitor Analysis

Monitor your competitors' traffic, keywords, and backlink profiles to identify opportunities.

#### 2. SEO Monitoring

Track domain authority, organic traffic, and keyword rankings over time.

#### 3. Market Research

Discover industry leaders, traffic patterns, and top-performing keywords in your niche.

#### 4. Lead Generation

Identify high-traffic websites in your target industry for partnership opportunities.

#### 5. Portfolio Tracking

Monitor multiple websites' performance metrics from a single dashboard.

### Performance

- **Speed**: ~35-40 seconds per domain
- **Success rate**: 95%+ on most domains
- **Concurrency**: Process up to 10 domains simultaneously
- **Data freshness**: SEMrush updates monthly (scrapes latest available data)

### Notes

- **No SEMrush account required** - Scrapes publicly available data
- **Free Apify Datacenter proxies included** - Works great for most use cases
- **Rate limiting**: Respect SEMrush's infrastructure (max 10 concurrent requests)
- Some metrics may be unavailable for low-traffic domains
- Data is as fresh as SEMrush's latest update (typically monthly)

### Technical Details

- **Runtime**: Node.js with Playwright
- **Browser**: Chromium (headless mode)
- **Rendering**: Full JavaScript execution for dynamic content
- **Extraction**: Regex pattern matching on rendered page text
- **Proxy support**: Free Apify Datacenter proxies (default) or custom proxies

### Support

Having issues? Check these common solutions:

1. **Timeout errors**: Increase the `navigationTimeout` parameter (e.g., 120 seconds) or reduce `maxConcurrency`
2. **Slow loading sites**: Increase the `pageLoadDelay` parameter (e.g., 8000ms) to allow more time for dynamic content
3. **Missing data**: Some domains may have limited public data on SEMrush
4. **Proxy blocks**: Try increasing `navigationTimeout` or reducing `maxConcurrency`. Datacenter proxies work well for SEMrush.
5. **Rate limiting**: Reduce `maxConcurrency` (e.g., set to 1 or 2) to slow down request rate

For bugs or feature requests, please contact support.

### Example Integrations

#### Zapier/Make.com

Trigger analysis when competitors launch new pages or campaigns.

#### Google Sheets

Export data automatically to spreadsheets for reporting.

#### Slack/Discord

Get alerts when competitors' traffic changes significantly.

#### Custom Dashboards

Feed data into Tableau, Power BI, or custom analytics platforms.

***

No SEMrush subscription required. Get started analyzing domains today!

# Actor input Schema

## `domains` (type: `array`):

List of domains to scrape SEMrush data for (e.g., amazon.com, apple.com)

## `maxConcurrency` (type: `integer`):

Maximum number of pages to scrape concurrently

## `navigationTimeout` (type: `integer`):

Maximum time to wait for page navigation (default: 60s). Increase if timeout errors occur.

## `pageLoadDelay` (type: `integer`):

Additional wait time for dynamic content to load (default: 5000ms)

## `proxyConfiguration` (type: `object`):

Proxy settings for scraping. Free Apify Datacenter proxies included.

## Actor input object example

```json
{
  "domains": [
    "amazon.com",
    "apple.com",
    "microsoft.com"
  ],
  "maxConcurrency": 3,
  "navigationTimeout": 60,
  "pageLoadDelay": 5000,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "make.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("devnaz/semrush-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["make.com"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("devnaz/semrush-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "make.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call devnaz/semrush-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,devnaz/semrush-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bL147gFarcajpfr5W/builds/gLdOvHITfMQ6OdyiL/openapi.json