# DataDome Web Scraper (`ecomscrape/datadome-web-scraper`) Actor

DataDome Web Scraper extracts data from DataDome-protected websites. You can customize parameters such as proxies, timeouts, and JavaScript execution, making it ideal for reports, spreadsheets, and applications.

- **URL**: https://apify.com/ecomscrape/datadome-web-scraper.md
- **Developed by:** [ecomscrape](https://apify.com/ecomscrape) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 58 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$20.00/month + usage

To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period. You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#rental-actors

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Contact

If you encounter any issues or need to exchange information, please feel free to contact us through the following link:
[My profile](https://apify.com/ecomscrape)

## What does Datadome web Scraper do?

### Introduction

Datadome is one of the most sophisticated bot protection systems available today, designed to stop automated traffic while allowing genuine users to browse freely. With its ability to process 5 trillion+ signals per day and achieve 99.99% accuracy in distinguishing bots from humans, Datadome presents a significant challenge for legitimate data collection needs.

However, businesses often require access to publicly available data from Datadome-protected websites for market research, competitive analysis, and business intelligence. This data is legally accessible as long as scraping rates remain reasonable and don't impact website performance. Our Datadome Web Scraper addresses this need by providing a reliable solution to bypass Datadome protection while maintaining ethical data collection practices.

Whether you're analyzing marketplace data from platforms like Etsy with over 4 million sellers and 60 million products, conducting competitor research, or gathering market insights, this scraper enables seamless data extraction from Datadome-protected websites.

### Scraper Overview and Key Features

The Datadome Web Scraper is an advanced data extraction tool specifically engineered to overcome Datadome's sophisticated bot detection mechanisms. Unlike traditional scrapers that struggle with Datadome's AI-powered behavior analysis and CAPTCHA challenges, our solution employs multiple bypass techniques to ensure consistent data collection.

**Core Capabilities:**

- **Intelligent CAPTCHA Bypass**: Automatically handles Datadome CAPTCHA challenges without manual intervention
- **Residential Proxy Integration**: Uses high-quality residential proxies to mimic genuine user traffic patterns
- **Advanced Anti-Detection**: Implements browser fingerprinting evasion and behavioral mimicry
- **JavaScript Execution**: Full support for dynamic content rendering and custom script execution
- **Retry Logic**: Configurable retry mechanisms to handle temporary blocks or rate limiting
- **Geographic Targeting**: Proxy selection based on target market location for optimal success rates

**Target Users:**

- E-commerce businesses conducting competitive analysis
- Market researchers requiring pricing and product data
- Digital marketing agencies tracking competitor activities
- Data analysts studying marketplace trends
- Business intelligence professionals gathering market insights

The scraper is particularly effective for extracting data from e-commerce platforms, search results pages, and product listings where Datadome protection is commonly implemented.

### Input and Output Specifications

Example url: https://www.etsy.com/market/top\_sellers?ref=pagination\&page=3

Example url: https://www.pokemoncenter.com/

Example Screenshot of product information page:

![](https://i.ibb.co/C5NMLz9/Screenshot-from-2025-01-23-16-09-10.png)

#### Input Format

The scraper accepts JSON configuration with the following parameters:

```json
{
  "max_retries_per_url": 2, // Maximum waiting time when accessing the links you provided.
  "proxy": { // Add a proxy to ensure that during the data collection process, you are not detected as a bot.
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL" 
    ],
    "apifyProxyCountry": "SG" // You should choose an Country that coincides with the Country you want to collect data from
  },
  "urls": [ // Links to web pages.
    "https://www.etsy.com/market/top_sellers?ref=pagination&page=3",
    "https://www.pokemoncenter.com/"
  ],
  "js_script": "return 10 + 10 + 20", // JS script you want to run
  "js_timeout": 10,
  "retrieve_result_from_js_script": true, // Retrieve result from JS script
  "page_is_loaded_before_running_script": true, // Page is loaded before running script
  "execute_js_async": false, // Execute JS async
  "retrieve_html_from_url_after_loaded": true, // Retrieve page HTML from url after loaded
}
```

**Input Parameters Explained:**

- **max\_retries\_per\_url**: Controls the maximum number of retry attempts for each URL when encountering blocks or timeouts. Higher values increase success rates but extend execution time.
- **proxy.useApifyProxy**: Enables residential proxy rotation to avoid IP-based detection by Datadome's systems.
- **proxy.apifyProxyGroups**: Specifies proxy type - "RESIDENTIAL" proxies provide the highest success rate against Datadome.
- **proxy.apifyProxyCountry**: Geographic location of proxies should match your target market for optimal performance.
- **urls**: Array of target URLs to scrape. Each URL will be processed with the same configuration parameters.
- **js\_script**: Custom JavaScript code to execute on each page, useful for interacting with dynamic content or extracting specific data.
- **js\_timeout**: Maximum execution time for JavaScript scripts to prevent hanging operations.
- **retrieve\_result\_from\_js\_script**: When true, captures and returns the result of your custom JavaScript execution.
- **page\_is\_loaded\_before\_running\_script**: Ensures complete page loading before script execution for accurate data extraction.
- **execute\_js\_async**: Controls JavaScript execution mode - synchronous execution provides more predictable results.
- **retrieve\_html\_from\_url\_after\_loaded**: Captures the complete HTML content after all dynamic elements have loaded.

#### Output Format

You get the output from the Datadome web Scraper stored in a tab. The following is an example of the Information Fields collected after running the Actor.

```json
[ // List of product information
  {
    "url": "https://www.etsy.com/market/top_sellers?ref=pagination&page=3",
    "result_from_js_script": 40,
    "html": "<!DOCTYPE html>...</html>" // HTML from web page
  }, // ... Many other product details
] 
```

The scraper returns structured data containing:

**Primary Output Fields:**

1. **URL**: The exact URL that was scraped, useful for tracking and data organization
   - *Purpose*: Enables data source verification and batch processing tracking
   - *Example*: `"https://www.etsy.com/market/top_sellers?ref=pagination&page=3"`

2. **HTML**: Complete HTML content of the scraped page after JavaScript execution
   - *Purpose*: Contains all page data for parsing product information, pricing, descriptions, and metadata
   - *Usage*: Parse with HTML parsers to extract specific elements like product titles, prices, seller information, ratings, and availability
   - *Data richness*: Includes both static HTML and dynamically loaded content

3. **Result from JS Script**: Output from your custom JavaScript execution
   - *Purpose*: Provides processed data or specific calculations performed on the target page
   - *Example applications*: Product count calculations, price comparisons, availability checks, or custom data transformations
   - *Format*: Can return strings, numbers, objects, or arrays depending on your script logic

The HTML output typically contains rich marketplace data including product information, descriptions, listed dates, images, seller info, pricing, and customer reviews that can be parsed for comprehensive market analysis.

### Usage Guide

**Step 1: Configuration Setup**
Configure your input parameters based on your target website and data requirements. For Datadome-protected sites, always use residential proxies and match the proxy country to your target market.

**Step 2: URL Preparation**
Prepare your target URLs ensuring they represent the specific pages containing your desired data. For marketplace scraping, this might include product listings, search results, or seller pages.

**Step 3: JavaScript Customization**
Write custom JavaScript for data extraction or page interaction. This is particularly useful for extracting hidden JSON data embedded in pages rather than parsing HTML elements.

**Step 4: Execution and Monitoring**
Run the scraper and monitor retry attempts. The built-in retry logic handles temporary blocks, but persistent failures may require proxy rotation or rate limiting adjustments.

**Best Practices:**

- Use reasonable delays between requests to avoid triggering additional protection layers
- Implement data validation to ensure extracted information meets quality standards
- Maintain reasonable scraping rates to avoid impacting website performance
- Regularly rotate proxy locations and user agents for sustained access

**Common Issue Resolution:**

- If experiencing high block rates, reduce request frequency and increase retry attempts
- For JavaScript timeout errors, increase the js\_timeout parameter
- When facing persistent CAPTCHAs, verify proxy quality and geographic alignment

### Benefits and Applications

**Time Efficiency**: Automated data collection eliminates manual browsing and data entry, reducing research time from hours to minutes for large datasets.

**Business Intelligence**: Extract competitor pricing strategies, product assortments, and marketing techniques to maintain competitive advantage. Build comprehensive databases for market analysis and customer research.

**Market Research**: Identify trending products through search results analysis and keyword research. Monitor marketplace dynamics, seasonal trends, and emerging opportunities.

**Competitive Analysis**: Track competitor product launches, pricing changes, and inventory levels across multiple platforms simultaneously.

**Scalable Data Collection**: Process thousands of URLs efficiently with built-in retry logic and proxy rotation, enabling enterprise-level data gathering operations.

**Real-World Applications**:

- E-commerce price monitoring and dynamic pricing strategies
- Product development insights from customer reviews and feedback analysis
- Market penetration analysis for new business ventures
- Supply chain optimization through seller and product availability tracking

### Conclusion

The Datadome Web Scraper provides a robust solution for accessing valuable business data from protected websites while maintaining ethical scraping practices. By combining advanced bypass techniques with user-friendly configuration options, it enables businesses to gather competitive intelligence and market insights efficiently.

Ready to unlock valuable data from Datadome-protected websites? Configure your scraper today and start extracting the insights your business needs to stay competitive in the digital marketplace.

## Your feedback

We are always working to improve Actors' performance. So, if you have any technical feedback about Datadome web Scraper or simply found a bug, please create an issue on the Actor's Issues tab in Apify Console.

# Actor input Schema

## `urls` (type: `array`):

Add the URLs of the Specific web page urls you want to scrape. You can paste URLs one by one, or use the Bulk edit section to add a prepared list.

## `js_script` (type: `string`):

Add the JS script you want to run and get result.

## `retrieve_result_from_js_script` (type: `boolean`):

Retrieve result from JS script.

## `page_is_loaded_before_running_script` (type: `boolean`):

The option allows waiting until the page has loaded to run the script or running before the page is loaded.

## `execute_js_async` (type: `boolean`):

Execute JS async.

## `retrieve_html_from_url_after_loaded` (type: `boolean`):

Retrieve page HTML from url after loaded

## `js_timeout` (type: `integer`):

Timeout on JS script run in seconds

## `max_retries_per_url` (type: `integer`):

Limit the number of retries for each URL, If an error occurs during the data scraping process.

## `proxy` (type: `object`):

Select proxies to be used by your scraper.

## Actor input object example

```json
{
  "urls": [
    "https://www.etsy.com/market/top_sellers?ref=pagination&page=3"
  ],
  "js_script": "return 10 + 10 + 20",
  "retrieve_result_from_js_script": true,
  "page_is_loaded_before_running_script": true,
  "retrieve_html_from_url_after_loaded": true,
  "js_timeout": 10,
  "max_retries_per_url": 2,
  "proxy": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://www.etsy.com/market/top_sellers?ref=pagination&page=3"
    ],
    "js_script": "return 10 + 10 + 20",
    "page_is_loaded_before_running_script": true,
    "execute_js_async": false,
    "retrieve_html_from_url_after_loaded": true,
    "js_timeout": 10,
    "max_retries_per_url": 2,
    "proxy": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("ecomscrape/datadome-web-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://www.etsy.com/market/top_sellers?ref=pagination&page=3"],
    "js_script": "return 10 + 10 + 20",
    "page_is_loaded_before_running_script": True,
    "execute_js_async": False,
    "retrieve_html_from_url_after_loaded": True,
    "js_timeout": 10,
    "max_retries_per_url": 2,
    "proxy": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("ecomscrape/datadome-web-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://www.etsy.com/market/top_sellers?ref=pagination&page=3"
  ],
  "js_script": "return 10 + 10 + 20",
  "page_is_loaded_before_running_script": true,
  "execute_js_async": false,
  "retrieve_html_from_url_after_loaded": true,
  "js_timeout": 10,
  "max_retries_per_url": 2,
  "proxy": {
    "useApifyProxy": false
  }
}' |
apify call ecomscrape/datadome-web-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ecomscrape/datadome-web-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/UdDqpzcxMRqmBOkUQ/builds/zvD4g8TZZaV7TZYQP/openapi.json