The ScraperAPI MCP server enables LLM clients to retrieve and process web scraping requests using the ScraperAPI services.

• Features
• Architecture
• Installation
• API Reference
• Configuration
• Development

• Full implementation of the Model Context Protocol specification
• Seamless integration with ScraperAPI for web scraping
• Simple setup with Python

          ┌───────────────┐     ┌───────────────────────┐     ┌───────────────┐
          │  LLM Client   │────▶│  Scraper MCP Server   │────▶│    AI Model   │
          └───────────────┘     └───────────────────────┘     └───────────────┘
                                            │
                                            ▼
                                  ┌──────────────────┐
                                  │  ScraperAPI API  │
                                  └──────────────────┘

The ScraperAPI MCP Server is designed to run as a local server on your machine, your LLM client will launch it automatically when configured.

• Python 3.11+

Add this to your client configuration file:

{
  "mcpServers": {
    "ScraperAPI": {
      "command": "python",
      "args": ["-m", "scraperapi_mcp_server"],
      "env": {
        "API_KEY": "<YOUR_SCRAPERAPI_API_KEY>"
      }
    }
  }
}

which <YOUR_COMMAND>

• scrape
  • Scrape a URL from the internet using ScraperAPI
  • Parameters:
    • url (string, required): URL to scrape
    • render (boolean, optional): Whether to render the page using JavaScript. Defaults to False. Set to True only if the page requires JavaScript rendering to display its content.
    • country_code (string, optional): Activate country geotargeting (ISO 2-letter code)
    • premium (boolean, optional): Activate premium residential and mobile IPs
    • ultra_premium (boolean, optional): Activate advanced bypass mechanisms. Can not combine with premium
    • device_type (string, optional): Set request to use mobile or desktop user agents
  • Returns: The scraped content as a string

• Please scrape this URL <URL>. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.
• Can you scrape URL <URL> to extract <SPECIFIC_DATA>? If the request returns missing/incomplete<SPECIFIC_DATA>, set render=true to enable JS Rendering.

• API_KEY: Your ScraperAPI API key.

1. Open Claude Desktop Application
2. Access the Settings Menu
3. Click on the settings icon (typically a gear or three dots in the upper right corner)
4. Select the "Developer" tab
5. Click on "Edit Config" and paste the JSON configuration file.

1. Clone the repository:

   git clone https://github.com/scraperapi/scraperapi-mcp
   cd scraperapi-mcp

2. Install dependencies and run the package locally:

   • Using Python:

     # Create virtual environment and activate it
     python -m venv .venv
     source .venv/bin/activate # MacOS/Linux
     # OR
     .venv/Scripts/activate # Windows
     
     # Install the local package in editable mode
     pip install -e .

• Using Python:

  python -m scraperapi_mcp_server

python3 -m scraperapi_mcp_server --debug

This project uses pytest for testing.

# Install the package with test dependencies
pip install -e ".[test]"

# Run All Tests
pytest

# Run Specific Test
pytest <TEST_FILE_PATH>