# Universal Web Snapshot - HTML, Text, Markdown Capture (`fetchcraft/universal-web-snapshot`) Actor

Capture a clean HTML, plain-text, or markdown snapshot of any URL. Built for archival, change detection, and downstream LLM input. Stores rendered title, final URL, status, fetched-at timestamp. $0.003 per snapshot. Free preview run.

- **URL**: https://apify.com/fetchcraft/universal-web-snapshot.md
- **Developed by:** [Emily Ward](https://apify.com/fetchcraft) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$3.00 / 1,000 snapshot takens

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Universal Web Snapshot

![Universal Web Snapshot - HTML, Text, Markdown Capture](https://ai-sales-personalizer.vercel.app/og/universal-web-snapshot.png)

**Snapshot any URL via a connector chain: static HTML first, Playwright browser if static returns thin content, Wayback Machine if both fail.**

Pure scraper. No API keys. No LLM dependency. Designed to be robust against the three most common scraping failure modes:

| Failure mode | Tier that handles it |
| --- | --- |
| Site returns HTML but it's a JS-only SPA (Notion, Linear, Figma) | Playwright browser |
| Site blocks datacenter IPs | Apify proxy routing |
| Site is down right now but has Wayback snapshots | Wayback Machine |

### What you get back per URL

| Field | Description |
| --- | --- |
| `input_url`, `final_url`, `status` | Identity + HTTP response |
| `tier_used` | "static" / "browser" / "wayback" - tells you which connector won |
| `archived_at` | If Wayback was used, the snapshot timestamp |
| `signals.title`, `og_title`, `meta_description`, `og_description`, `canonical_url` | Standard meta fields |
| `signals.h1[]`, `h2[]` | First N headings (typically your hero copy) |
| `signals.json_ld[]` | Any JSON-LD structured data the page exposes |
| `text_excerpt` (first 2000 chars), `text_full` (up to 20k), `text_length` | Cleaned text content |
| `screenshot_url` | Optional PNG (Playwright tier only) saved to actor KV store |
| `tries[]` | Per-tier result so you can see what was attempted |
| `elapsed_ms` | Total processing time |

### Pricing

**$0.05 per successful snapshot.** Failed URLs (all three tiers exhausted) are not charged.

| Use case | URLs | Cost |
| --- | --- | --- |
| Quick competitor sweep | 20 | $1.00 |
| Daily monitoring (100 URLs/day) | 3,000/mo | $150/mo |
| One-off content audit | 500 | $25.00 |

### How the connector chain works

```
1. Static fetch (cheap, fast)        ─┐
   ├─ HTTP 200 + > 300 chars         │ Done. tier_used="static".
   └─ Thin/blocked/missing content   ─┤
                                      │
2. Playwright browser (heavier)      ─┤
   ├─ Renders JS, waits for network  │ Done. tier_used="browser".
   └─ Browser failure / still thin   ─┤
                                      │
3. Wayback Machine (last resort)     ─┤
   ├─ Finds archived snapshot        │ Done. tier_used="wayback".
   └─ No archive available           ─┘ Error returned.
```

The buyer is only charged for successful snapshots, regardless of how many tiers were tried.

### Use cases

- **Pricing intel**: monitor competitor pricing pages even when they are JS-rendered.
- **Content audits**: snapshot 500 URLs across a domain, get clean text for analysis.
- **AI training data**: prepare structured input for LLM pipelines from any web source.
- **Compliance / legal**: archive a copy of pages with a Wayback timestamp for evidence.
- **SEO research**: extract title + meta + h1/h2 across competitor sites.
- **Investor due diligence**: snapshot a company's site at a point in time.

### Why this is a connector / plugin architecture

The actor's `src/lib/scraping.js` exposes:

- `fetchStatic(url, opts)` - tier 1 implementation
- `fetchBrowser(url, opts)` - tier 2 implementation
- `fetchWayback(url, opts)` - tier 3 implementation
- `smartFetch(url, opts)` - the orchestrator
- `cleanHtml(html)`, `extractSignals(html)`, `normalizeUrl(input)` - shared utilities

Any future scraper actor can import the same lib. This means a Pricing Watcher v2 (or any other actor that needs robust scraping) gets the same multi-tier fetch for free. The pattern lives once.

### What this actor does NOT do

- It does not log into authenticated sites.
- It does not download non-HTML assets (PDFs, videos, etc).
- It does not paginate within a single URL (use a separate crawler for that).

### Tags

`scraping` `web-snapshot` `wayback` `playwright` `connector` `static-fetch` `headless-chrome` `content-extraction` `seo`

***

Made by Emily Ward, Cancel Costs.

### Pairs well with

- **[pricing-page-watcher](https://apify.com/fetchcraft/pricing-page-watcher)**: Snapshot competitor pricing pages, diff over time. $0.005 per check.
- **[shopify-store-detector](https://apify.com/fetchcraft/shopify-store-detector)**: Snapshot Shopify pages and extract stack. $0.03 per store.
- **[wordpress-stack-detector](https://apify.com/fetchcraft/wordpress-stack-detector)**: Snapshot WP pages and detect plugins. $0.02 per site.

### Integrations

This actor works out of the box with every Apify-supported integration:

- **API**: call via Apify API or any official SDK (Python, JavaScript, PHP, .NET). Returns a clean dataset URL.
- **Schedule**: set a daily, weekly, or custom cron cadence in Apify Console. Combine with `notification` for fresh feeds.
- **Webhooks**: wire `ACTOR.RUN.SUCCEEDED` to Slack, Discord, Zapier, Make, n8n, Pipedream, or any HTTPS endpoint.
- **MCP**: this actor is discoverable through Apify's hosted MCP server at `mcp.apify.com` for Claude, Cursor, Cline, Windsurf, and other MCP clients.
- **n8n / Make / Zapier**: native HTTP-Request integration. Trigger the actor on schedule, pipe results to Google Sheets, Airtable, your CRM, or any database.

### Try it free

Every Apify user gets **$5/month in free platform credits** (around 250 events at this actor's per-event price). Run preview mode first to confirm output shape before scaling.

New to Apify? [Sign up here](https://apify.com/sign-up?fpr=fetchcraft) to get free credits on signup.

### What's New

- 2026-06-03: Metadata, categories, and SEO refreshed. Latest version live on Apify Store.

### Last Updated

2026-06-03

# Actor input Schema

## `urls` (type: `array`):

List of URLs to fetch. Each URL is tried via the static-fetch tier first, then the Playwright tier if static returns thin content, then the Wayback Machine if both fail.

## `enable_browser` (type: `boolean`):

If on, fall back to a real Chromium browser when static HTML is too thin (typical for SPAs like Notion, Linear, Figma). Adds ~3-8 seconds per URL to runs that need it.

## `enable_wayback` (type: `boolean`):

If on and both static + browser fail, fetch the latest Wayback Machine snapshot for the URL.

## `want_screenshot` (type: `boolean`):

If on (and the Playwright tier was used), save a PNG to the actor's key-value store and include a URL in the output. Adds ~1 second + extra storage.

## `content_must_match` (type: `string`):

If set, the fetch is considered 'thin' (and the next tier is tried) unless the returned HTML matches this case-insensitive regex. Useful for SPAs where static HTML loads but the actual content is JS-rendered.

## `proxy_groups` (type: `array`):

Apify proxy groups to route the static-fetch tier through. Leave default to use shared datacenter proxies. Empty array = no proxy.

## `max_concurrency` (type: `integer`):

How many URLs to snapshot in parallel. Lower if you have many heavy browser-tier URLs.

## Actor input object example

```json
{
  "urls": [
    "https://linear.app/pricing",
    "https://www.notion.so/pricing"
  ],
  "enable_browser": true,
  "enable_wayback": true,
  "want_screenshot": false,
  "proxy_groups": [
    "BUYPROXIES94952"
  ],
  "max_concurrency": 3
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://linear.app/pricing",
        "https://www.notion.so/pricing"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fetchcraft/universal-web-snapshot").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://linear.app/pricing",
        "https://www.notion.so/pricing",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("fetchcraft/universal-web-snapshot").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://linear.app/pricing",
    "https://www.notion.so/pricing"
  ]
}' |
apify call fetchcraft/universal-web-snapshot --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fetchcraft/universal-web-snapshot"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/CwtYGRpaIzEdeCdez/builds/6V1vIXuZfD0LqyLvq/openapi.json