Website to Markdown API – Clean RAG & LLM Content

Convert public websites into clean Markdown through an API-ready Apify Actor for RAG, LLM ingestion, semantic search, indexing, audits, and content analysis. Each successful page includes readable Markdown plus SEO metadata, headings, canonical URL, links, JSON-LD, crawl depth, and timestamps.

Try the API workflow first

Use the public Example Task Convert a Website to Markdown via API for a ready-to-run bounded example, then replace the URL with your own public site.

{
  "startUrls": [{ "url": "https://example.com" }],
  "maxPages": 10,
  "maxDepth": 1,
  "sameDomainOnly": true,
  "useSitemaps": true,
  "respectRobotsTxt": true
}

Start with one page for a quick test, then raise maxPages and maxDepth. Use include/exclude patterns and custom CSS removal selectors to control what enters the dataset.

Example result shape

{
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "statusCode": 200,
  "title": "Example Domain",
  "description": null,
  "canonicalUrl": null,
  "language": "en",
  "headings": ["Example Domain"],
  "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
  "textLength": 125,
  "links": ["https://www.iana.org/domains/example"],
  "jsonLd": [],
  "depth": 0,
  "scrapedAt": "2026-08-09T10:37:49.000Z"
}

Why use this website-to-Markdown API

• Readable Markdown instead of raw HTML.
• API-ready output for RAG ingestion, vector indexing, agents, and LLM pipelines.
• Main-content preference with scripts, forms, navigation, footers, and common noise removed.
• Sitemap discovery and robots.txt support.
• Same-domain control, crawl depth, and hard page caps.
• Include/exclude URL fragments and custom removal selectors.
• Built-in SSRF protection for localhost, private IPs, metadata hosts, and credentialed URLs.
• One paid result only for each successful HTML page; skipped and failed pages are summarized in OUTPUT.

Best for

• Website-to-Markdown API workflows for RAG and LLM ingestion.
• Preparing small and medium websites for semantic search or vector indexing.
• Creating Markdown corpora for LLM analysis.
• Extracting documentation, blogs, help centers, and public knowledge bases.
• SEO and content inventory workflows.

Use it as an MCP tool

Expose this Actor directly to an MCP-compatible AI client through Apify's hosted MCP server:

{
  "mcpServers": {
    "signal-lab-markdown": {
      "url": "https://mcp.apify.com?tools=signal_lab/website-to-markdown-crawler"
    }
  }
}

On the first connection, Apify opens its OAuth flow; no API token needs to be pasted into the configuration. Your agent can then call the Actor with a bounded input such as: “Crawl this documentation site to clean Markdown, stay on the same domain, and stop after 20 pages.”

See the official Apify MCP setup guide for supported clients and authentication options.

Pricing and limitations

Current launch pricing starts at $1 per 1,000 successful Markdown page results; see the live Pricing tab for the current start event, tiered pricing, and exact estimate.

This is an HTTP crawler. It does not execute client-side JavaScript or bypass login, paywalls, CAPTCHAs, robots rules, or anti-bot controls. Dynamic single-page applications may return incomplete content. Use only where you have a lawful basis and respect site terms, copyright, privacy, and crawl-rate expectations.

Free product guide

See the Website to Markdown workflow, use cases, and expected output, or browse all Signal Lab data tools.