# Katana Web Crawler (ProjectDiscovery) (`rl1987/pd-katana`) Actor

Crawl websites with ProjectDiscovery Katana and stream results as structured JSONL into an Apify dataset.

- **URL**: https://apify.com/rl1987/pd-katana.md
- **Developed by:** [R.L.](https://apify.com/rl1987) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 4 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.01 / actor start

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Katana Web Crawler – Fast Website URL & Endpoint Extractor

**Katana Web Crawler** is a fast, no-code **website crawler and URL extractor** built on [**ProjectDiscovery Katana**](https://github.com/projectdiscovery/katana), the popular open-source crawling and spidering framework. Point it at any website, and it **crawls every page, link, JavaScript file, form, and API endpoint**, then delivers the results as a clean, **structured dataset you can download as JSON, CSV, Excel, or XML** — or pull straight from the [Apify API](https://docs.apify.com/api/v2).

Run it in the cloud with **zero setup**: no Go install, no binaries, no servers. You get **headless Chromium crawling, JavaScript endpoint discovery, proxy rotation, scheduling, and integrations** (Make, Zapier, Google Sheets, Slack, and more) out of the box.

> **Try it now:** paste a URL like `https://example.com`, hit **Start**, and watch discovered URLs stream into your dataset in seconds.

### What is Katana Web Crawler?

Katana Web Crawler is a serverless [Apify Actor](https://apify.com/actors) that **scrapes all URLs from a website** and maps its complete structure. It's the fastest way to **crawl a site, list every reachable link, and export the URLs** without writing code. Use it for **web crawling, site mapping, URL discovery, attack-surface enumeration, SEO audits, and API endpoint discovery**.

### What does Katana Web Crawler do?

- 🕷️ **Crawls entire websites** in standard (fast HTTP) or **headless Chromium** mode.
- 🔗 **Extracts every URL** — links, assets, scripts, forms, and inputs.
- 🧩 **Discovers hidden API endpoints** parsed from **JavaScript files** (`js-crawl` + `jsluice`).
- 📡 **Captures XHR / fetch API calls** made by JavaScript-heavy single-page apps.
- 📂 **Finds known files** such as `robots.txt` and `sitemap.xml`.
- 🎯 **Filters precisely** with scope, regex, extension, and DSL conditions.
- 🛠️ **Detects technologies** and extracts forms while it crawls.
- 📊 **Exports flat, ready-to-use data** — every field is its own column.

### Why use Katana Web Crawler?

Whether you're a **security researcher, penetration tester, SEO specialist, or developer**, this crawler turns any website into structured data in minutes:

- **Attack-surface mapping** — enumerate every reachable endpoint of a web app before a pentest or bug-bounty hunt.
- **Security reconnaissance** — feed the URL list into scanners like **Nuclei, httpx, or ffuf**.
- **API endpoint discovery** — surface undocumented REST/GraphQL endpoints referenced in JavaScript.
- **SEO & site audits** — inventory pages, assets, redirects, and broken links at scale.
- **Content & asset discovery** — find every script, image, PDF, and download link on a domain.
- **Migration & QA** — snapshot a site's full URL structure before and after a release.

### How to use Katana Web Crawler to crawl a website

No coding required — you can run your first crawl in under a minute:

1. Click **Try for free** / open the Actor and go to the **Input** tab.
2. Add one or more **Target URLs** (e.g. `https://example.com`).
3. Set the crawl **depth**, and optionally enable **JavaScript crawling** or **headless** mode for modern web apps.
4. (Optional) Add **scope, regex, or extension filters** to collect only what you need.
5. Click **Start** and watch URLs stream into the **Output** tab in real time.
6. **Download** the dataset as JSON, CSV, Excel, or XML — or fetch it via the [Apify API](https://docs.apify.com/api/v2).

Prefer automation? **Schedule** recurring crawls or trigger the Actor from your own app, Make, or Zapier.

### Features at a glance

| Capability | Supported |
|------------|-----------|
| Standard (HTTP) crawling | ✅ |
| Headless Chromium crawling | ✅ |
| JavaScript endpoint parsing (js-crawl, jsluice) | ✅ |
| XHR / fetch API extraction | ✅ |
| Form extraction & auto-fill | ✅ |
| Technology detection | ✅ |
| robots.txt & sitemap.xml crawling | ✅ |
| Scope, regex, extension & DSL filters | ✅ |
| Proxy support (HTTP/SOCKS5) | ✅ |
| Rate limiting & concurrency control | ✅ |
| JSON / CSV / Excel / XML export | ✅ |
| API access, scheduling & integrations | ✅ |

### Input

All fields are configured on the **Input** tab, grouped into sections. The only required field is `urls` — everything else has a sensible default. Almost the entire Katana CLI is exposed; the tables below group the main options.

**Targets**

| Field | Type | Description |
|-------|------|-------------|
| `urls` | array | Target URLs / domains to crawl (required). |
| `exclude` | array | Exclude hosts by `cdn`, `private-ips`, CIDR, IP, or regex. |

**Configuration**

| Field | Type | Description |
|-------|------|-------------|
| `depth` | integer | Maximum crawl depth (default `3`). |
| `crawlDuration` | integer | Max crawl time per target in seconds (`0` = unlimited). |
| `strategy` | string | `depth-first` or `breadth-first`. |
| `jsCrawl` / `jsluice` | boolean | Parse endpoints in JavaScript (jsluice = deeper, memory-heavy). |
| `knownFiles` | string | Crawl `robots.txt` / `sitemap.xml` (`none`/`all`/`robotstxt`/`sitemapxml`). |
| `automaticFormFill` / `formExtraction` | boolean | Auto-fill forms / extract form elements. |
| `techDetect` / `knowledgeBase` | boolean | Technology detection / knowledge-base classification. |
| `pathClimb` | boolean | Auto-crawl parent paths. |
| `ignoreQueryParams` / `filterSimilar` / `filterSimilarThreshold` | mixed | Collapse duplicate/similar URLs. |
| `disableRedirects` / `tlsImpersonate` | boolean | Stop following redirects / JA3 TLS randomization. |
| `retry` / `timeout` / `timeStable` / `maxResponseSize` | integer | Request tuning. |
| `headers` / `resolvers` | array | Custom headers/cookies / DNS resolvers. |

**Headless (Chromium)**

| Field | Type | Description |
|-------|------|-------------|
| `headless` / `hybrid` | boolean | Headless / hybrid Chromium crawling. |
| `xhrExtraction` | boolean | Capture XHR request URL & method. |
| `noIncognito` / `headlessOptions` / `chromeWsUrl` | mixed | Chrome behaviour, extra flags, remote Chrome. |
| `captchaSolverProvider` / `captchaSolverKey` | string | Captcha solver integration (key is secret). |

**Scope**

| Field | Type | Description |
|-------|------|-------------|
| `fieldScope` | string | `dn`, `rdn`, `fqdn`, or a custom regex. |
| `crawlScope` / `crawlOutScope` | array | In-scope / out-of-scope URL regexes. |
| `noScope` / `displayOutScope` | boolean | Disable scoping / show external endpoints. |

**Filters**

| Field | Type | Description |
|-------|------|-------------|
| `matchRegex` / `filterRegex` | array | Match or drop output URLs by regex. |
| `extensionMatch` / `extensionFilter` / `noDefaultExtFilter` | mixed | Filter by extension. |
| `matchCondition` / `filterCondition` | string | DSL-based response conditions. |
| `filterPageType` | array | Drop `error` / `captcha` / `parked` pages. |
| `disableUniqueFilter` | boolean | Disable duplicate-content filtering. |

**Rate limiting**

| Field | Type | Description |
|-------|------|-------------|
| `concurrency` / `parallelism` | integer | Concurrent fetchers / inputs. |
| `rateLimit` / `rateLimitMinute` / `delay` | integer | Requests per second / minute / inter-request delay. |

**Output & advanced**

| Field | Type | Description |
|-------|------|-------------|
| `flattenOutput` | boolean | Flatten nested fields into columns (default `true`). |
| `proxyUrl` | string | HTTP/SOCKS5 proxy URL. |
| `omitBody` / `omitRaw` | boolean | Trim response body / raw dumps (default `true`). |
| `storeField` / `excludeOutputFields` | mixed | Add / remove output fields. |
| `verbose` | boolean | Verbose Katana logs (to run log, not dataset). |
| `extraArgs` | array | Any remaining raw Katana flags, passed verbatim. |

Example input:

```json
{
    "urls": ["https://example.com/"],
    "depth": 2,
    "jsCrawl": true,
    "knownFiles": "all"
}
```

### Output

Each dataset item represents one crawled URL. By default (`flattenOutput: true`) the nested `request` / `response` objects are flattened into dotted top-level fields, so **every field — including each response header — is its own column**, ready for the table view and CSV/Excel exports:

```json
{
    "timestamp": "2026-07-03T12:00:00.000000Z",
    "request.method": "GET",
    "request.endpoint": "https://example.com/about",
    "request.tag": "a",
    "request.attribute": "href",
    "request.source": "https://example.com/",
    "response.status_code": 200,
    "response.content_length": 1256,
    "response.headers.Content-Type": "text/html; charset=UTF-8",
    "response.technologies": ["Nginx"]
}
```

Set `flattenOutput: false` to keep the raw nested Katana structure instead.

**Note on field counts:** Katana emits a row per discovered endpoint. Pages it actually fetches carry full response fields (status, headers, technologies, and — unless `omitBody`/`omitRaw` are on — the body and raw request/response). Endpoints it only discovers (child links, JS/XHR endpoints) are lighter, request-only rows. So some rows legitimately have fewer fields than others.

You can download the dataset in various formats such as **JSON, CSV, Excel, or XML**.

#### Data table

| Field | Description |
|-------|-------------|
| `timestamp` | When the endpoint was crawled. |
| `request.method` | HTTP method. |
| `request.endpoint` | The discovered URL. |
| `request.source` | Page the URL was found on. |
| `response.status_code` | HTTP status returned. |
| `response.content_length` | Response size in bytes. |
| `response.technologies` | Detected technologies (with `techDetect`). |

### How much does it cost to crawl a website?

This Actor uses **pay-per-event** pricing:

- **$0.01 flat fee per run** — charged once when the crawl starts, **no matter how many URLs you crawl**. Crawl 10 pages or 100,000; the Actor fee is the same.
- **Plus Apify platform usage** (compute, proxy, data transfer) — passed through at cost, so you only pay for the resources your crawl actually consumes.

New Apify accounts include **free monthly usage credits**, enough to run plenty of crawls at no cost.

**Tips to control cost:** start with a low `depth`, set a bounded `crawlDuration`, and reserve `headless` mode for sites that truly need JavaScript rendering.

### Tips and advanced options

- Keep `omitBody` and `omitRaw` enabled (default) to keep dataset items small; disable them to capture full response bodies and raw request/response dumps.
- Output is flattened by default so all fields appear as columns; set `flattenOutput: false` for raw nested JSON.
- For JavaScript-heavy sites and SPAs, enable `headless` + `jsCrawl` + `xhrExtraction` and give the crawl a longer `crawlDuration` so pages have time to render and get fetched.
- Use `crawlScope` / `fieldScope` to keep the crawl on-domain and avoid wandering off to external sites.
- Set `crawlDuration` to cap runtime and cost on large targets.
- Use `extraArgs` for any flag not exposed in the schema, e.g. `["-headers", "Cookie: session=abc"]`.

### Frequently asked questions

**How do I crawl all URLs on a website?** Add the site to `urls`, set a `depth` (2–3 is a good start), and run the Actor. Every discovered URL lands in the dataset, ready to export.

**Can it crawl JavaScript-heavy or single-page apps?** Yes. Enable `headless` for real Chromium rendering, plus `jsCrawl`/`jsluice` to parse endpoints out of JavaScript and `xhrExtraction` to capture API calls.

**Can it discover API endpoints?** Yes — `jsCrawl` and `jsluice` extract REST/GraphQL endpoints referenced in JavaScript, and `xhrExtraction` records live XHR/fetch requests.

**What output formats are supported?** JSON, CSV, Excel (XLSX), XML, JSONL, and RSS — via the Output tab or the Apify API.

**Can I schedule recurring crawls?** Yes. Use Apify **Schedules**, or trigger the Actor from Make, Zapier, or your own code through the API.

**Is web crawling legal?** Crawling publicly available data is generally legal, but you are responsible for complying with the target site's Terms of Service, `robots.txt`, and applicable laws. Only crawl sites you own or are authorized to test.

**Which Katana version is used?** The Docker image pins a specific Katana release (see `KATANA_VERSION` in the Dockerfile).

### Support

Found a bug or want a feature? Open an issue on the Actor's **Issues** tab — we actively maintain this Actor and welcome feedback.

### Resources

- [Katana on GitHub](https://github.com/projectdiscovery/katana)
- [Apify SDK for Python](https://docs.apify.com/sdk/python)
- [Apify Platform documentation](https://docs.apify.com/platform)
- [Apify integrations](https://apify.com/integrations)

### Data pipeline toolkit

Part of the **Data pipeline toolkit** — small, chainable Actors for cleaning, transforming, and generating data inside a larger pipeline:

- [jq Helper – transform JSON with jq](https://apify.com/rl1987/jq-helper) — Run jq programs over inline JSON or a linked Apify dataset.
- [DuckDB Helper – SQL over CSV, JSON, Parquet, Excel, SQLite](https://apify.com/rl1987/duckdb-wrapper) — Run a DuckDB SQL query over remote/local files, push results to a dataset.
- [Regex Helper](https://apify.com/rl1987/regex-helper) — Apply named regular expressions to strings, extract structured matches.
- [URL Wrangler](https://apify.com/rl1987/url-wrangler) — Join, decompose, and rewrite URLs and query params in batch.
- [ZIP Code Helper](https://apify.com/rl1987/zip-code-helper) — Resolves US ZIP codes into city, state, county, and more.
- [Postal Address Normaliser](https://apify.com/rl1987/postal-address-normaliser) — Parses and normalises postal addresses using libpostal.
- [Phone Number Wrangler](https://apify.com/rl1987/phone-number-wrangler) — Validate, format, and parse phone numbers using libphonenumber.
- [UUID Generator](https://apify.com/rl1987/uuid-generator) — Generate bulk UUIDs (v1, v3, v4, v5, v7) on demand.
- [Secure Password & Passphrase Generator](https://apify.com/rl1987/password-generator) — Generate secure passwords and diceware passphrases per NIST guidance.
- [Thumbnail Maker](https://apify.com/rl1987/thumbnail-maker) — Generates thumbnails from image URLs using ImageMagick.
- [ProjectDiscovery Notify](https://apify.com/rl1987/pd-notify) — Stream records to Slack, Discord, Telegram, Email, and more.

### Did you find this useful?

⭐ Rate this actor on Apify! Your feedback helps other users find it and helps us keep improving it.

# Actor input Schema

## `urls` (type: `array`):

One or more target URLs / domains to crawl. Each entry is passed to Katana as a seed (-u).

## `exclude` (type: `array`):

Exclude hosts matching a filter: 'cdn', 'private-ips', a CIDR, an IP, or a regex (Katana -exclude).

## `depth` (type: `integer`):

Maximum depth to crawl (Katana -depth).

## `crawlDuration` (type: `integer`):

Maximum time to crawl per target in seconds (Katana -crawl-duration). 0 = unlimited.

## `strategy` (type: `string`):

Traversal order (Katana -strategy).

## `jsCrawl` (type: `boolean`):

Parse & crawl endpoints inside JavaScript files (Katana -js-crawl).

## `jsluice` (type: `boolean`):

Enable jsluice parsing in JavaScript files. Memory intensive (Katana -jsluice).

## `knownFiles` (type: `string`):

Crawl known files such as robots.txt and sitemap.xml (requires depth >= 3).

## `automaticFormFill` (type: `boolean`):

Automatically fill and submit forms while crawling. Experimental (Katana -automatic-form-fill).

## `formExtraction` (type: `boolean`):

Extract form, input, textarea & select elements into output (Katana -form-extraction).

## `techDetect` (type: `boolean`):

Enable technology detection on crawled responses (Katana -tech-detect).

## `knowledgeBase` (type: `boolean`):

Enable knowledge base classification (Katana -knowledge-base).

## `pathClimb` (type: `boolean`):

Auto-crawl parent paths of discovered URLs (Katana -path-climb).

## `ignoreQueryParams` (type: `boolean`):

Ignore crawling the same path with different query-param values (Katana -ignore-query-params).

## `filterSimilar` (type: `boolean`):

Filter crawling of similar-looking URLs, e.g. /users/123 and /users/456 (Katana -filter-similar).

## `filterSimilarThreshold` (type: `integer`):

Distinct values before a path position is treated as a parameter (Katana -filter-similar-threshold). 0 = default.

## `disableRedirects` (type: `boolean`):

Do not follow HTTP redirects (Katana -disable-redirects).

## `tlsImpersonate` (type: `boolean`):

Enable experimental client-hello (JA3) TLS randomization (Katana -tls-impersonate).

## `retry` (type: `integer`):

Number of times to retry a failed request (Katana -retry). 0 = Katana default.

## `timeout` (type: `integer`):

Time to wait for a request in seconds (Katana -timeout).

## `timeStable` (type: `integer`):

Time to wait until the page is stable, headless mode (Katana -time-stable). 0 = default.

## `maxResponseSize` (type: `integer`):

Maximum response size to read (Katana -max-response-size). 0 = Katana default (4 MB).

## `headers` (type: `array`):

Custom headers/cookies for every request, in 'Name: value' format (Katana -headers).

## `resolvers` (type: `array`):

Custom DNS resolvers, e.g. 1.1.1.1, 8.8.8.8 (Katana -resolvers).

## `headless` (type: `boolean`):

Enable headless Chromium crawling. Slower but handles JS-heavy sites (Katana -headless).

## `hybrid` (type: `boolean`):

Enable headless hybrid crawling combining standard and headless (Katana -hybrid).

## `xhrExtraction` (type: `boolean`):

Extract XHR request URL & method into output (Katana -xhr-extraction).

## `noIncognito` (type: `boolean`):

Start headless Chrome without incognito mode (Katana -no-incognito).

## `headlessOptions` (type: `array`):

Additional flags passed to headless Chrome (Katana -headless-options).

## `chromeWsUrl` (type: `string`):

Connect to a Chrome instance launched elsewhere via its debugger WebSocket URL (Katana -chrome-ws-url).

## `captchaSolverProvider` (type: `string`):

Captcha solver provider, e.g. capsolver (Katana -captcha-solver-provider).

## `captchaSolverKey` (type: `string`):

API key for the captcha solver provider (Katana -captcha-solver-key).

## `fieldScope` (type: `string`):

Pre-defined scope (dn, rdn, fqdn) or a custom regex (Katana -field-scope).

## `crawlScope` (type: `array`):

Regexes of URLs that are in scope and should be followed (Katana -crawl-scope).

## `crawlOutScope` (type: `array`):

Regexes of URLs that are out of scope and should not be followed (Katana -crawl-out-scope).

## `noScope` (type: `boolean`):

Disable the host-based default scope, crawling everything reachable (Katana -no-scope).

## `displayOutScope` (type: `boolean`):

Display external endpoints found during scoped crawling (Katana -display-out-scope).

## `matchRegex` (type: `array`):

Only output URLs matching these regexes (Katana -match-regex).

## `filterRegex` (type: `array`):

Drop output URLs matching these regexes (Katana -filter-regex).

## `extensionMatch` (type: `array`):

Only output URLs with these extensions, e.g. php, html, js (Katana -extension-match).

## `extensionFilter` (type: `array`):

Drop output URLs with these extensions, e.g. png, css (Katana -extension-filter).

## `noDefaultExtFilter` (type: `boolean`):

Remove the built-in default extension filter list (Katana -no-default-ext-filter).

## `matchCondition` (type: `string`):

Match responses with a DSL-based condition (Katana -match-condition).

## `filterCondition` (type: `string`):

Filter responses with a DSL-based condition (Katana -filter-condition).

## `filterPageType` (type: `array`):

Filter responses by page type: error, captcha, parked (Katana -filter-page-type).

## `disableUniqueFilter` (type: `boolean`):

Disable duplicate-content filtering (Katana -disable-unique-filter).

## `concurrency` (type: `integer`):

Number of concurrent fetchers (Katana -concurrency).

## `parallelism` (type: `integer`):

Number of inputs processed in parallel (Katana -parallelism).

## `rateLimit` (type: `integer`):

Maximum requests to send per second (Katana -rate-limit).

## `rateLimitMinute` (type: `integer`):

Maximum requests to send per minute (Katana -rate-limit-minute). 0 = disabled.

## `delay` (type: `integer`):

Delay between each request in seconds (Katana -delay). 0 = none.

## `proxyConfiguration` (type: `object`):

Proxy to route Katana's requests through. Use Apify Proxy (datacenter or residential), or supply your own proxy URLs. Leave 'Use Apify Proxy' off and empty for no proxy.

## `proxyUrl` (type: `string`):

HTTP/SOCKS5 proxy, e.g. http://user:pass@host:port (Katana -proxy). Takes precedence over Proxy configuration when set. Leave empty to use Proxy configuration instead.

## `flattenOutput` (type: `boolean`):

Flatten each record's nested request/response objects into dotted top-level fields (e.g. response.status\_code) so every field shows as its own dataset column. Disable to keep the raw nested Katana JSON.

## `omitBody` (type: `boolean`):

Omit the response body from JSONL output to reduce dataset size (Katana -omit-body).

## `omitRaw` (type: `boolean`):

Omit raw request/response dumps from JSONL output (Katana -omit-raw).

## `storeField` (type: `string`):

Additional per-host field to store, e.g. url,path,fqdn,rdn (Katana -store-field).

## `excludeOutputFields` (type: `array`):

Fields to exclude from the JSONL output records (Katana -exclude-output-fields).

## `verbose` (type: `boolean`):

Enable verbose Katana logging (Katana -verbose). Logs go to the run log, not the dataset.

## `extraArgs` (type: `array`):

Additional raw Katana CLI flags for anything not exposed above, passed verbatim, e.g. \["-time-stable", "2"].

## Actor input object example

```json
{
  "urls": [
    "https://example.com/",
    "https://test.com/"
  ],
  "depth": 3,
  "crawlDuration": 0,
  "strategy": "depth-first",
  "jsCrawl": false,
  "jsluice": false,
  "knownFiles": "none",
  "automaticFormFill": false,
  "formExtraction": false,
  "techDetect": false,
  "knowledgeBase": false,
  "pathClimb": false,
  "ignoreQueryParams": false,
  "filterSimilar": false,
  "filterSimilarThreshold": 0,
  "disableRedirects": false,
  "tlsImpersonate": false,
  "retry": 0,
  "timeout": 10,
  "timeStable": 0,
  "maxResponseSize": 0,
  "headers": [
    "Cookie: session=abc",
    "User-Agent: my-crawler"
  ],
  "headless": false,
  "hybrid": false,
  "xhrExtraction": false,
  "noIncognito": false,
  "fieldScope": "rdn",
  "noScope": false,
  "displayOutScope": false,
  "noDefaultExtFilter": false,
  "disableUniqueFilter": false,
  "concurrency": 10,
  "parallelism": 10,
  "rateLimit": 150,
  "rateLimitMinute": 0,
  "delay": 0,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "flattenOutput": true,
  "omitBody": true,
  "omitRaw": true,
  "verbose": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("rl1987/pd-katana").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://example.com/"] }

# Run the Actor and wait for it to finish
run = client.actor("rl1987/pd-katana").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com/"
  ]
}' |
apify call rl1987/pd-katana --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rl1987/pd-katana"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/JKK7rE3vf2q9L539a/builds/epbbgw3LQt5pWWATQ/openapi.json