# Startpage Search Scraper (`searchapi/startpage-search-scraper`) Actor

Scrapes Startpage web results with titles, URLs, snippets, sitelinks, dates, language, region, and source metadata. Signed proxy URLs are excluded from the dataset.

- **URL**: https://apify.com/searchapi/startpage-search-scraper.md
- **Developed by:** [Search API](https://apify.com/searchapi) (community)
- **Categories:** SEO tools, Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 86.2% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.99 / 1,000 search results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Startpage Search Scraper

Collect organic results from Startpage's public Web search as normalized, API-ready records. The Actor supports one search or a fair batch of up to 20 searches, native session-preserving pagination, result filters, Safe Search, and bounded browser runtime controls.

The Actor extracts Startpage's embedded structured result payload first, accepts both verified `web-google` and `web-bing` groups, maps validated first-party JSON responses when available, and retains a semantic DOM fallback. Advertisements, challenge pages, signed URLs, raw responses, and unrelated result widgets are never stored.

### Input

Provide either `query` or `queries`.

```json
{
  "queries": ["privacy search", "secure browsers"],
  "maxItems": 50,
  "maxPages": 5,
  "safeSearch": "moderate",
  "sortBy": "relevance",
  "includeDomains": [],
  "excludeKeywords": [],
  "requireSitelinks": false,
  "language": "english",
  "maxConcurrency": 3,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}
```

Important fields:

| Field | Purpose |
| --- | --- |
| `query` / `queries` | One phrase or up to 20 unique phrases. |
| `maxItems` | Global output limit, 1–500. |
| `maxPages` | Native Startpage pages inspected per query, 1–10. |
| `safeSearch` | `moderate`, `strict`, or `off`. |
| `sortBy` | Startpage `relevance`, `domain`, or `title`. |
| `includeDomains` / `excludeDomains` | Domain allow/block filters. |
| `includeKeywords` / `excludeKeywords` | Case-insensitive title/snippet filters. |
| `requireSitelinks` | Keep only results with source-provided sitelinks. |
| `language` | Startpage language value such as `english` or `deutsch`. |
| `region` | Optional country provenance label; proxy country controls network geolocation. |
| `maxConcurrency` | Parallel query limit, memory-capped to protect browser startup. |
| `proxyConfiguration` | No proxy, Apify Residential, or custom proxy URLs. |

`GOOGLE_SERP` is not supported. It is an HTTP proxy for supported Google Search/Shopping requests, while this Actor visits Startpage through an HTTPS Firefox session. Use Apify Residential or custom proxies when direct access is unreliable.

### Output

Each record contains non-empty values only. The 27-field contract includes:

- stable `id`, global `position`, query/source positions, and source `page`
- `title`, canonical `url`, `domain`, displayed URL, and full snippet
- source name, direct public favicon, source engine index, and source-provided sitelinks when available
- organic result type, Startpage source group, retrieval method, and extraction method
- primary query, every matched query, public search URL, Safe Search, locale provenance, and scrape time

Optional fields are omitted when Startpage does not provide them; the Actor does not emit `null`, empty strings, empty arrays, fabricated favicons, placeholder values, or signed proxy links.

### Pagination and reliability

Startpage pagination is a page-generated POST form containing session state. The Actor clicks and validates that native next-page form instead of reconstructing a fragile `page=` URL. Results from all requested pages and queries are buffered, normalized, filtered, and deduplicated before one atomic dataset write. If a later page is blocked or changes shape, the run fails without publishing a partial dataset.

Sessions, persistent cookies, consistent Crawlee Firefox fingerprints, bounded retries, and optional Residential/custom proxies improve ordinary compatibility. CAPTCHA and unusual-traffic pages are detected and rejected; the Actor does not solve or bypass CAPTCHAs.

### Local development

```bash
npm ci
npx playwright install firefox
npm test
apify run --purge --input-file qa-inputs/local-pagination.json
node validate-datasets.js
```

The `qa-inputs` directory includes single, batch, pagination, filtered, sitelink, clean-empty, low-concurrency, invalid-input, incompatible-proxy, and cloud Residential cases.

# Actor input Schema

## `query` (type: `string`):

One phrase to enter in Startpage Web search.

## `queries` (type: `array`):

Up to 20 searches; output is selected fairly across queries.

## `maxItems` (type: `integer`):

Overall output limit across all searches.

## `maxPages` (type: `integer`):

Maximum native Startpage Web pages inspected per search.

## `safeSearch` (type: `string`):

Startpage Safe Search level sent with every query and preserved by native pagination.

## `sortBy` (type: `string`):

Order each query bucket before fair cross-query selection.

## `includeDomains` (type: `array`):

Optional article-domain allowlist.

## `excludeDomains` (type: `array`):

Optional article-domain blocklist.

## `includeKeywords` (type: `array`):

Keep results whose title or snippet contains at least one value.

## `excludeKeywords` (type: `array`):

Remove results whose title or snippet contains any value.

## `requireSitelinks` (type: `boolean`):

Keep only source results containing Startpage-provided sitelinks.

## `language` (type: `string`):

Startpage result-language preference, such as english, deutsch, or francais.

## `region` (type: `string`):

Optional locale provenance label such as us, gb, de, or in; use proxy country selection for network geolocation.

## `maxConcurrency` (type: `integer`):

Maximum searches processed in parallel, capped to run memory.

## `maxRequestRetries` (type: `integer`):

Bounded retries for temporary browser, proxy, or target failures.

## `navigationTimeoutSecs` (type: `integer`):

Maximum seconds per browser navigation.

## `requestHandlerTimeoutSecs` (type: `integer`):

Maximum seconds for one query handler including pagination.

## `proxyConfiguration` (type: `object`):

Optional Apify Residential or custom proxy. GOOGLE\_SERP is incompatible with Startpage HTTPS browser traffic.

## `debug` (type: `boolean`):

Log field names and structural counts only; never raw HTML or response bodies.

## Actor input object example

```json
{
  "query": "privacy search",
  "queries": [],
  "maxItems": 50,
  "maxPages": 5,
  "safeSearch": "moderate",
  "sortBy": "relevance",
  "includeDomains": [],
  "excludeDomains": [],
  "includeKeywords": [],
  "excludeKeywords": [],
  "requireSitelinks": false,
  "language": "english",
  "region": "",
  "maxConcurrency": 3,
  "maxRequestRetries": 2,
  "navigationTimeoutSecs": 45,
  "requestHandlerTimeoutSecs": 180,
  "debug": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "privacy search"
};

// Run the Actor and wait for it to finish
const run = await client.actor("searchapi/startpage-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "privacy search" }

# Run the Actor and wait for it to finish
run = client.actor("searchapi/startpage-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "privacy search"
}' |
apify call searchapi/startpage-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,searchapi/startpage-search-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/628RGLHLDXYV7e9pX/builds/3ABoaa4f9OEW2n80c/openapi.json