# Stack Overflow Scraper — Questions, Answers & Tags for RAG (`ahampton83/stackoverflow-scraper`) Actor

Fetch Stack Overflow questions, answers, and comments as clean structured data with markdown formatting. Search by keyword, explore tags, or get full question threads. MCP-enabled for Claude, Cursor, and AI agents.

- **URL**: https://apify.com/ahampton83/stackoverflow-scraper.md
- **Developed by:** [Aaron Hampton](https://apify.com/ahampton83) (community)
- **Categories:** AI, Developer tools, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Stack Overflow Scraper — Questions, Answers & Tags for RAG

Fetch Stack Overflow questions, answers, and comments as clean structured data with markdown-formatted bodies. Search by keyword, explore top questions for a tag, or get full question threads with accepted answers.

Perfect for RAG pipelines, developer research, AI agents, and technical Q\&A datasets.

### Features

- **Keyword search** — search all Stack Overflow questions by keyword and tags
- **Full question threads** — get questions with bodies, answers, and comments in clean markdown
- **Tag exploration** — fetch top-voted questions for any tag (python, javascript, react, etc.)
- **User profiles** — fetch user reputation, badges, and top questions
- **Clean markdown** — all HTML bodies converted to readable markdown (code blocks, lists, links, images, tables, blockquotes)
- **Answer ordering** — sorted by score with accepted answers first
- **Dual-mode** — run as a normal Apify Actor OR connect as an MCP server for Claude, Cursor, and other AI agents
- **No browser required** — uses the free Stack Exchange API, no proxies or auth needed

### Use Cases

- **RAG pipelines** — feed Stack Overflow Q\&A into vector databases for code-aware AI assistants
- **Developer research** — find canonical answers to programming questions across technologies
- **AI agents** — let Claude/Cursor search Stack Overflow via MCP tools
- **Documentation** — collect top answers for a tag to build curated knowledge bases
- **Trend analysis** — track popular topics and technologies by tag and score

### Input (Normal Actor Mode)

| Field | Type | Description |
|-------|------|-------------|
| `mode` | enum | `search`, `question`, `tag`, or `user` |
| `query` | string | Search keyword (mode=search) |
| `questionId` | integer | Stack Overflow question ID (mode=question) |
| `tag` | string | Tag name like "python" (mode=tag) |
| `userId` | integer | Stack Overflow user ID (mode=user) |
| `tags` | array | Filter search by tags (mode=search) |
| `acceptedOnly` | boolean | Only questions with accepted answers (mode=search) |
| `maxQuestions` | integer | Max questions to return (default 10) |
| `includeAnswers` | boolean | Fetch answers for each question (default true) |
| `includeComments` | boolean | Fetch comments on questions (default false) |
| `maxAnswersPerQuestion` | integer | Max answers per question (default 5) |
| `sortBy` | enum | `relevance`, `votes`, or `newest` (default relevance) |

### Output

````json
{
  "id": 393272,
  "title": "How to reverse a list in Python",
  "url": "https://stackoverflow.com/questions/393272/...",
  "score": 2534,
  "viewCount": 1200000,
  "answerCount": 15,
  "tags": ["python", "list", "reverse"],
  "isAnswered": true,
  "acceptedAnswerId": 393280,
  "createdAt": "2010-10-13T10:00:00.000Z",
  "bodyMarkdown": "I want to reverse a list:\n```python\nmy_list = [1, 2, 3]\n```",
  "answers": [
    {
      "id": 393280,
      "score": 1800,
      "isAccepted": true,
      "createdAt": "2010-10-13T10:05:00.000Z",
      "bodyMarkdown": "Use slicing:\n```python\nreversed_list = my_list[::-1]\n```"
    }
  ]
}
````

### MCP Tools

When connected as an MCP server, the following tools are available:

| Tool | Description |
|------|-------------|
| `search_questions` | Search Stack Overflow by keyword. Returns titles, scores, tags, and optionally answers. |
| `get_question` | Fetch a full question with markdown body, answers (sorted accepted-first), and optional comments. |
| `get_top_answers` | Get top-voted questions for a tag with their best answers. Great for discovering canonical Q\&A. |

### Pricing

Pay-per-event. First results are free; subsequent events billed at:

| Event | Price |
|-------|-------|
| Question fetched | $0.002 |
| Answer fetched | $0.001 |
| MCP tool call | $0.005 |
| Actor start | $0.00005 |

Volume discounts available at higher tiers.

### Data Source

This Actor uses the [Stack Exchange API](https://api.stackexchange.com/) — a free, public API with generous rate limits (300 requests/2min without a key, 10,000/2min with a key). No authentication or proxies required.

# Actor input Schema

## `mode` (type: `string`):

search = keyword search, question = full question with answers, tag = top questions for a tag, user = user profile + top questions

## `query` (type: `string`):

Keyword search query (required for mode=search). Searches SO question titles via the Stack Exchange API.

## `questionId` (type: `integer`):

Stack Overflow question ID (required for mode=question). E.g., 393272 for 'How to reverse a list in Python'.

## `tag` (type: `string`):

Stack Overflow tag name (required for mode=tag). E.g., 'python', 'javascript', 'react'.

## `userId` (type: `integer`):

Stack Overflow user ID (required for mode=user). E.g., 22656 for Jon Skeet.

## `tags` (type: `array`):

Filter search results to questions with these tags (mode=search).

## `acceptedOnly` (type: `boolean`):

Filter to questions that have an accepted answer (mode=search).

## `maxQuestions` (type: `integer`):

Maximum number of questions to return.

## `includeAnswers` (type: `boolean`):

Fetch answers for each question.

## `includeComments` (type: `boolean`):

Fetch comments for each question.

## `maxAnswersPerQuestion` (type: `integer`):

Maximum answers to fetch per question.

## `sortBy` (type: `string`):

Sort order for search results.

## `outputFormat` (type: `string`):

markdown = include markdown-formatted bodies, json = raw structured data only.

## Actor input object example

```json
{
  "mode": "search",
  "query": "",
  "questionId": 0,
  "tag": "",
  "userId": 0,
  "tags": [],
  "acceptedOnly": false,
  "maxQuestions": 10,
  "includeAnswers": true,
  "includeComments": false,
  "maxAnswersPerQuestion": 5,
  "sortBy": "relevance",
  "outputFormat": "markdown"
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("ahampton83/stackoverflow-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("ahampton83/stackoverflow-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call ahampton83/stackoverflow-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ahampton83/stackoverflow-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/uDQyJuuCVfLurCJaf/builds/pkplQRP3JIbj2lu2U/openapi.json