# Blog Scraper (`naive_zing/blog-scraper`) Actor

Company Blog Scraper, Blog Post Scraper, Corporate Blog Crawler, Automatic Blog Discovery, Blog Content Extractor, Article Metadata Scraper, Multi-Domain Blog Scraper, Competitor Blog Analysis, Content Marketing Scraper, Blog Post Metadata Extraction, Company Announcements Scraper.

- **URL**: https://apify.com/naive\_zing/blog-scraper.md
- **Developed by:** [Wyald](https://apify.com/naive_zing) (community)
- **Categories:** Automation, Developer tools
- **Stats:** 38 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $33.00 / 1,000 standard-fetches

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Blog Scraper

A robust Apify Actor designed to scrape blog posts from company websites. Given a list of company domains and a maximum number of posts to fetch, this scraper automatically discovers blog sections, extracts blog posts, and collects comprehensive content and metadata.

#### Targeted Keywords

- **Primary**: Blog Scraper, Content Extraction, Company Blog Crawler, Article Scraper
- **Secondary**: Blog Post Metadata, Content Marketing Analysis, Blog Content Aggregation, Corporate Blog Mining

### Features

✅ **Automatic Blog Discovery**: Intelligently finds blog sections on company websites
✅ **Smart Content Extraction**: Extracts comprehensive blog post data including:
\*   Title
\*   Author
\*   Publication date
\*   Full article content
\*   Excerpt/summary
\*   Tags
\*   Category
\*   URL
✅ **Configurable Limits**: Set maximum number of posts per domain (up to 50)
✅ **Multiple Domain Support**: Scrape from multiple company websites in a single run
✅ **Structured Output**: Returns clean JSON data with all metadata
✅ **Fast & Lightweight**: Uses `crawlee` with BeautifulSoup for efficient HTTP-based scraping (no headless browser overhead)

### Input

| Field | Type | Description | Required | Default |
|---|---|---|---|---|
| `company_urls` | Array | List of company domain URLs or homepage URLs to scrape (e.g., `["https://stripe.com/", "shopify.com"]`). | Yes | - |
| `max_blogposts_to_fetch` | Number | Maximum number of blog posts to fetch per domain (1-50) | No | 10 |
| `max_concurrency` | Number | Number of concurrent requests | No | 2 |

#### Input Example

```json
{
    "company_urls": [
        "https://www.stripe.com/",
        "https://shopify.com/",
        "https://ai-bees.io/"
    ],
    "max_blogposts_to_fetch": 10,
    "max_concurrency": 2
}
```

### Output Example

```json
{
  "url": "https://www.stripe.com/blog/example-post",
  "domain": "www.stripe.com",
  "post_title": "How we scaled our payment infrastructure",
  "author": "Jane Doe",
  "published_date": "2024-01-15",
  "content": "Full article content here...",
  "excerpt": "Learn how we scaled our payment infrastructure to handle millions of transactions...",
  "tags": ["engineering", "infrastructure", "scaling"],
  "category": "Engineering",
  "scraped_at": "2024-01-20T10:30:00.000Z"
}
```

### How It Works

1. **Domain Analysis**: The scraper starts by visiting each provided company domain
2. **Blog Detection**: It automatically searches for blog sections using common patterns (/blog, /news, /articles, etc.)
3. **Post Discovery**: Once in the blog section, it identifies individual blog post URLs
4. **Content Extraction**: For each post, it extracts:
   - Structured metadata (title, author, date)
   - Full article content
   - Additional metadata (tags, categories)
5. **Limit Enforcement**: Respects the `number_of_blog_posts_to_fetch` limit per domain

### Usage Tips

- **URL Format**: You can provide URLs with or without `https://` - the scraper will normalize them
- **Rate Limiting**: The scraper includes automatic delays to be respectful to target websites
- **Post Limits**: Maximum 50 posts per domain to prevent excessive scraping
- **Concurrency**: Adjust `max_concurrency` based on target website capacity (default: 2)

### Use Cases

- **Content Marketing Analysis**: Analyze competitor blog strategies
- **Content Aggregation**: Collect blog content for research or analysis
- **Market Intelligence**: Monitor company announcements and thought leadership
- **SEO Research**: Study content patterns and topics from successful blogs
- **Training Data**: Collect blog content for ML/AI model training

### Notes

- The scraper respects robots.txt and includes reasonable delays between requests
- Blog structure varies by website - extraction quality depends on site structure
- Some blogs may require authentication or have anti-scraping measures
- Always ensure you have permission to scrape the target websites

# Actor input Schema

## `company_urls` (type: `array`):

List of company domain URLs or homepage URLs to scrape for blog posts. Can be simple strings (e.g., 'stripe.com') or full URLs (e.g., 'https://www.stripe.com/').

## `max_blogposts_to_fetch` (type: `integer`):

Maximum number of blog posts to fetch per domain. Must be between 1 and 50.

## `max_concurrency` (type: `integer`):

Number of concurrent requests. Higher values speed up scraping but may trigger rate limiting.

## Actor input object example

```json
{
  "company_urls": [
    "https://www.stripe.com/",
    "https://www.shopify.com/",
    "https://ai-bees.io/"
  ],
  "max_blogposts_to_fetch": 10,
  "max_concurrency": 4
}
```

# Actor output Schema

## `dataset` (type: `string`):

The dataset containing all scraped blog posts.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "company_urls": [
        "https://www.stripe.com/",
        "https://www.shopify.com/",
        "https://ai-bees.io/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("naive_zing/blog-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "company_urls": [
        "https://www.stripe.com/",
        "https://www.shopify.com/",
        "https://ai-bees.io/",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("naive_zing/blog-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "company_urls": [
    "https://www.stripe.com/",
    "https://www.shopify.com/",
    "https://ai-bees.io/"
  ]
}' |
apify call naive_zing/blog-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,naive_zing/blog-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/FIA8XPgf4etbGFjEy/builds/LQndUu39cqi4Dr2iw/openapi.json