# Unpaywall Article Metadata Scraper (`parseforge/unpaywall-scraper`) Actor

Scrape scholarly article metadata from the Unpaywall database by title search. Returns DOI, journal, publisher, and open-access status per article. No API key required.

- **URL**: https://apify.com/parseforge/unpaywall-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Other
- **Stats:** 9 total users, 0 monthly users, 90.9% runs succeeded, 1 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.66 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Unpaywall Article Metadata Scraper

**Scrape open-access and closed-access scholarly articles from the Unpaywall database by search query, up to a million per run.** Every article returns its DOI, title, journal, publisher, and open-access status. No API key needed. Export to CSV, JSON, Excel, or XML.

Finding free full-text versions of scholarly articles usually means checking each publisher site or using a browser extension one paper at a time. This Actor queries the Unpaywall database directly by article title, returning a flat row per match with the DOI, journal, publisher, and whether the article is open access. No registration or API token required.

| Who uses it | What they scrape Unpaywall for |
|---|---|
| Academic researchers | Quickly check if a set of papers they cite or review are freely available. |
| Librarians | Audit the open-access coverage of their institution's publication list. |
| Systematic reviewers | Filter a large set of candidate articles by open-access status before full-text retrieval. |
| Science journalists | Verify whether a study behind a paywall has a free preprint or open-access version. |

### What it does

This Actor searches the Unpaywall database by article title and returns each matching article as a flat row with its DOI, journal, publisher, and open-access status.

- 🔍 **Title search:** Enter one or more search terms; the Actor finds articles whose titles contain all of them.
- 📄 **Open Access filter:** Choose to see all articles, only open-access ones, or only closed-access ones.
- 📊 **Scalable collection:** Set a maximum from 1 to 1,000,000 articles per run.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Unpaywall data

**📈 Audit institutional open-access compliance.**

A university library runs the Actor on a list of recent faculty publications to see which ones are freely available and which need deposit in the institutional repository.

**🔍 Pre-screen articles for a systematic review.**

A research team searches by topic keywords, filters for open-access only, and exports the DOIs to a reference manager for full-text screening.

**📊 Monitor open-access trends in a field.**

A science policy analyst runs periodic searches on a set of keywords and tracks the proportion of open-access articles over time.

**📚 Verify free access to cited references.**

A journalist or fact-checker pastes the titles of cited studies into the Actor to confirm whether readers can access the original sources without a paywall.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API key** | Unpaywall's public API is used directly; no registration or token needed. |
| **Open Access status** | Each result includes a clear yes/no on whether the article is free to read. |
| **Flat schema** | Every article returns the same fields: DOI, title, journal, publisher, and more. |
| **Export ready** | Download as CSV, JSON, Excel, or XML for analysis in any tool. |

### How it compares

No other Store actor targets Unpaywall the same way, so the honest comparison is with the alternatives teams actually weigh.

| | Unpaywall Article Metadata Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When Unpaywall changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor from a search query and an optional open-access filter. The query runs against the Unpaywall database and the filter is applied as results are collected, so only matching articles reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "query": "software",
  "maxItems": 10,
  "is_oa": "any"
}
```

A larger pull:

```json
{
  "query": "software",
  "maxItems": 200,
  "is_oa": "any"
}
```

### Pricing

Pay-per-result: **$0.065 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $6.50 |
| 1,000 results | $65.00 |
| 10,000 results | $650.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Unpaywall Article Metadata Scraper](https://apify.com/parseforge/unpaywall-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Unpaywall through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/unpaywall-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that your search terms are present in article titles. Unpaywall requires an exact match of all words in the title. Try a broader or shorter query.

**The run is taking a long time.**

Large queries can take time because the Actor paces requests to respect Unpaywall's rate limits. Reduce the maximum articles setting for a faster test run.

**Some articles show 'Closed Access' even though I know they are free.**

Unpaywall's database may not have indexed every free version. The status reflects what Unpaywall knows at the time of the query.

**Can I search for a DOI directly?**

No. The input only accepts title search terms. If you have a DOI, you can look it up separately on the Unpaywall website.

**The output has fewer articles than I expected.**

Unpaywall's index covers a large but not complete set of articles. Also, the title search requires all words to match. Try using fewer or more common words.

### FAQ

| Question | Answer |
|---|---|
| What is Unpaywall? | Unpaywall is a database of over 50 million scholarly articles, indexed by DOI, that tracks whether each article is freely available from a legal source such as a repository or publisher. |
| Do I need an API key? | No. The Actor uses Unpaywall's public API endpoints, which do not require authentication for basic queries. |
| How does the title search work? | The search term you enter must appear in the article title. Multiple words act as an AND filter: all words must be present in the title for a match. |
| What does 'Open Access Only' mean? | It filters results to articles that Unpaywall has identified as freely readable from a legal source, such as a preprint server, repository, or open-access journal. |
| Can I get the full text of the article? | No. The Actor returns metadata and the open-access status, but does not download or store the full article text. You can use the DOI to access the article separately. |
| What fields are returned per article? | Each row includes the DOI, title, journal name, publisher, open-access status, and several other metadata fields. The exact field list is shown in the sample output after a run. |
| Is there a rate limit? | Unpaywall's public API has rate limits, but the Actor handles them automatically by pacing requests. For very large runs, it may take longer but will complete. |
| Can I search by author or journal? | No. The current input only supports searching by article title. Author and journal search are not available. |
| What is the maximum number of articles I can collect? | You can set a maximum up to 1,000,000 articles per run. The actual number collected depends on how many articles match your query. |
| Can I export the data to Excel? | Yes. The Actor supports export to CSV, JSON, Excel, and XML formats from the Apify dataset. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by OurResearch. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `query` (type: `string`):

Search terms to find articles by title in the Unpaywall database. The title must contain all search terms to match.

## `maxItems` (type: `integer`):

Maximum number of articles to collect per run.

## `is_oa` (type: `string`):

Filter articles by open access status. 'Any' shows all articles, 'Open Access Only' shows only free-to-read articles, 'Closed Access Only' shows only non-open access articles.

## Actor input object example

```json
{
  "query": "software",
  "maxItems": 10,
  "is_oa": "any"
}
```

# Actor output Schema

## `articles` (type: `string`):

Complete dataset with all scraped articles including open access status, bibliographic information, citation data, and direct links

## `overview` (type: `string`):

Overview view of articles with key fields displayed in a table format

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "software",
    "maxItems": 10,
    "is_oa": "any"
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/unpaywall-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "software",
    "maxItems": 10,
    "is_oa": "any",
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/unpaywall-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "software",
  "maxItems": 10,
  "is_oa": "any"
}' |
apify call parseforge/unpaywall-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/unpaywall-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/77h8k5FJ3W628TFfg/builds/aPjZecJnJx8gHozhg/openapi.json