# OSF Open Science Framework Scraper (`parseforge/osf-scraper`) Actor

Scrapes public OSF research projects, preprints, and registrations by keyword, provider, or subject. Returns each record as a flat row with title, authors, DOI, and metadata.

- **URL**: https://apify.com/parseforge/osf-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Other, Education, Automation
- **Stats:** 2 total users, 0 monthly users, 89.7% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $11.24 / 1,000 result items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### OSF Open Science Framework Scraper

**Scrape public research projects, preprints, and registrations from the Open Science Framework, up to a million per run.** Each record comes with its title, authors, DOI, subjects, and full metadata. No API key required. Export to CSV, JSON, Excel, or XML.

The OSF Open Science Framework hosts millions of open-access research outputs, but its API requires pagination handling and rate-limit management. This scraper reads the public feeds directly, filtered by resource type, keyword, preprint provider, or subject, and returns each match in one fixed schema.

| Who uses it | What they scrape OSF for |
|---|---|
| Meta-researchers | Finding replication studies and pre-registered designs across disciplines. |
| Librarians | Building institutional preprint collections by provider and subject. |
| Data journalists | Tracking the growth of open science practices over time. |
| Academic administrators | Monitoring faculty research outputs deposited in public repositories. |

### What it does

This Actor collects OSF research projects, preprints, or registrations by keyword, provider, or subject, and returns each one as a flat row.

- 📋 **Three resource modes:** collect research projects (nodes), preprints, or registrations with one selector.
- 🔍 **Keyword search:** filter by title, description, or tags with a free-text query.
- 🏷️ **Subject filtering:** narrow preprints and registrations by comma-separated subject tags like 'Psychology,Education'.
- 📦 **Provider targeting:** restrict preprints to a specific community provider such as psyarxiv, socarxiv, or engrxiv.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with OSF data

**📈 Monitor open science trends.**

A meta-researcher runs the scraper weekly with the keyword 'replication' in registrations mode to count new pre-registered replication attempts.

**📚 Build a subject repository.**

A librarian collects all PsyArXiv preprints tagged 'Clinical Psychology' to populate an institutional preprint portal.

**🔎 Discover pre-registered studies.**

A graduate student searches registrations by keyword to find existing pre-registrations before designing their own study.

**📊 Audit institutional output.**

A research administrator searches projects by their university name to track public deposits by faculty.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API key** | Reads public OSF feeds directly, no registration or token needed. |
| **Fixed schema** | Every record returns the same flat structure for easy analysis. |
| **Bulk collection** | Set a high maxItems value and collect up to a million records per run. |
| **Multi-format export** | Download your dataset as CSV, JSON, Excel, or XML. |

### How it compares

No other Store actor targets OSF the same way, so the honest comparison is with the alternatives teams actually weigh.

| | OSF Open Science Framework Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When OSF changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor with a keyword query and choose a resource mode, then optionally narrow results by preprint provider or subject tags. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "maxItems": 10,
  "mode": "preprints",
  "query": "replication study"
}
```

A larger pull:

```json
{
  "maxItems": 200,
  "mode": "preprints",
  "query": "replication study"
}
```

### Pricing

Pay-per-result: **$0.01499 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $1.50 |
| 1,000 results | $14.99 |
| 10,000 results | $149.90 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [OSF Open Science Framework Scraper](https://apify.com/parseforge/osf-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to OSF through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/osf-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that your keyword and filters match existing OSF content. Try a broader query or remove the provider and subject filters to see if results appear.

**The provider filter is not working.**

Ensure you are in preprints mode. The provider filter only applies to preprints. Also verify the provider slug is spelled correctly, e.g. 'psyarxiv' not 'PsyArXiv'.

**I set maxItems to 1000 but only got 200 records.**

The scraper stops when it has collected the requested number or when no more matching records exist. Your query likely returned only 200 matches.

**The run is taking a long time.**

Large maxItems values and broad queries can take time. Reduce maxItems or add filters to narrow the search and speed up collection.

### FAQ

| Question | Answer |
|---|---|
| Do I need an OSF account or API key? | No. The scraper reads the public OSF feeds, so no login, account, or API key is required. |
| What is the difference between nodes, preprints, and registrations? | Nodes are general research projects. Preprints are open-access papers shared before peer review. Registrations are time-stamped, pre-registered study designs. |
| Can I filter by a specific preprint provider? | Yes. In preprints mode, enter a provider slug like 'psyarxiv', 'socarxiv', or 'engrxiv' to restrict results to that community. |
| How do subject filters work? | Enter comma-separated subjects such as 'Psychology,Education'. The filter applies to preprints and registrations and matches against the OSF subject taxonomy. |
| How many records can I collect in one run? | You can set maxItems up to 1,000,000. The actual number collected depends on how many results match your query and filters. |
| What output formats are supported? | The dataset can be exported to CSV, JSON, Excel, or XML from the Apify platform. |
| Does this scraper get full-text PDFs? | No. It collects metadata such as title, authors, DOI, and subjects. Full-text files are not downloaded. |
| Can I search without a keyword? | Yes. Leave the keyword field empty to browse all records in the selected mode, optionally filtered by provider or subject. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Center for Open Science. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

How many projects to collect per run.

## `mode` (type: `string`):

Nodes: public research projects. Preprints: open-access preprint papers. Registrations: pre-registered studies.

## `query` (type: `string`):

Free text search. Filters the selected resource type by title, description, or tags.

## `provider` (type: `string`):

Preprints mode only. Provider slug (e.g. osf, psyarxiv, socarxiv, engrxiv, biohackrxiv). Leave empty for all providers.

## `subjects` (type: `string`):

Comma-separated subject filters (e.g. 'Psychology,Education'). Applies to preprints and registrations.

## Actor input object example

```json
{
  "maxItems": 10,
  "mode": "preprints",
  "query": "replication study"
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "mode": "preprints",
    "query": "replication study"
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/osf-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "mode": "preprints",
    "query": "replication study",
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/osf-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "mode": "preprints",
  "query": "replication study"
}' |
apify call parseforge/osf-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/osf-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/yVwybCitP7GoQtmte/builds/0ToMvYk4RsU4ptxiF/openapi.json