# OSF Projects Scraper (`parseforge/osf-projects-scraper`) Actor

Scrapes public OSF project listings by search query or research category and returns each project as a flat row with title, description, contributors, and tags.

- **URL**: https://apify.com/parseforge/osf-projects-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Automation, Integrations, Education
- **Stats:** 2 total users, 1 monthly users, 90.3% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $7.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### OSF Projects Scraper

**Scrape OSF project metadata by search term or category, up to a million per run.** Each project comes with its title, description, contributors, tags, and date created. No API key required. Export to CSV, JSON, Excel, or XML.

The OSF (Open Science Framework) is a public research repository, but browsing it manually page by page is slow when you need a structured list of projects. This actor reads the public project listings directly, filtered by a free-text query or a predefined research category, and returns each match in one fixed schema.

| Who uses it | What they scrape OSF for |
|---|---|
| Research analysts | Discover which labs are working on a specific methodology this quarter. |
| Librarians | Build a catalog of open-access projects in a discipline for their institution. |
| Grant officers | Find preliminary data and active projects before issuing a call for proposals. |
| Meta-scientists | Collect a corpus of pre-registrations to study research practices over time. |

### What it does

This Actor collects OSF project records by search term or category and returns each one as a flat row.

- 🔍 **Free-text search:** supply a query and the actor returns matching projects, same as typing into the OSF search bar.
- 📂 **Category filter:** restrict results to a single research phase, such as Data, Analysis, or Hypothesis.
- 📊 **Flat structured output:** every project is a row with a consistent schema, ready for pivot tables or a database.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with OSF data

**📈 Monitor a research field.**

A research analyst runs the actor weekly with a query like "gut microbiome" to track new pre-registrations and open datasets in that niche.

**🗂️ Populate an institutional repository.**

A university librarian scrapes all projects tagged with their institution's name to create a searchable index of faculty output.

**📋 Audit pre-registrations.**

A meta-scientist collects every project in the Hypothesis category for a given year to compare planned analyses with published papers.

**🔬 Find potential collaborators.**

A principal investigator searches for a novel technique to identify labs that have already built the relevant instrumentation.

### Why choose this scraper

|  | What you get |
|---|---|
| **No API key** | Reads the public OSF project feed without registration or OAuth. |
| **Fixed schema** | Every project lands in your dataset with the same columns, no matter the category. |
| **Bulk collection** | Collect up to a million projects in a single run for large-scale meta-research. |

### How it compares

No other Store actor targets OSF the same way, so the honest comparison is with the alternatives teams actually weigh.

| | OSF Projects Scraper | Build it in-house | By hand |
|---|---|---|---|
| Setup | Run it now, zero config | Days of engineering | None, but hours per pull |
| When OSF changes | Maintained for you | You fix it | You re-learn the page |
| Proxies, retries, anti-bot | Built in | Your problem | Browser only |
| Output | Fixed JSON schema, CSV/Excel export | Whatever you build | Copy-paste |
| Cost | Pay per result | Engineering time | Analyst hours |

### Configure the run

Drive the Actor with a search term, a category, or both together, and set a ceiling on the number of projects collected. The Input tab lists every parameter.

A first run with the defaults:

```json
{
  "maxItems": 10
}
```

A larger pull:

```json
{
  "maxItems": 200
}
```

### Pricing

Pay-per-result: **$0.0085 per result** collected. You pay only for the results written to your dataset.

| Results collected | Approximate cost |
|---|---|
| 100 results | $0.85 |
| 1,000 results | $8.50 |
| 10,000 results | $85.00 |

New Apify accounts start with $5 in free credit.

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 results per run.

### Run it

1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [OSF Projects Scraper](https://apify.com/parseforge/osf-projects-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to OSF through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com/?tools=parseforge/osf-projects-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that your search term is spelled correctly and is not too narrow. Try running the same query directly on osf.io to confirm that projects exist. Also verify that the category filter is not excluding all matches.

**The actor stopped before reaching my maxItems.**

This happens when OSF has no more projects matching your filters. The actor returns every publicly available match. Try broadening your search term or setting the category to "Any".

**Some projects seem to be missing from the results.**

Only public projects are returned. Private, draft, or embargoed projects are not accessible. Also, OSF search indexing may have a slight delay for very recently created projects.

**The run is taking a long time.**

Large maxItems values mean the actor must follow many pages of results. Reduce the maximum or narrow your filters to speed up the run.

**I see an error about the website being unreachable.**

OSF may be experiencing downtime. Wait a few minutes and retry the run. If the problem persists, check status.osf.io for incident reports.

### FAQ

| Question | Answer |
|---|---|
| Do I need an OSF account or API token? | No. This actor reads the public project listings that are visible to anyone on the web, so no login or token is required. |
| What is the maximum number of projects I can scrape? | You can set the maximum up to 1,000,000 projects per run. The actual number returned depends on how many projects match your filters on OSF. |
| Can I scrape private or embargoed OSF projects? | No. This actor only accesses public project metadata. Private, draft, and embargoed projects are not visible in the public feeds. |
| What does the category filter do? | It restricts results to a single OSF category, such as Data, Analysis, or Hypothesis. Leave it on "Any" to search across all categories. |
| How does the search query work? | The query is a free-text term that OSF matches against project titles, descriptions, and tags. It works the same way as typing into the search box on osf.io. |
| Can I combine a search term and a category? | Yes. When both are provided, the actor returns only projects that match the search term and belong to the selected category. |
| What export formats are supported? | You can export your dataset in CSV, JSON, Excel, or XML from the Apify platform. |
| Does this actor follow pagination? | Yes. It automatically follows the paginated project listings on OSF until it reaches your requested maximum or the end of the results. |
| Can I schedule this actor to run automatically? | Yes. Apify supports scheduled runs, so you can set this actor to collect new OSF projects daily, weekly, or on any cron schedule. |
| What data fields does each project row contain? | Each row includes the project title, description, contributors, tags, date created, and other metadata that OSF exposes publicly. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Center for Open Science. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

How many projects to collect per run.

## `query` (type: `string`):

Optional free-text query.

## `category` (type: `string`):

Filter by OSF category.

## Actor input object example

```json
{
  "maxItems": 10,
  "category": ""
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "query": ""
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/osf-projects-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "query": "",
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/osf-projects-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "query": ""
}' |
apify call parseforge/osf-projects-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/osf-projects-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/rcE0tpUeHQh4kQdj9/builds/SyOMnqBdBu0e4HS53/openapi.json