# OpenAlex Scholarly Works Scraper (`taroyamada/openalex-research-intelligence`) Actor

Extract publication, author, institution, source, citation, topic, DOI, and open-access signals from the official OpenAlex API.

- **URL**: https://apify.com/taroyamada/openalex-research-intelligence.md
- **Developed by:** [naoki anzai](https://apify.com/taroyamada) (community)
- **Categories:** Developer tools, Business, Automation
- **Stats:** 4 total users, 0 monthly users, 93.5% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$4.00 / 1,000 openalex research rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## OpenAlex Research Intelligence

Research and intelligence teams enter topics, works, authors, institutions, or concepts.
The Actor discovers and monitors official OpenAlex metadata, then emits only new work signals on recurring monitor runs.
Use the companion PubMed and NIH Actors for the buyer-facing report, citation export, and grant/funding context; this Actor's runtime and pricing remain row-based.

### Store Quickstart

Use a bounded scheduled watch to establish a free baseline and then receive only new work, author, and source rows on change. The first run stores the baseline; later changed runs are the alert step in the research workflow.

```json
{
  "searchTerms": ["retrieval augmented generation"],
  "fromDate": "2025-01-01",
  "sort": "publication_date:desc",
  "limitPerSource": 10,
  "maxWorks": 20,
  "monitor": true,
  "monitorKey": "rag-literature-watch",
  "initialRunMode": "baseline_only",
  "delivery": "dataset",
  "dryRun": false
}
```

The first baseline emits zero rows and zero charges. Repeat the same input on a schedule; unchanged runs remain zero-row/zero-charge, while new works produce structured change rows that can be passed to the report/export workflow below.

### Run the next report

- Turn broad discovery into a focused literature report and citation export with [PubMed Literature Watch & Research Report](https://apify.com/taroyamada/pubmed-research-intelligence).
- Compare the related award portfolio with [NIH RePORTER Funding Landscape Report](https://apify.com/taroyamada/nih-research-funding-landscape-report).

### Research cluster workflow

1. **Discover:** use OpenAlex search, author, institution, concept, or work filters to find a bounded research set.
2. **Alert:** enable `monitor: true` with a stable `monitorKey`; only unseen works are delivered after the baseline.
3. **Deliver:** send the resulting work IDs or search terms to the PubMed/NIH companion Actors when a report, citation export, or funding-output view is required.

OpenAlex does not fabricate a report, export, or synthetic alert row. Its existing runtime emits `work_summary`, `author_signal`, and `source_summary` rows; monitor mode makes those rows change-only. This keeps the Store description aligned with the deployed contract.

### Use cases

Use this actor for academic / bibliometric research — query 250M+ scholarly works by topic, author, institution, or citation. Auth-free, official-API-first, with a stable output schema and documented source compliance.

### Monitor mode - new-paper alerts (recurring)

Schedule this actor and get **only the works you haven't seen before** on each run - a stateful "new research" alert, not a full re-dump. Ideal for watching an **author, institution, or concept** for new publications.

- Set `monitor: true` and run on a schedule (Apify Schedules).
- The first run defaults to `initialRunMode: "baseline_only"`: it establishes state, emits **0 rows**, and makes **0 PPE charges**.
- Use `initialRunMode: "emit_backfill"` when you explicitly want the first baseline delivered as charged rows for a billing proof or initial export.
- Every later run emits **only new works** (plus their author/source rows). If the state is unchanged, it emits **0 rows** and makes **0 charges**.
- State is namespaced per query — watching several? Give each a distinct `monitorKey`.
- A state read/save failure fails closed: the actor does not charge rows or advance the cursor.

```json
{
  "institutionIds": ["I63966007"],
  "sort": "publication_date:desc",
  "monitor": true,
  "initialRunMode": "baseline_only",
  "delivery": "webhook",
  "webhookUrl": "https://your-endpoint.example/new-papers"
}
```

> Tip: entity queries (`authorIds` / `institutionIds` / `conceptIds`) are the most reliable for scheduled monitoring.

### Inputs

| Field | Default | Notes |
|---|---:|---|
| `searchTerms` | `[]` | Research queries for OpenAlex works. |
| `workIds` | `[]` | OpenAlex work IDs or URLs. |
| `authorIds` | `[]` | OpenAlex author IDs. |
| `institutionIds` | `[]` | OpenAlex institution IDs. |
| `conceptIds` | `[]` | OpenAlex concept IDs. |
| `fromDate` / `toDate` | empty | Publication date filters in `YYYY-MM-DD`. |
| `sort` | `cited_by_count:desc` | Citation, publication date, or relevance sort. |
| `limitPerSource` | `25` | Works fetched per search/filter source. |
| `maxWorks` | `100` | Global unique work cap. |
| `includeAbstract` | `false` | Reconstruct abstract text from OpenAlex inverted index when available. |
| `mailto` | empty | Optional polite-pool email parameter for OpenAlex. |
| `monitor` | `false` | **Monitor mode:** emit only works not seen on a prior run (stateful new-results alert). Pair with a schedule. |
| `monitorKey` | empty | Optional label to namespace monitor state across several watched queries. |
| `initialRunMode` | `baseline_only` | First monitor run: `baseline_only` records state with 0 rows/charges; `emit_backfill` delivers the current baseline as charged rows. |
| `delivery` | `dataset` | `dataset` or `webhook`. |
| `dryRun` | `false` | Skip dataset/webhook delivery. |

At least one of `searchTerms`, `workIds`, `authorIds`, `institutionIds`, or `conceptIds` is required.

### Dataset Rows and downstream deliverables

`work_summary`

- title, DOI, OpenAlex ID, publication year/date, type
- citation count, open-access status, retraction flag
- primary source, publisher, landing page, PDF URL
- topics, concepts, countries, institutions

`author_signal`

- work ID/title, author name and OpenAlex ID
- author order/position, corresponding author flag
- affiliated institutions and countries

`source_summary`

- work ID/title, journal or source name
- source type, ISSN, host organization, open-access source flag

The rows above are the deployed OpenAlex deliverables. A recurring monitor run is the alert step; report/export packaging is intentionally handled by the linked PubMed and NIH Actors so buyers receive source-specific outputs with the appropriate interpretation limits.

### Example Input

```json
{
  "searchTerms": ["large language models", "retrieval augmented generation"],
  "fromDate": "2024-01-01",
  "sort": "cited_by_count:desc",
  "limitPerSource": 10,
  "maxWorks": 20,
  "includeAbstract": false,
  "delivery": "dataset",
  "dryRun": false
}
```

### Sample output

See [`docs/sample-output.json`](docs/sample-output.json) for a representative `work_summary`, `author_signal`, and `source_summary` payload. Run the actor once with the Quickstart input to establish state before scheduling.

### Local Development

```bash
npm install
npm test
node src/index.js
```

`output/result.json` contains the full payload. Apify dataset delivery writes flattened rows.

### Limitations

- OpenAlex coverage is broad but not identical to Crossref, PubMed, Semantic Scholar, or publisher APIs.
- Citation counts and metadata can lag source publications.
- Relevance sort is used only for search sources; non-search filters fall back to citation sort.
- `includeAbstract` can increase payload size substantially.

### Monitoring example

See **Monitor mode** above to schedule new-paper alerts for an author, institution, or concept.

### Billing safety

- `monitor: false` preserves one-shot behavior and delivers each output row through the configured PAY\_PER\_EVENT event.
- `monitor: true` persists the seen-work cursor only after delivery and the `PHASE89_DELIVERY_AUDIT` receipt succeed.
- The runtime verifies the `chargedCount` returned by `Actor.pushData` for every billable row. Missing charge receipts fail closed.

# Actor input Schema

## `searchTerms` (type: `array`):

Research keywords to search in OpenAlex works.

## `workIds` (type: `array`):

OpenAlex work IDs or URLs such as W2741809807 or https://openalex.org/W2741809807.

## `authorIds` (type: `array`):

OpenAlex author IDs or URLs. Fetches works by author.

## `institutionIds` (type: `array`):

OpenAlex institution IDs or URLs. Fetches works associated with the institution.

## `conceptIds` (type: `array`):

OpenAlex concept IDs or URLs. Fetches works tagged to the concept.

## `fromDate` (type: `string`):

Optional publication date floor, YYYY-MM-DD.

## `toDate` (type: `string`):

Optional publication date ceiling, YYYY-MM-DD.

## `sort` (type: `string`):

Sort order for OpenAlex work results.

## `limitPerSource` (type: `integer`):

Maximum number of works to fetch for each source input.

## `maxWorks` (type: `integer`):

Global cap on unique works emitted by one run.

## `includeAbstract` (type: `boolean`):

When enabled, reconstruct abstract text from OpenAlex inverted indexes when available.

## `mailto` (type: `string`):

Optional email sent as OpenAlex mailto query parameter.

## `timeoutMs` (type: `integer`):

Per-request timeout in milliseconds.

## `delivery` (type: `string`):

Send flattened rows to the dataset or POST the full payload to a webhook.

## `webhookUrl` (type: `string`):

Required when delivery is webhook.

## `monitor` (type: `boolean`):

On each run, emit only works not seen on a previous run — a stateful new-results alert. Pair with a schedule for recurring monitoring. State persists across runs per query.

## `monitorKey` (type: `string`):

Optional label to namespace monitor state when watching several queries. Leave blank to derive it automatically from your query inputs.

## `initialRunMode` (type: `string`):

For monitor=true only: baseline\_only records the first snapshot with 0 rows and 0 charges; emit\_backfill emits the current baseline as charged rows. Later unchanged runs always emit 0 rows and 0 charges.

## `dryRun` (type: `boolean`):

Run without saving dataset rows or sending a webhook.

## Actor input object example

```json
{
  "searchTerms": [
    "large language models",
    "retrieval augmented generation"
  ],
  "sort": "cited_by_count:desc",
  "limitPerSource": 25,
  "maxWorks": 100,
  "includeAbstract": false,
  "timeoutMs": 20000,
  "delivery": "dataset",
  "monitor": false,
  "initialRunMode": "baseline_only",
  "dryRun": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchTerms": [
        "large language models",
        "retrieval augmented generation"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("taroyamada/openalex-research-intelligence").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "searchTerms": [
        "large language models",
        "retrieval augmented generation",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("taroyamada/openalex-research-intelligence").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchTerms": [
    "large language models",
    "retrieval augmented generation"
  ]
}' |
apify call taroyamada/openalex-research-intelligence --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,taroyamada/openalex-research-intelligence"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/SDSneHx5rQ9obAjyK/builds/1yoREt6DEdnTrjVRz/openapi.json