# Google Scholar Paper and Author Data Scraper (`khadinakbar/google-scholar-scraper`) Actor

Search Google Scholar and extract papers, citation formats, author profiles, publication lists, citation histories, and co-author relationships. Receive structured scholarly records with direct or SerpApi source provenance.

- **URL**: https://apify.com/khadinakbar/google-scholar-scraper.md
- **Developed by:** [Khadin Akbar](https://apify.com/khadinakbar) (community)
- **Categories:** Developer tools, AI, Agents
- **Stats:** 7 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 result returneds

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google Scholar Paper and Author Data Scraper

Search Google Scholar and extract structured papers, citation formats, author profiles, publication lists, citation histories, and co-author relationships. The Actor combines a direct Camoufox research path with managed or bring-your-own SerpApi routing and returns provenance with each record.

Use it for literature discovery, bibliometric analysis, researcher enrichment, citation workflows, knowledge graphs, and AI research agents.

### Best fit

- Researchers assembling literature-review candidates around a topic.
- Academic teams organizing author profiles and publication portfolios.
- Data analysts studying citation history and co-author networks.
- Reference managers collecting formatted citations and export links.
- AI agents retrieving scholarly evidence for downstream synthesis.

### A practical research scenario

A research analyst begins with a Scholar search for transformer architecture and narrows the result set by year. Each paper provides a `resultId` plus author identifiers where Scholar exposes them. The analyst can route the paper IDs into citation mode and the author IDs into profile, publication, citation-history, or co-author modes. The result is a connected evidence set that moves from topic discovery to reference formatting and researcher context.

This workflow keeps every step explicit and lets an agent choose the smallest mode needed for the current question.

### Modes

| Mode | Input | Result |
| --- | --- | --- |
| `search` | `queries` | Papers, authors, venues, years, citation counts, links, and result IDs. |
| `cite` | `resultIds` | Formatted citation strings and bibliographic export links. |
| `author_profile` | `authorIds` | Affiliation, interests, citation totals, h-index, and i10-index. |
| `author_articles` | `authorIds` | Publications associated with each Scholar author. |
| `author_citation` | `authorIds` | Citation totals organized by year. |
| `author_co_authors` | `authorIds` | Co-author names, IDs, and relationship records. |

### Quick start

#### Search for papers

```json
{
  "mode": "search",
  "queries": ["transformer architecture"],
  "yearFrom": 2020,
  "maxResults": 20,
  "resultsLanguage": "en"
}
```

#### Read an author profile

```json
{
  "mode": "author_profile",
  "authorIds": ["LSsXyncAAAAJ"]
}
```

#### Collect citation formats

```json
{
  "mode": "cite",
  "resultIds": ["u-CT435A0vkJ"]
}
```

Citation mode uses the configured SerpApi route. The Actor includes a managed key, and the optional `serpApiKey` field is marked secret for teams that prefer their own provider account.

### Input reference

| Field | Purpose |
| --- | --- |
| `mode` | Selects the Scholar operation. |
| `queries` | Search phrases for paper discovery. |
| `resultIds` | Scholar paper or cluster IDs for citation export. |
| `authorIds` | Scholar author identifiers for author modes. |
| `maxResults` | Upper bound for records per target. |
| `yearFrom`, `yearTo` | Narrows paper searches by publication year. |
| `sortByDate` | Requests date-oriented paper ordering. |
| `includePatents`, `includeCaseLaw` | Selects additional Scholar document surfaces. |
| `reviewArticlesOnly` | Focuses search on review literature. |
| `languageRestrict` | Applies a Scholar language restriction. |
| `authorSort` | Orders author publications by relevance, date, or title. |
| `resultsLanguage` | Selects the interface and returned language context. |
| `forceSerpApi` | Routes supported work directly through SerpApi. |
| `serpApiKey` | Optional secret-marked bring-your-own provider key. |
| `proxyConfiguration` | Controls direct Scholar browsing sessions. |

### Output data

Paper records can include `title`, `authors`, `publicationInfo`, `year`, `snippet`, `citedByCount`, `resultId`, article links, PDF links, version counts, provenance, and capture time.

Author modes can add `name`, `authorId`, affiliation, interests, profile image, citation totals, h-index, i10-index, year-by-year citation data, publication records, and co-author edges. Citation mode can add formatted citations and export links.

```json
{
  "mode": "search",
  "query": "transformer architecture",
  "position": 1,
  "title": "Attention Is All You Need",
  "resultId": "u-CT435A0vkJ",
  "year": 2017,
  "authors": [
    {
      "name": "A Vaswani",
      "authorId": "author-id"
    }
  ],
  "source": "serpapi"
}
```

The `source` field identifies whether a result came from the direct Camoufox path or SerpApi routing.

### AI agent workflows

The mode-specific contract supports compact tool calls through Apify MCP and direct API integrations.

Example agent request:

> Find papers about retrieval-augmented generation, keep recent publications, return paper IDs and author IDs, then collect citation formats for the selected papers.

Useful routing guidance:

- Start with `search` when the agent has a topic or research question.
- Use `cite` when the workflow already has Scholar result IDs.
- Use `author_profile` for researcher metrics and identity context.
- Use `author_articles` for publication discovery around a known author.
- Use `author_citation` and `author_co_authors` for bibliometric graphs.
- Preserve `source` and `resultId` in downstream evidence records.

### Run through the API

```bash
curl -X POST "https://api.apify.com/v2/acts/khadinakbar~google-scholar-scraper/runs" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "search",
    "queries": ["retrieval augmented generation"],
    "maxResults": 20,
    "resultsLanguage": "en"
  }'
```

The Apify token stays in the `Authorization` header. Provider credentials supplied through the Console remain in the secret-marked input field.

### Data sourcing and recovery

The primary research path uses Camoufox browser sessions with residential routing and parses Scholar paper and author pages directly. When Scholar presents its traffic-verification surface, the Actor can continue the same target through the configured SerpApi route. Citation export uses SerpApi because that workflow maps cleanly to the provider's structured citation endpoint.

Managed and bring-your-own provider results share the same dataset contract, caps, and billing flow. Record provenance remains visible through `source`.

### Pricing

This Actor uses Pay per event pricing with platform usage passed through. Result events cover paper, citation, citation-history, and co-author records, while author profiles have their own event. Treat the live Pricing tab as the current source of truth for event prices and billing details.

Use `maxResults`, focused queries, and year filters to align each run with the research question.

### Best results

- Begin with a focused query and a practical result cap.
- Use year and document filters when the research brief has a defined scope.
- Chain `resultId` and `authorId` fields into the corresponding enrichment modes.
- Use direct SerpApi routing for time-sensitive research batches and citation export.
- Retain provenance fields when combining Scholar data with external evidence.

### Related Actors

- Use [Google Patents Scraper](https://apify.com/khadinakbar/google-patents-scraper) when scholarly research should connect to patents, assignees, inventors, and patent citations.
- Use [Google SERP Scraper](https://apify.com/khadinakbar/scrape-google-serp) when the research brief expands from academic literature to the broader web.

### Builder's note

I designed the six modes as a connected research path rather than unrelated endpoints. Search records expose the IDs needed by citation and author workflows, while the `source` field keeps direct and provider-backed evidence easy to audit in a combined dataset.

### Responsible use

This Actor collects publicly available scholarly metadata. Use the results for legitimate research, indexing, and analysis in line with applicable laws, source terms, citation practices, and your organization's data-governance requirements.

# Actor input Schema

## `mode` (type: `string`):

Which Google Scholar operation to run. 'search' finds papers by keyword; 'cite' exports citation formats for a paper; 'author\_profile' returns one author's metrics; 'author\_articles' lists an author's publications; 'author\_citation' returns an author's year-by-year citation history; 'author\_co\_authors' returns an author's co-author network. Defaults to 'search'. Each mode reads different fields below.

## `queries` (type: `array`):

Keyword queries to run on Google Scholar, one search per entry (e.g. 'transformer architecture'). Supports Scholar operators like author:hinton or source:nature. Used only when mode is 'search'. NOT a Scholar URL and NOT an author ID — use 'authorIds' for author modes.

## `resultIds` (type: `array`):

Google Scholar result IDs (cluster IDs) to fetch citation-export formats for, one per entry. Get these from the 'resultId' field of a prior 'search' run (e.g. 'TY8gM2sAAAAJ'). Used only when mode is 'cite'. NOT a paper title or URL.

## `authorIds` (type: `array`):

Google Scholar author IDs to look up, one per entry (e.g. 'LSsXyncAAAAJ'). Find an ID in a Scholar profile URL (the 'user=' value) or in the 'authors\[].authorId' field of a 'search' run. Used by author\_profile, author\_articles, author\_citation, and author\_co\_authors. NOT an author name.

## `maxResults` (type: `integer`):

Maximum papers to return per search query or per author publication list, and your hard cost cap (you are never charged for more). Applies to 'search' and 'author\_articles'. Defaults to 100. Author-profile, citation-history, and cite modes ignore this (they return a single composite record per target).

## `yearFrom` (type: `integer`):

Earliest publication year to include in results (e.g. 2018). Applies only to mode 'search'. Leave empty for no lower bound. Pair with 'yearTo' for a closed range.

## `yearTo` (type: `integer`):

Latest publication year to include in results (e.g. 2024). Applies only to mode 'search'. Leave empty for no upper bound. Pair with 'yearFrom' for a closed range.

## `sortByDate` (type: `boolean`):

When true, sorts search results by newest first instead of by relevance. Applies only to mode 'search'. Defaults to false (relevance). Useful for tracking the latest publications on a topic.

## `includePatents` (type: `boolean`):

When true, includes patents in search results. Applies only to mode 'search'. Defaults to false (patents excluded). Mutually exclusive with 'includeCaseLaw' — case law takes priority if both are set.

## `includeCaseLaw` (type: `boolean`):

When true, searches US court case law instead of articles. Applies only to mode 'search'. Defaults to false. Overrides 'includePatents' when both are enabled.

## `reviewArticlesOnly` (type: `boolean`):

When true, restricts search results to review articles only. Applies only to mode 'search'. Defaults to false (all article types). Useful for systematic reviews and meta-analyses.

## `languageRestrict` (type: `string`):

Restrict search results to specific languages using Scholar 'lang\_xx' codes joined by '|' (e.g. 'lang\_en' or 'lang\_en|lang\_de'). Applies only to mode 'search'. Leave empty for all languages. This is different from 'resultsLanguage', which sets the Scholar interface language.

## `authorSort` (type: `string`):

Order for an author's publication list: 'relevance' (citation count, default), 'date' (newest first), or 'title' (alphabetical). Applies only to mode 'author\_articles'. Other modes ignore this field.

## `resultsLanguage` (type: `string`):

Two-letter language code for the Google Scholar interface and result display (e.g. 'en', 'de', 'es'). Applies to all modes. Defaults to 'en'. To filter results BY language instead, use 'languageRestrict'.

## `forceSerpApi` (type: `boolean`):

When true, skips the Camoufox direct scrape and goes straight to the SerpApi path (fastest, most reliable). Requires a SerpApi key (managed or BYOK). Defaults to false (try direct scrape first, fall back automatically). Turn on for time-sensitive or large jobs.

## `serpApiKey` (type: `string`):

Optional: your own SerpApi key to power the reliable fallback at the standard per-result price (you pay SerpApi directly for the upstream calls). Leave empty to use our managed fallback automatically when the direct scrape is blocked. Get a key at serpapi.com. Stored as a secret and never logged.

## `maxRetries` (type: `integer`):

How many times to retry a blocked direct-scrape request before falling back to SerpApi. Applies to the Camoufox path only. Defaults to 3. Higher values cost more compute for little gain on captcha-heavy targets.

## `proxyConfiguration` (type: `object`):

Proxy settings for the direct-scrape path. Residential proxies are strongly recommended — Google Scholar blocks datacenter IPs almost immediately. Defaults to Apify residential proxies. The SerpApi fallback path does not use this.

## Actor input object example

```json
{
  "mode": "search",
  "queries": [
    "large language models"
  ],
  "maxResults": 5,
  "sortByDate": false,
  "includePatents": false,
  "includeCaseLaw": false,
  "reviewArticlesOnly": false,
  "authorSort": "relevance",
  "resultsLanguage": "en",
  "forceSerpApi": true,
  "maxRetries": 3,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `usage` (type: `string`):

No description

## `output` (type: `string`):

Compact terminal outcome and persisted-row counters.

## `runSummary` (type: `string`):

Detailed terminal diagnostics, coverage, storage, and charge counters.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "large language models"
    ],
    "maxResults": 5,
    "forceSerpApi": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("khadinakbar/google-scholar-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["large language models"],
    "maxResults": 5,
    "forceSerpApi": True,
}

# Run the Actor and wait for it to finish
run = client.actor("khadinakbar/google-scholar-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "large language models"
  ],
  "maxResults": 5,
  "forceSerpApi": true
}' |
apify call khadinakbar/google-scholar-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,khadinakbar/google-scholar-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wdyrrYNDOmgQkqZUq/builds/coceS2c3nl0n9y8ZT/openapi.json