arXiv Scraper - Papers, Authors, Abstracts & PDFs

Disclaimer: This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by arXiv (Cornell University) or any of its subsidiaries. All trademarks mentioned are the property of their respective owners. "arXiv" is referenced only to describe the publicly available website this Actor collects data from.

Search arXiv and get clean, structured rows for every paper: title, full abstract, authors, categories, publication date, DOI and journal reference where available, and a direct PDF link. Search by keyword (or advanced field syntax), or fetch exact papers by arXiv ID. Built on the official arXiv API - no key, no browser.

What you get per paper

Field	Meaning
title / summary	Title and the full abstract
authors / authorCount	Author list and count
primaryCategory / categories	e.g. cs.AI, cs.LG, stat.ML
published / updated	First submission and last revision dates
pdfUrl / arxivUrl	Direct PDF and the abstract page
doi / journalRef / comment	DOI and journal reference once published; author comment

Input

{ "searchQueries": ["large language models"], "maxItemsPerQuery": 100, "sortBy": "submitted" }

• Search queries - one per line. Plain terms work, and so does arXiv field syntax: cat:cs.AI, au:hinton, ti:transformer, abs:diffusion.
• arXiv IDs - or fetch specific papers by ID (2312.00752).
• Import from a file - paste a whole list, or link a public .txt/.csv, a Google Sheet/Drive link, or an Apify key-value-store record.
• Sort by relevance, newest submitted, or recently updated. Max papers per query / total bound the run.
• Output fields - tick only the columns you want.

Leave everything empty and the run returns a small free sample so you can see the shape first.

Reliability

Reads the official arXiv API - a public Atom feed, no key, no proxy, no anti-bot. The actor paces its requests to stay within arXiv's guidance. A run that returns nothing bills $0.

Honest limits

• doi and journalRef are only present once a preprint is formally published in a journal - most arXiv preprints have neither, which is the true state of the record, not a gap.
• Search covers arXiv's index, not the full text of the PDFs; you get the abstract and metadata plus the PDF link to download separately.

Pricing

$0.002 per paper on the FREE tier (tiered pricing lowers it with volume). Only papers actually saved are billed; an empty run costs nothing.

Output views

• Papers - title, authors, category, date, DOI and the PDF/abstract links.

Use with AI assistants (MCP)

Available through the Apify MCP server - an agent can pull the latest papers on a topic (with abstracts and PDF links) for a literature review or RAG pipeline.

Agent-ready: autonomous payments (x402 & Skyfire)

This actor is agent-ready - AI agents can discover it, run it, and pay for it autonomously, with no Apify account and no human in the loop. It uses pay-per-event pricing and limited permissions, so it qualifies for Apify's agentic-payment standards:

• x402 - an open, HTTP-native payment protocol. Agents pay per run in USDC on the Base network directly through the Apify MCP server - no account, no API key.
• Skyfire - agent-to-service payments for fully autonomous AI-agent workflows.

Building an AI agent, MCP tool, or autonomous data pipeline? This scraper is ready to plug in and pay as it goes.

Automate & schedule

Run this Actor on autopilot and pull results into your own stack:

• Apify API - start runs, fetch datasets and manage schedules over REST.
• apify-client for JavaScript and apify-client for Python - official SDKs.
• Schedules - run it hourly, daily or weekly and keep your dataset current.
• Webhooks - trigger downstream actions (CRM import, Slack alert, email sequence) the moment a run finishes.

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'MY_APIFY_TOKEN' });

const run = await client.actor('scrapesage/arxiv-scraper').call({
    "searchQueries": [
        "large language models"
    ],
    "maxItemsPerQuery": 25
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Got ${items.length} records`);

Integrate with any app

Connect the dataset to thousands of apps - no code required:

• Make - multi-step automation scenarios.
• Zapier - push new records straight into your CRM or spreadsheet.
• Slack - get notified when a scheduled run finds something new.
• Google Drive / Sheets - auto-export every run to a spreadsheet.
• Airbyte - pipe results into your data warehouse.
• GitHub - trigger runs from commits or releases.

FAQ

How is this Actor billed? Pay-per-event: you pay only for the results it delivers, with no monthly rental and no start fee. The per-event price is shown on the Pricing tab.

Can I schedule it and get results automatically? Yes - create a Schedule and add a webhook or an integration (Google Sheets, Slack, Make, Zapier) to push each run's dataset wherever you need it.

Which export formats are available? Every run's dataset can be downloaded as JSON, CSV, Excel (XLSX), XML, HTML or RSS from the Apify Console or the API.

Can I run it from code or an AI agent? Yes - through the Apify API and client libraries, or from Claude, ChatGPT and other assistants via the Apify MCP server.

Is it legal to scrape arXiv? This Actor collects publicly available data only. You are responsible for using the output in compliance with applicable laws (including data-protection law where personal data is involved) and the source's terms. See the Disclaimer below.

Is this an official arXiv tool? No. It is an independent, third-party Actor with no affiliation to, endorsement by or sponsorship from arXiv (Cornell University). See the Disclaimer below.

Disclaimer

This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by arXiv (Cornell University) or any of its subsidiaries. All trademarks mentioned are the property of their respective owners.

"arXiv" and any related marks are the property of their respective owners and are used here only in a descriptive, nominative sense - to identify the publicly accessible website from which this Actor collects data. This Actor is not an official arXiv product, is not authorised or certified by arXiv (Cornell University), and does not distribute arXiv software. It collects only publicly available information; you are responsible for ensuring your use of that data complies with applicable laws, regulations and the terms of the source website.

Need help?

Open an issue on the Actor's Issues tab, or visit the Apify help center. Feature requests are welcome - this Actor is actively maintained.