# GitHub Scraper — Repositories & User Profiles (`hipersoft/github-scraper`) Actor

Search GitHub and scrape repositories (stars, forks, language, topics, license, dates) or user profiles (name, company, location, bio, followers, public email, repos). Uses the GitHub REST API; optional token for higher rate limits. For developer lead generation, OSS research and recruiting.

- **URL**: https://apify.com/hipersoft/github-scraper.md
- **Developed by:** [hiper soft](https://apify.com/hipersoft) (community)
- **Categories:** Developer tools, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.0004 / item scraped

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## GitHub Scraper — Repositories & User Profiles, Stars, Topics & Leads

Search GitHub and scrape repositories (stars, forks, language, topics, license, dates) or user profiles (name, company, location, bio, followers, public email, repos) into clean, structured JSON. It uses the official GitHub REST API and works out of the box with no login — add an optional token for higher rate limits. Built for developer lead generation, open-source research and technical recruiting.

![GitHub Scraper input — mode, GitHub search query, max items, sort, order and enrich-profiles fields in the Apify Console](https://api.apify.com/v2/key-value-stores/SUfvnaFLd9z9eBCtV/records/github-scraper-input.png?v=2)

### Features

- 🔀 **Two modes** — search `repositories` or `users` with the same powerful GitHub query syntax.
- 🔎 **Full query power** — use qualifiers like `stars:>10000 language:typescript` or `location:berlin followers:>500`.
- ⭐ **Repo metrics** — `stars`, `forks`, `watchers`, `openIssues`, `language`, `topics`, `license` and activity dates.
- 👤 **Profile enrichment** — optionally fetch each user's full profile for `email`, `company`, `followers` and `bio`.
- ↕️ **Sort & order** — sort repos by stars/forks/updated or users by followers/repositories/joined, ascending or descending.
- 🔑 **Optional token** — supply a personal access token for ~5000 requests/hour instead of unauthenticated limits.
- ⚙️ **Reliable by design** — respects rate-limit resets and retries on 403/429 so large runs complete cleanly.

### What you get

![GitHub Scraper output example — a repositories dataset table plus the full field list (stars, forks, language, topics, license and more) as structured JSON](https://api.apify.com/v2/key-value-stores/SUfvnaFLd9z9eBCtV/records/github-scraper-output.png?v=5)

Repository mode returns one clean JSON record per repo:

```json
{
  "type": "repository",
  "id": 20929025,
  "name": "TypeScript",
  "fullName": "microsoft/TypeScript",
  "owner": "microsoft",
  "ownerType": "Organization",
  "description": "TypeScript is a superset of JavaScript that compiles to clean JavaScript output.",
  "url": "https://github.com/microsoft/TypeScript",
  "homepage": "https://www.typescriptlang.org/",
  "stars": 99123,
  "forks": 12987,
  "watchers": 99123,
  "openIssues": 5721,
  "language": "TypeScript",
  "topics": ["typescript", "javascript", "compiler"],
  "license": "Apache-2.0",
  "isFork": false,
  "isArchived": false,
  "size": 1234567,
  "defaultBranch": "main",
  "createdAt": "2014-06-17T15:28:39Z",
  "updatedAt": "2024-05-30T08:11:02Z",
  "pushedAt": "2024-05-30T07:59:41Z"
}
```

In `users` mode each record instead includes `login`, `name`, `company`, `location`, `email`, `blog`, `twitter`, `hireable`, `followers`, `following`, `publicRepos`, `publicGists` and `bio`.

#### Output schema (repositories mode)

Every repository record contains these fields:

| Field | Type | Description |
|-------|------|-------------|
| `type` | string | Record type — `repository`. |
| `id` | integer | GitHub's numeric repository ID. |
| `name` | string | Repository name, e.g. `TypeScript`. |
| `fullName` | string | Owner and name, e.g. `microsoft/TypeScript`. |
| `owner` | string | Owner login (user or organization). |
| `ownerType` | string | `User` or `Organization`. |
| `description` | string | The repository's short description. |
| `url` | string (URL) | Public GitHub URL of the repository. |
| `homepage` | string (URL) | Project homepage, when set. |
| `stars` | integer | Stargazer count. |
| `forks` | integer | Number of forks. |
| `watchers` | integer | Number of watchers. |
| `openIssues` | integer | Count of open issues and pull requests. |
| `language` | string | Primary programming language. |
| `topics` | array | List of topic tags on the repository. |
| `license` | string | SPDX license identifier, e.g. `MIT`, `Apache-2.0`. |
| `isFork` | boolean | Whether the repository is a fork. |
| `isArchived` | boolean | Whether the repository is archived. |
| `size` | integer | Repository size in kilobytes. |
| `defaultBranch` | string | Default branch name, e.g. `main`. |
| `createdAt` | string (ISO date) | When the repository was created. |
| `updatedAt` | string (ISO date) | Last metadata update timestamp. |
| `pushedAt` | string (ISO date) | Last push (code change) timestamp. |

### Input

Set what to scrape (`repositories` or `users`), a GitHub search `query`, and how many results — then click **Start**. No login or setup required.

```json
{
  "mode": "repositories",
  "query": "stars:>10000 language:typescript",
  "maxItems": 100,
  "sort": "stars",
  "order": "desc",
  "enrichUserProfiles": true,
  "githubToken": ""
}
```

| Field | Description |
|-------|-------------|
| `mode` | Scrape `repositories` or `users` / profiles. |
| `query` | A GitHub search query, e.g. `stars:>1000 language:python topic:ai` or `location:berlin language:go`. |
| `maxItems` | Max repositories or users to return (GitHub caps search at 1000 per query). |
| `sort` / `order` | Sort field (stars, forks, followers, joined…) and direction (`desc` or `asc`). |
| `enrichUserProfiles` | In users mode, fetch each user's full profile (email, company, followers, bio). |
| `githubToken` | Optional personal access token for higher rate limits; no scopes needed for public data. |

### Use cases

- Build developer lead lists from user profiles filtered by location, language and followers.
- Research the top repositories in a technology or topic for competitive analysis.
- Source open-source maintainers with public emails for outreach or recruiting.
- Track stars, forks and activity across a curated set of projects over time.

### Pricing

Pay-per-event: you're billed a small amount per run and per item scraped — you only pay for what you get. See the **Pricing** tab for current rates.

### FAQ

**Do I need a GitHub token or API key?**
No. The actor works out of the box against the official [GitHub](https://github.com/) REST API with no login. A personal access token is optional — supply one to raise the rate limit to about 5000 requests/hour; no scopes are needed for public data.

**How many repositories or users can I scrape per run?**
Up to 1000 per query — GitHub caps its search API at 1000 results per query. Use `maxItems` to set your own lower limit and refine the `query` to stay within that window.

**Is scraping GitHub allowed?**
This actor uses GitHub's own public REST API and reads only publicly available data, respecting rate-limit resets and retrying on 403/429. Review GitHub's Acceptable Use Policies before large-scale or commercial reuse, especially of profile emails.

**What's the output format?**
Structured JSON — in `repositories` mode, one record per repo (`stars`, `forks`, `language`, `topics`, `license`, dates); in `users` mode, one profile per user (`login`, `company`, `location`, `email`, `followers`, `bio`). Export as JSON, CSV, Excel or via the API.

**Can I filter by stars, language or location?**
Yes. Use GitHub's query qualifiers directly, e.g. `stars:>10000 language:typescript` for repos or `location:berlin followers:>500` for users, and sort by stars, forks, updated, followers or joined.

**Can I connect the GitHub Scraper to Zapier, Make, Slack or Google Drive?**
Yes. The GitHub Scraper can be connected with almost any cloud service or web app thanks to [integrations on the Apify platform](https://apify.com/integrations). It works with [Make](https://apify.com/integrations/make), [Zapier](https://apify.com/integrations/zapier), [Slack](https://docs.apify.com/platform/integrations/slack), [Airbyte](https://docs.apify.com/platform/integrations/airbyte), [GitHub](https://docs.apify.com/platform/integrations/github), [Google Drive](https://docs.apify.com/platform/integrations/drive) and [many more](https://apify.com/integrations), plus the [Apify API](https://docs.apify.com/api/v2), JavaScript/Python clients and MCP. Or use [webhooks](https://docs.apify.com/platform/integrations/webhooks) to trigger an action whenever a run finishes — get a notification or kick off another process downstream.

### Related Actors

Doing developer research, lead-gen or recruiting? Combine this with our other developer-data scrapers:

- [npm Package Scraper](https://apify.com/hipersoft/npm-scraper) — search and look up npm packages with metadata, dependencies and downloads.
- [PyPI Package Scraper](https://apify.com/hipersoft/pypi-scraper) — metadata, dependencies, licenses and downloads for Python packages.
- [Stack Overflow & Stack Exchange Scraper](https://apify.com/hipersoft/stackexchange-scraper) — questions by tag or search across Stack Exchange sites.
- [Hacker News Scraper](https://apify.com/hipersoft/hacker-news-scraper) — stories, comments and Ask/Show HN by keyword and filters.

### Notes

This actor uses the public GitHub REST API and reads only publicly available data. It is an independent tool and is not affiliated with, endorsed by, or sponsored by GitHub, Inc. or Microsoft; "GitHub" is a trademark of GitHub, Inc.

# Actor input Schema

## `mode` (type: `string`):

Search repositories or user profiles.

## `query` (type: `string`):

A GitHub search query. Repos: "stars:>1000 language:python topic:ai". Users: "location:berlin language:go followers:>500".

## `maxItems` (type: `integer`):

Max repositories or users to return (GitHub search caps at 1000 per query).

## `sort` (type: `string`):

Repos: stars, forks, help-wanted-issues, updated. Users: followers, repositories, joined. Empty = best match.

## `order` (type: `string`):

desc or asc.

## `enrichUserProfiles` (type: `boolean`):

For users mode: fetch each user's full public profile (email, company, followers, bio). Costs one extra request per user.

## `githubToken` (type: `string`):

Optional personal access token for higher rate limits (~30 search req/min, 5000/hr vs ~10/min unauthenticated). No scopes needed for public data.

## `proxyConfiguration` (type: `object`):

Optional. The GitHub API works without a proxy.

## Actor input object example

```json
{
  "mode": "repositories",
  "query": "stars:>10000 language:typescript",
  "maxItems": 100,
  "sort": "stars",
  "order": "desc",
  "enrichUserProfiles": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

The scraped results as dataset items.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "stars:>10000 language:typescript"
};

// Run the Actor and wait for it to finish
const run = await client.actor("hipersoft/github-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "stars:>10000 language:typescript" }

# Run the Actor and wait for it to finish
run = client.actor("hipersoft/github-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "stars:>10000 language:typescript"
}' |
apify call hipersoft/github-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,hipersoft/github-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/CoqrH3fexiCXnKJTw/builds/qYiY6z6nF3axOI9Eb/openapi.json