GitHub Scraper

Look up GitHub repositories by name, or search them with GitHub's own query syntax. Stars, forks, watchers, open issues, language, topics, licence, archived status and, importantly, real push activity.

Two rate-limit buckets, and why that matters to you

This is the thing to understand before you size a run. GitHub does not have one rate limit, it has several, and the two this Actor touches behave completely differently:

What you do	Bucket	Unauthenticated	With a free token
Look up a repo by name	core	60 per hour	5,000 per hour
Search repositories	search	10 per minute	30 per minute

They reset independently. A run that has spent its search budget still has 59 repo lookups available, and the reverse. Treating them as one ceiling stops runs that still have budget, so this Actor tracks each bucket by the x-ratelimit-resource header GitHub returns and reports both in the run summary.

Sixty repositories an hour is the entire unauthenticated budget. For any real work, create a personal access token: it takes a minute, needs no approval, and needs no scopes at all for public repository data. Paste it into token.

The Actor stops with two requests to spare rather than draining a bucket, and says which one it stopped on.

pushed_at, not updated_at

The single most useful field here, and the one most easily got wrong.

updated_at moves when anything about the repo changes, including a description edit, a topic being added or a star arriving. A repository with no commit for four years can show an updated_at from last week and look alive.

pushed_at moves only when code is actually pushed. days_since_push is computed from that, and the run summary counts how many of your repositories have had no push in over a year.

Combined with archived, that is the honest answer to "is this dependency maintained?" — a question the star count cannot answer at all.

fork_ratio and what it tells you

Forks divided by stars. Most libraries sit around 0.08 to 0.25: encode/httpx is 0.082, psf/requests 0.186, torvalds/linux 0.261.

A ratio far above that usually means the repository is copied rather than depended on — a template, a course, a boilerplate, a coding-challenge starter. That is a useful filter when you are looking for libraries and the search results keep handing you awesome-lists.

Two modes

By name, which uses the core bucket:

repos: ["encode/httpx", "psf/requests"]

Full GitHub URLs are accepted and normalised, so you can paste https://github.com/psf/requests straight in.

By search, which uses the search bucket and GitHub's own query syntax:

query: "language:python stars:>20000"
sort: stars

Anything GitHub search accepts works here: topic:scraper, org:apify, created:>2024-01-01, license:mit, and combinations.

Fields

• Identity: full_name, owner, owner_type, name, description, url
• Popularity: stars, forks, watchers, fork_ratio
• Work in progress: open_issues
• Classification: language, topics, license
• Activity: created_at, updated_at, pushed_at, days_since_push
• Health: archived, disabled, is_fork
• Other: homepage, default_branch, size_kb

license returns the SPDX id, and null when GitHub reports NOASSERTION — which is what it sends for a licence file it cannot identify. Passing that through would give you a licence apparently called NOASSERTION, so it does not.

Input reference

Field	Type	Default
repos	list of owner/name or URLs	["encode/httpx","psf/requests"]
query	GitHub search syntax	—
sort	stars, forks, updated, help-wanted-issues	stars
order	desc, asc	desc
token	free personal access token	—
limit	1-1000	100
retries	1-6	3

GitHub caps any search at 1,000 results however you page it. When your query matches more than that, the run summary says so rather than quietly stopping.

Typical uses

• Dependency health review. Feed your dependencies' repositories in and sort by days_since_push, filtering on archived. That is the maintenance picture a star count hides.
• Licence audit. license across every repository you depend on.
• Ecosystem and competitor research. Search a topic or language and rank by stars, then use fork_ratio to separate libraries from templates.
• Technology tracking. created:> queries sorted by stars show what is new and gaining traction in a language.
• Organisation overview. org:<name> returns everything a company publishes, with activity attached.

Notes

A repository that does not exist and one that is private are indistinguishable without a token that can see it: GitHub answers 404 to both, deliberately, so the API cannot be used to probe for private repositories. The error names both possibilities rather than guessing.

A 401 means the token was rejected and is not retried, because retrying a bad credential cannot help. A 403 with x-ratelimit-remaining: 0 is a spent bucket rather than an auth problem, and is reported as such.