Obscura: A Lightweight Rust Headless Browser for AI Web Scraping
Obscura is a Rust headless browser built for scraping JS-heavy sites into markdown with far less memory than Chrome-based tools. Here's how it works.
What is Obscura?
Obscura is an open-source headless browser written in Rust. It runs actual JavaScript through the V8 engine, which means it can render modern, JavaScript-heavy websites the same way Chrome does, but without shipping a full copy of Chrome to do it. Instead of a sprawling browser install, Obscura ships as a single self-contained binary with no Node, no Chrome, and no extra dependencies to install.
The pitch is simple: give people the ability to scrape real, rendered web pages without paying the memory and startup cost that comes with Puppeteer or Playwright driving headless Chrome.
TL;DR
- Obscura is a Rust-based headless browser that renders JavaScript through V8 while using a fraction of the memory that Chrome-based headless setups need.
- It ships as one binary with no Chrome, Node, or extra runtime dependencies, so installation is just downloading and extracting a file.
- It converts rendered pages into clean markdown, which is a format well suited for feeding directly into large language models.
- It pairs naturally with local models, since a demonstrated pipeline scraped a page with Obscura and piped the markdown straight into a Qwen model running on Ollama, with nothing leaving the machine.
- It supports MCP (Model Context Protocol), letting an AI agent call Obscura as a tool over standard input and output rather than requiring a custom scraping script.
- It’s built in layers (crates), with a CDP-compatible server on top and separate packages handling HTTP fetching, HTML parsing, and JavaScript execution underneath.
One coffee. One working app.
You bring the idea. Remy manages the project.
Why does headless scraping need an alternative to Chrome?
Most of the modern web doesn’t render with a plain HTTP request. Pages built on frameworks like React, Vue, or Next.js often return a near-empty HTML shell until JavaScript runs and fills in the content. Scraping tools that only fetch raw HTML miss most of what a real visitor sees.
The standard fix has been headless Chrome, automated through Puppeteer or Playwright. That works, but it’s expensive to run at scale. A single headless Chrome instance can consume well over 200 megabytes of RAM and take seconds to spin up. Multiply that across dozens or hundreds of concurrent scraping jobs, and infrastructure costs and startup latency add up fast, especially in pipelines that need to scrape many pages quickly for data collection, fine-tuning datasets, or agent workflows.
Obscura targets exactly that gap. According to its own benchmarks as demonstrated in testing, it uses around 30 megabytes of memory and loads pages in roughly 85 milliseconds, with near-instant startup since there’s no browser engine to boot.
How do you install and run Obscura?
Getting Obscura running doesn’t require Rust toolchains, package managers, or compiling anything. The binary is downloaded and extracted (typically with a tar -xzf style command), and that’s it. There’s no separate Chrome install, no Node runtime, no npm packages.
A basic workflow looks like this:
- Download and untar the Obscura binary.
- Run a quick sanity check command to confirm it works.
- Point it at a URL and tell it to output the rendered page as markdown, saved to a file.
In a real test, scraping a full personal blog (home page, tabs, and multiple blog post summaries) completed in roughly 15 seconds and produced a markdown file with around 267 lines of content. Memory usage during the run stayed minimal, consistent with the lightweight design.
One practical note from that same test: Obscura will sometimes surface harmless warnings, like an inline script error from a theme’s dark mode toggle. Those don’t stop the page from rendering correctly, but they’re worth watching for since scraped output can also include page furniture like headers and footers that need cleanup before the markdown is production ready.
How does Obscura fit into a local AI pipeline?
The interesting part isn’t just scraping, it’s what happens after. Since Obscura outputs clean markdown, that text can be piped directly into a local language model without any conversion step.
In a demonstrated setup, a single blog post was scraped with Obscura and the resulting markdown was piped straight into a Qwen model (a 27 billion parameter variant) running locally through Ollama, with a prompt asking for a five-bullet summary. The model, running with reasoning enabled, processed the scraped markdown and returned an accurate summary based only on the content Obscura had pulled from the page.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
The appeal here is that nothing leaves the machine. Obscura handles fetching and rendering, the local model handles reading and summarizing, and the entire pipeline runs without hitting a third-party API. That matters for anyone scraping internal documents, intranet pages, or any content where sending data to an external LLM API isn’t an option.
This pipeline can also be wrapped into a reusable script that takes any URL as input, scrapes it with Obscura, and forwards the markdown to a model through the Ollama API automatically, removing the need to manually chain commands for every new page.
What does Obscura’s architecture look like?
Obscura is organized into eight separate crates (Rust’s term for packages), each handling one layer of the scraping process. Requests flow downward through these layers in order:
- A Puppeteer-compatible client sends a WebSocket frame to a CDP (Chrome DevTools Protocol) server, so tools built expecting a Chrome-like interface can still talk to Obscura.
- The CDP server routes the request by session ID to a dispatcher.
- The dispatcher calls a navigation handler (like
page.navigate). - The browser layer performs the navigation and waits for the page to settle, then fans out to three lower-level packages: one for the HTTP fetch, one for parsing HTML into a document tree, and one for executing the page’s inline JavaScript through V8.
Calls only move through adjacent layers, never sideways, which keeps each piece narrow and easier to reason about. All pages share a single V8 isolate running on one thread, so a lock serializes JavaScript execution across whatever pages are being processed at once.
Does Obscura support AI agents directly?
Yes. Obscura speaks MCP (Model Context Protocol), which means it can act as a tool that an AI agent calls directly rather than requiring a human to run a scraping script. In a demonstrated example, an MCP server for Obscura was addressed over standard input/output (STDIO), given a URL to navigate to, and asked to hand back the page as markdown. The tool call succeeded, confirmed the page title, and returned the markdown, which was then piped into the local Ollama model just like the manual scraping example.
This matters for anyone building agent workflows where an LLM decides which pages to visit and what to do with the content. Instead of hardcoding scraping logic, the agent can call Obscura as one of its available tools and get structured markdown back.
Is Obscura worth using instead of Puppeteer or Playwright?
For lightweight, high-volume scraping of JavaScript-rendered pages, Obscura’s memory and startup profile make it an attractive option, especially for anyone running scraping jobs on constrained hardware or at scale where every extra 200 megabytes of RAM per browser instance adds up. It’s also appealing for local-first AI pipelines, since it pairs cleanly with local models through Ollama without needing an external service.
That said, Obscura is a newer, more focused tool than Puppeteer or Playwright, which have years of ecosystem maturity, broader compatibility with complex sites, and extensive community tooling. Scraped output from Obscura may need pre- and post-processing to strip out headers, footers, and other page furniture before it’s clean enough for production use. Anyone planning to run it at scale should build a full pipeline around it rather than treating the raw markdown output as final.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
For use cases like gathering data for model fine-tuning, scraping internal or intranet sites where data can’t leave the network, or building agent tools that need fast, low-overhead page access, Obscura is a reasonable option worth testing against existing Chrome-based workflows.
Frequently Asked Questions
What programming language is Obscura written in?
Obscura is written in Rust and distributed as a single compiled binary, which is why it avoids the dependency overhead of Node-based tools like Puppeteer or Playwright.
Does Obscura require Chrome or Node.js to run?
No. Obscura runs its own JavaScript execution through V8 directly and doesn’t require a Chrome installation, a Node runtime, or any package manager setup.
Can Obscura output scraped pages as markdown?
Yes. Obscura can render a page and convert its content into markdown, which is a format well suited for passing directly into large language models for summarization or analysis.
Does Obscura work with local LLMs like those run through Ollama?
Yes. Markdown output from Obscura can be piped directly into a local model served through Ollama, allowing an entire scrape-and-summarize pipeline to run without any data leaving the local machine.
Does Obscura support the Model Context Protocol (MCP)?
Yes. Obscura includes an MCP server, which allows AI agents to call it as a tool over standard input/output, navigate to a page, and receive back markdown content programmatically.
