Sitemap
Open in app

Sign in

Medium Logo
Write
Search

Sign in

I Built a Zero-Cost Paper Replication Pipeline Using Feynman and Colab-MCP

6 min readMay 20, 2026

--

From reading a paper to running real experiments — no cloud bill, no lab access required

Press enter or click to view image in full size
Generated By Gemini

For the past two months, I’ve been working on one question: can you build a research pipeline that approaches “AI Scientist” quality using only free tools?

The answer is yes — and it works better than I expected.

The setup uses two tools: Feynman (an AI research agent) and colab-mcp (Google’s MCP server for Colab). Combined, they form a pipeline that takes you from literature discovery all the way to running actual GPU experiments, fully automated.

The Problem

ML research has two real barriers.

The first is compute. Your local machine can’t run serious training jobs. Cloud GPUs charge by the hour. The A100 clusters live in labs that most people don’t have access to.

The second is information density. arXiv publishes hundreds of papers a day. Manual triage plus replication verification is a full-time job on its own.

This pipeline attacks both.

The Two Tools

Feynman (feynman.is)

An open-source AI research agent that runs as a local CLI. Core commands:

  • feynman "your question" → quick Q&A on field state
  • feynman lit "topic" → deep literature review with consensus and open debates
  • feynman deepresearch "topic" → multi-agent parallel investigation
  • feynman audit 2401.12345 → audit paper claims vs actual code
  • feynman replicate "paper title" → generate a full replication plan

The last two are the most useful. audit surfaces the gap between what a paper claims and what the code actually does — hyperparameter mismatches, data processing shortcuts, metric definition inconsistencies. replicate outputs a structured plan: environment setup, dependency list, hyperparameters, expected metric ranges, step-by-step experiment flow.

colab-mcp (github.com/googlecolab/colab-mcp)

Released by Google in March 2026. It exposes Google Colab as an MCP server, letting any MCP-compatible agent directly control a live notebook.

In practice: you tell Claude Code “run this in Colab,” and it creates cells, writes code, executes them, and reads the output — all automatically, inside the Colab tab open in your browser.

That Colab instance has a free T4 GPU.

Pipeline Architecture

feynman  (research intelligence layer)
    ↓  generates replication plan
Claude Code  (orchestration layer, configured with colab-mcp)
    ↓  parses and executes
colab-mcp → Google Colab  (GPU compute layer)
    ↓  returns results
feynman  (analyzes gaps, drafts report)

Feynman reads papers. Colab runs experiments. Claude Code is the nervous system connecting them.

Prerequisites

Five tools, all free:

Tool Install Role uv curl -LsSf https://astral.sh/uv/install.sh | sh Python package manager Feynman curl -fsSL https://feynman.is/install | bash Research agent Claude Code npm install -g @anthropic-ai/claude-code Orchestration layer colab-mcp See setup below Colab MCP server Google Colab Open in browser GPU execution environment

Setup

Step 1: Install colab-mcp

The official repo removed GPU runtime switching in issue #41. Use the community fork instead — it restores GPU control and pre-registers all tools so they work immediately with Claude Code:

git clone https://github.com/SebastianGilPinzon/colab-mcp.git
cd colab-mcp

If you just want to verify the connection first, the official version is more stable:

uvx git+https://github.com/googlecolab/colab-mcp

Step 2: Configure Claude Code MCP

Create .mcp.json in your project root.

Official version:

{
  "mcpServers": {
    "colab-mcp": {
      "command": "uvx",
      "args": ["git+https://github.com/googlecolab/colab-mcp"],
      "timeout": 30000
    }
  }
}

Community fork (with GPU control):

{
  "mcpServers": {
    "colab-mcp": {
      "command": "uv",
      "args": ["run", "colab-mcp", "--client-oauth-config", "oauth.json"],
      "cwd": "/path/to/SebastianGilPinzon/colab-mcp",
      "timeout": 30000
    }
  }
}

Step 3: Install Feynman

curl -fsSL https://feynman.is/install | bash
feynman --version

Step 4: Open Colab and add a keep-alive cell

Open colab.google.com, create a new notebook, keep the tab open.

Get Bayar Hwasaii’s stories in your inbox

Join Medium for free to get updates from this writer.

Run this in the first cell to prevent idle disconnection:

import threading, time
def keep_alive():
    while True:
        time.sleep(55)
        print(".", end="", flush=True)t = threading.Thread(target=keep_alive, daemon=True)
t.start()
print("Keep-alive started")

That’s the full setup.

The Five-Phase Workflow

Phase 1: Literature Recon

Get a clear picture of the field before picking a paper to replicate:

# Quick field overview
feynman "what do we know about space debris detection with deep learning"
# Deep review with consensus and open debates
feynman lit "YOLOv8 for space debris"# Multi-agent parallel investigation for unfamiliar territory
feynman deepresearch "vision transformer object detection 2024"

Phase 2: Paper Audit

Once you’ve picked a paper, audit it before touching any code:

feynman audit 2401.12345
# Output: claim-vs-code diff report
# Focus on: hyperparameters, data preprocessing, metric definitions

This step gets skipped constantly. It shouldn’t be. Most replication failures trace back to exactly the discrepancies this step surfaces.

Phase 3: Generate Replication Plan

feynman replicate "Enhanced YOLOv8 for space debris detection" > replicate_plan.md

The output includes:

  • Dataset acquisition instructions
  • Full dependency list (pip-ready)
  • Hyperparameter configuration
  • Expected metrics (accuracy / mAP / loss ranges)
  • Step-by-step experiment flow

Phase 4: Execute in Colab

Hand the plan to Claude Code:

claude "
Read replicate_plan.md and execute the following in Colab:
1. Create a markdown cell describing the experiment goal
2. Install all dependencies
3. Download and prepare the dataset
4. Implement the model
5. Run training and inference
6. Output metrics and compare against the paper
Log the output of each step.
"

You’ll watch cells appear in your Colab tab, code executing line by line.

GPU selection reference:

Task Config Time estimate Inference / quick validation Colab T4 (free) Minutes Medium training (< 10 epochs) Colab T4 (free) Hours Large-scale training Colab Pro A100 / RunPod On demand Fast burst jobs Modal (Feynman native) Seconds cold start

To switch GPU: Runtime → Change runtime type → T4 GPU. With the community fork, you can tell Claude Code directly.

Phase 5: Result Analysis

Once the experiment finishes:

feynman "
Given these experimental results: [paste Colab output]
And this paper: [arXiv ID or title]
Analyze: what explains the gaps?
Are they within acceptable variance or do they indicate reproducibility issues?
"

Common Failure Modes

Problem Cause Fix Colab disconnects mid-run Idle timeout Run the keep-alive cell GPU switching fails Official colab-mcp removed runtime control Use community fork or switch manually in browser MCP tools don’t appear Official version relies on tools/list_changed notification Use community fork (pre-registers all tools) feynman replicate times out Docker container resource limit Skip Feynman's execution layer; pass the plan directly to Claude Code Colab output truncated Cell output length limit Write to file: open('results.txt', 'w')

Recommended Project Structure

research-pipeline/
├── .mcp.json              # colab-mcp config
├── papers/                # PDFs and arXiv IDs
├── plans/                 # feynman replicate outputs
│   └── replicate_plan.md
├── notebooks/             # exported .ipynb files
├── results/               # experiment outputs and metrics
└── reports/               # final analysis reports

What This Pipeline Can and Can’t Do

It handles: literature triage, code auditing, experiment replication, baseline comparison, result documentation. For compute-heavy research (ML, data analysis, simulation), the execution layer is largely covered.

It doesn’t replace: research direction judgment, interpretation of unexpected results, genuinely novel hypothesis generation. Those still require a human.

This is a force multiplier, not a research automaton. The distinction matters.

Conclusion

The core value of this setup in one line:

An independent researcher with no lab access can now validate ideas at lab speed.

Free GPU, automated experiment execution, AI-assisted literature review — two years ago this combination wasn’t possible. The tools exist now.

What’s Next

There’s still one hard constraint in this pipeline: Feynman’s execution layer runs in a Docker sandbox, completely separate from Colab. The two compute environments don’t talk to each other.

I’m planning to fork Feynman and replace its execution backend with colab-mcp. The goal is to close this loop:

Feynman generates hypothesis
    ↓
colab-mcp executes on real GPU
    ↓
Claude Code reads results, decides whether to iterate
    ↓
On target metric reached: Feynman /draft writes the report

Right now this is a pipeline. After the fork, it becomes an autonomous research loop. I’ll post updates as development progresses.

Resources

  • colab-mcp official repo: github.com/googlecolab/colab-mcp
  • Community fork (GPU support): github.com/SebastianGilPinzon/colab-mcp
  • Feynman: feynman.is
  • Google Colab MCP announcement: developers.googleblog.com/announcing-the-colab-mcp-server-connect-any-ai-agent-to-google-colab

Stack: Feynman + colab-mcp + Claude Code

Generative Ai Tools
Mcp Server
Agentic Ai
AI Agent
Colab

--

--

Bayar Hwasaii
Bayar Hwasaii
1 follower
62 following

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech