Reproducing Karpathy’s NanoChat on a Single GPU — Step by Step with AI Tools

A fully runnable notebook version of Karpathy’s NanoChat — reproduced using DeepWiki, Gemini, and Modal, all on a single GPU.

Modal’s Notebook

🧠 Background

Andrej Karpathy’s NanoChat distills the core mechanics of training and running a small chatbot from scratch — a transparent, educational glimpse into how large language models (LLMs) actually work under the hood.

However, the original repository expects a multi-GPU setup. For those experimenting in Colab or with limited compute, reproducing the project can be challenging.

This article shows how I reproduced NanoChat end-to-end on a single GPU using a chain of AI-assisted tools — DeepWiki, Gitingest, Gemini 2.5 Pro, and Modal. The result: a single Colab-style notebook that runs start-to-finish on one GPU.

🧩 Step 1: DeepWiki — Building the Colab Skeleton

I began by prompting DeepWiki with Deep Research enabled:

Re-produce the project in a single Colab Notebook that run in single GPU.

DeepWiki generated a detailed plan, reworking NanoChat’s multi-GPU pipeline into a single-device workflow. It covered everything — data preparation, tokenizer training, model definition, base model pre-training, and chat inference.

📄 Read the full DeepWiki output here:
👉 DeepWiki result

📦 Step 2: Flattening the Repository

Next, I used Gitingest to download the full GitHub repository as a single text file named:

karpathy-nanochat.txt

This consolidated file captures the entire repo — every script, every function — allowing it to be fed directly into an AI model for reconstruction without missing dependencies or exceeding token limits.

🤖 Step 3: Reconstructing the Colab Notebook with Gemini 2.5 Pro

With the flattened repo ready, I uploaded it to Gemini 2.5 Pro, alongside the DeepWiki-generated plan.

Here’s the full prompt I used:

Please help to re-produce the uploaded repository `karpathy-nanochat.txt` in a single Colab Notebook that run in single GPU according to the following steps:
[Paste the DeepWiki answer from Step 1 here]

Gemini then synthesized a single executable Colab notebook, cleanly segmented by tasks — from imports and data prep to training and inference — all optimized for a single-GPU environment. I made some minor fixes.

⚙️ Step 4: Running on a Single GPU via Modal

For execution, I used Modal.com notebooks, which provide an NVIDIA A100 80GB GPU under their $30/month free credit plan.

Get Lim Chee Kin’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

I ran the Base Model Pretraining stage with the flag:

--depth=4

This reduced model depth gave a faster training loop, allowing me to verify the pipeline without consuming the entire GPU budget.

🔗 Notebook link: https://modal.com/notebooks/limcheekin/_/nb-WiyBN5DvexQTakCvYQHE9l
(Includes full training logs, loss curves, and sample outputs.)

🧮 Step 5: DeepWiki Optimization Queries

To better understand NanoChat’s scaling behavior, I asked DeepWiki a few follow-up questions:

1. Optimal --device_batch_size for base model pre-training on an A100 80GB.
2. Estimated runtime for completing pre-training on that GPU.
3. Techniques to reduce total pre-training time.

📄 Full Q&A here:
👉 DeepWiki optimization Q&A

DeepWiki’s responses offered insights into GPU memory utilization, trade-offs of gradient checkpointing, and ways to accelerate experimentation (e.g., reduced sequence length, mixed precision).

💬 A Favor to Readers

If you ever manage to run the full pipeline at --depth >= 12 within a $30 budget, please share your results or notebook link in the comments section. Your experience could help others benchmark realistic training costs for small-scale LLM experiments.

🧭 Key Takeaways

• AI-assisted reproduction works. DeepWiki and Gemini can rebuild complex repos into runnable notebooks with minimal human intervention.
• Flattening the repo helps. Tools like Gitingest make large repos digestible for AI models.
• Accessible compute is enough. A single A100 GPU (or Colab Pro GPU) can run the full NanoChat pipeline when properly tuned.
• Iterative testing saves cost. Start small (--depth=4) to validate the setup before scaling to deeper models.

💡 Closing Thoughts

This workflow demonstrates a new frontier in AI-for-AI engineering — using generative models to reconstruct, optimize, and execute other AI projects.

Reproducing NanoChat on a single GPU isn’t just a technical exercise; it’s a glimpse at a near-future where codebases can be recompiled, refactored, and relaunched entirely through AI collaboration.