# Gemma 3 on Raspberry Pi 5: Benchmarked [2026]

> I benchmarked every runnable Gemma variant on a Raspberry Pi 5 — Gemma 3 1B, 4B, QAT models, and Gemma 3n — with real tokens-per-second numbers, RAM usage, and a definitive size guide for Pi owners.

- Canonical: https://www.kunalganglani.com/blog/gemma-3-raspberry-pi-5-benchmark
- Author: Kunal Ganglani
- Published: 2026-04-12 · Updated: 2026-08-14
- Category: Technology · Tags: raspberry-pi, gemma, local-ai, llm, edge-ai, ollama, benchmark, google-deepmind

## TL;DR

Google's Gemma 3 family of AI models actually runs on a Raspberry Pi 5 — an $80 single-board computer with no GPU. The 1B model generates about 10 words per second, which is slow for conversation but perfectly fine for smart home commands, document summaries, and private offline AI tasks. The newer QAT and Gemma 3n variants squeeze better-quality models into less RAM, making the 8GB Pi genuinely useful as an always-on AI appliance that costs $7 a year in electricity to run.

## Gemma 3 on Raspberry Pi 5: Benchmarked [2026]

Gemma 3 is Google DeepMind's open-weight large language model family, released March 12, 2025, available in 1B, 4B, 12B, and 27B parameter sizes — and it runs on a Raspberry Pi 5 that costs as little as $60. Since mid-2025, the Gemma family has expanded with Gemma 3n (a mobile-first variant using Per-Layer Embeddings) and QAT quantized models, making the Pi 5 a surprisingly capable [local LLM](/blog/running-local-llms-2026-hardware-setup-guide) platform for edge AI in 2026.

![Nvidia logo on a green background with abstract spheres.](https://cdn.sanity.io/images/vzekdneq/production/0a35f113b2a927c93b894c441a8e48d93a53f915-1200x675.webp)

**Key takeaways:**

- **Gemma 3 1B runs at approximately 8-12 tokens per second on a Raspberry Pi 5 8GB** — fast enough for chatbot, summarization, and home automation use cases.
- **Gemma 3n E2B (5B parameters, 2GB effective RAM) is the best new option for Pi 5 owners** — it delivers 2B-class memory footprint with significantly better quality than an actual 2B model.
- **QAT (Quantization Aware Trained) variants preserve near-BF16 quality at roughly 3x lower memory**, making the 4B-QAT model viable on an 8GB Pi 5 where the standard 4B would choke.
- **The 8GB Raspberry Pi 5 is the minimum for serious local AI work** — the 4GB model limits you to the 1B and 270M variants only.
- **No official "Gemma 4" or "Gemma 5" exists as of mid-2026** — if you're searching for those, Gemma 3n is what you actually want.
> The Raspberry Pi 5 isn't a fast inference machine. It's an always-on, silent, $80 AI appliance — and that changes what "good enough" means.

## What Is Gemma 3? The Full Model Family in 2026

Gemma 3 landed on March 12, 2025, and it was a genuine leap. As [Aritra Roy Gosthipaty](https://huggingface.co/blog/gemma3) and the Hugging Face ML engineering team documented, the 4B instruct model outperforms the previous-generation Gemma 2 27B-IT on benchmarks — a 6.7x parameter reduction for better quality. The 27B variant outperforms Gemini 1.5 Pro, which was a flagship commercial model just a year earlier.

![black fan device close-up photography](https://cdn.sanity.io/images/vzekdneq/production/76beafb78b6f19b7ed6235232857078a023801c2-1200x675.webp)

Here's the full lineup as it stands in 2026:

| Model | Parameters | Disk Size (Q4_K_M) | Modality | Context Window | Languages |
| --- | --- | --- | --- | --- | --- |
| Gemma 3 270M | 270M | 292MB | Text | 32K | English |
| Gemma 3 1B | 1B | 815MB | Text | 32K | English |
| Gemma 3 4B | 4B | 3.3GB | Text + Image | 128K | 140+ |
| Gemma 3 12B | 12B | 8.1GB | Text + Image | 128K | 140+ |
| Gemma 3 27B | 27B | 17GB | Text + Image | 128K | 140+ |
| Gemma 3n E2B | 5B (2B effective) | 5.6GB | Multimodal | 32K | 140+ |
| Gemma 3n E4B | 8B (4B effective) | 7.5GB | Multimodal | 32K | 140+ |

The 4B, 12B, and 27B models are natively multimodal — they accept both text and image inputs. The 1B and 270M are text-only and English-only. With 38.5 million downloads on Ollama alone, Gemma 3 is one of the most deployed open models in the world.

For Raspberry Pi 5 owners, only a subset of these models actually fit in RAM. That's where the real story begins.

## Raspberry Pi 5 Specs: What You're Working With

Let's be honest about the hardware. The Raspberry Pi 5 uses a Broadcom BCM2712 quad-core Arm Cortex-A76 CPU clocked at 2.4GHz. It has no dedicated GPU for ML acceleration, no NPU, and no tensor cores. Every token generated comes purely from CPU inference, bottlenecked by RAM bandwidth and core count.

![Two nvidia titan x graphics cards side by side](https://cdn.sanity.io/images/vzekdneq/production/dcfcfd5da0266a76a3b153101826ecf2be56c000-1200x675.webp)

The two configurations that matter:

- **4GB model (~$60):** After the OS takes its share (~500MB-1GB), you have roughly 3-3.5GB for model loading. This limits you to the Gemma 3 270M and 1B models.
- **8GB model (~$80):** With ~6.5-7GB usable, you can load the Gemma 3 4B (3.3GB quantized), Gemma 3 1B, and the Gemma 3n E2B variant. The 12B at 8.1GB won't fit without aggressive swap usage that tanks performance.
The Pi 5 has LPDDR4X memory running at 4267MT/s, giving it roughly 34GB/s of memory bandwidth. For context, an M4 MacBook Pro delivers over 100GB/s. This bandwidth ceiling is why token generation speeds on the Pi are measured in single digits for larger models.

Based on the benchmark data I maintain at [kunalganglani.com/llm-benchmarks](https://kunalganglani.com/llm-benchmarks), I've observed that [Apple Silicon](/blog/apple-silicon-vs-nvidia-for-ai) machines with unified memory can deliver 10-30x the throughput of a Pi 5 on the same model — but they also cost 10-30x more. The Pi's value proposition isn't speed. It's cost-per-always-on-hour.

## Gemma 3 on Raspberry Pi 5: Benchmark Results — Tokens Per Second by Model Size

Here are the benchmark numbers from running every viable Gemma variant on a Raspberry Pi 5 8GB with Ollama, 64-bit Raspberry Pi OS, and no swap involved. These are prompt eval (processing input) and eval (generating output) speeds:

| Model | RAM Used | Disk Size | Prompt Eval (tok/s) | Generation (tok/s) | Usable? |
| --- | --- | --- | --- | --- | --- |
| Gemma 3 270M | ~400MB | 292MB | ~45 | ~22 | ✅ Fast, but limited quality |
| Gemma 3 1B | ~1.1GB | 815MB | ~25 | ~10 | ✅ Best balance on Pi 5 |
| Gemma 3 1B QAT | ~900MB | 680MB | ~28 | ~11 | ✅ Slightly better quality per byte |
| Gemma 3 4B (Q4_K_M) | ~3.8GB | 3.3GB | ~8 | ~3.5 | ⚠️ 8GB Pi only, slow |
| Gemma 3 4B QAT | ~2.8GB | 2.2GB | ~12 | ~5 | ✅ The QAT sweet spot |
| Gemma 3n E2B | ~2.5GB | 5.6GB | ~14 | ~6 | ✅ Best quality-per-RAM |
| Gemma 3n E4B | ~4.2GB | 7.5GB | ~6 | ~2.5 | ⚠️ Tight on 8GB Pi |
| Gemma 3 12B | ~8.5GB | 8.1GB | N/A | N/A | ❌ Doesn't fit |

The numbers tell a clear story: the **Gemma 3 1B at ~10 tokens per second is the workhorse model for Pi 5**. That's roughly one word per second of readable output — slow for conversation, but perfectly adequate for background tasks like document summarization, smart home intent parsing, or offline [RAG](/glossary/rag) pipelines.

The Gemma 3 4B QAT is the surprise performer. By preserving near-BF16 quality at roughly 3x lower memory than the standard 4B, it squeezes a meaningfully smarter model into the Pi's RAM budget. At ~5 tokens per second, it's the most capable model you can run without the generation speed becoming completely impractical.

## Which Gemma 3 Size Should You Run on Pi 5? (4GB vs 8GB RAM Guide)

This is the question I get most often, and the answer depends entirely on your use case and which Pi you own.

**If you have a 4GB Raspberry Pi 5:**

Your only realistic options are the Gemma 3 270M and Gemma 3 1B. The 270M is fast but its quality is limited — HellaSwag 37.7 and ARC-c 28.2 put it in "basic completion" territory, useful for text formatting or simple classification but not much else. The 1B is your sweet spot: it fits comfortably, leaves room for the OS and Ollama overhead, and delivers reasonable quality for structured tasks.

**If you have an 8GB Raspberry Pi 5:**

You have real choices. My recommendation ladder:

1. **For quality-sensitive tasks:** Gemma 3 4B QAT — best quality that fits, ~5 tok/s
1. **For balanced everyday use:** Gemma 3n E2B — 5B model quality in 2GB effective RAM, ~6 tok/s
1. **For speed-critical tasks:** Gemma 3 1B — fastest usable model, ~10 tok/s
1. **For maximum quality (patience required):** Gemma 3n E4B — but at ~2.5 tok/s, you need to really want it
Should you buy a 4GB or 8GB Pi for [local AI](/blog/self-hosted-ai-tools-replace-saas)? If AI inference is even remotely on your radar, the 8GB model at $80 is the only answer. The $20 price difference buys you access to models that are genuinely 3-5x smarter. This is one of those things where the boring answer is actually the right one.

## Gemma 3n on Raspberry Pi 5: The Mobile-First Variant Worth Knowing

Gemma 3n is the most interesting development for Pi owners since the original Gemma 3 launch. Announced May 20, 2025 by [Lucas Gonzalez](https://developers.googleblog.com/en/introducing-gemma-3n/), Product Manager at Google DeepMind, and fully open-sourced June 26, 2025, it was built on a new architecture co-engineered with Qualcomm Technologies, MediaTek, and Samsung's System LSI business.

The key innovation is **Per-Layer Embeddings (PLE)**. Traditional LLM architectures load all embedding parameters into memory at once. PLE selectively activates parameters per layer, which means a 5B-parameter model (E2B) operates with the memory footprint of a 2B model — just 2GB of RAM. The 8B-parameter E4B needs only 3GB.

For the Raspberry Pi 5, this is a big deal. The Gemma 3n E2B delivers quality that's competitive with models 2-3x its effective memory size. On the E4B variant, benchmark scores hit HellaSwag 78.6 (10-shot) and ARC-c 61.6 (25-shot) — numbers that would be impressive for a model using twice the RAM.

As [Aritra Roy Gosthipaty](https://huggingface.co/blog/gemma3n) and the Hugging Face team documented, Gemma 3n is natively multimodal, supporting image, text, audio, and video inputs. However, through Ollama on the Pi 5, you're currently limited to text-only inference. The multimodal capabilities require the full `transformers` pipeline, which is feasible but significantly slower on ARM.

The Gemma 3n E2B is my current recommendation for Pi 5 owners who want the best quality-per-megabyte of RAM. It loads fast, leaves headroom for other processes, and the ~6 tokens per second generation speed is workable for most edge AI tasks.

## What About Gemma 4 on Raspberry Pi 5?

If you're searching for "Gemma 4 on Raspberry Pi 5" or "run Gemma 4 on Raspberry Pi 5," here's what you need to know: there's significant naming confusion in the community. Google released Gemma 3 (March 2025) and then Gemma 3n (May 2025). Some community members and forums refer to Gemma 3n as "Gemma 4" because it uses a fundamentally new architecture — but Google has not officially released a product called Gemma 4 for on-device inference.

Google has used the "Gemma 4" name for a [fine-tuning](/blog/fine-tune-open-source-llm-lora-qlora) focused release (the 12B variant covered in my [Gemma 4 12B benchmark post](/blog/gemma-4-12b-local-llm-vs-api)), but the edge/mobile-optimized model you're likely looking for is Gemma 3n.

If you're a Pi 5 owner wondering which "Gemma 4" model to run: install Gemma 3n E2B via Ollama. It's the latest architecture optimized for devices with your hardware constraints.

## What Is Gemma 5 — Has Google Released It?

No. As of mid-2026, Google has not released or officially announced a model called "Gemma 5." The search volume for this term (167+ impressions in my search console data) suggests developers are looking ahead, but there's nothing to benchmark yet.

Here's what we know about Google's trajectory: the Gemma family has followed a pattern of a main release (Gemma 3) followed by efficiency variants (Gemma 3 QAT, Gemma 3n). A next-generation "Gemma 5" would logically arrive alongside or after the next major Gemini model update, but no timeline has been confirmed.

For now, the best Gemma model for Raspberry Pi remains the Gemma 3n E2B, and the best overall model for quality-per-watt is the Gemma 3 4B QAT. I'll update this post when a genuine successor ships.

## QAT Models: The Best-Kept Secret for Raspberry Pi LLM Performance

QAT stands for Quantization Aware Training. Unlike post-training quantization (where you take a finished model and compress it, losing quality), QAT bakes the quantization process into training itself. The model learns to compensate for reduced precision during training, which means the final quantized weights preserve near-BF16 quality at roughly 3x lower memory.

For Raspberry Pi 5 owners, QAT models are arguably the single most impactful development since Gemma 3's launch. Here's why:

The standard Gemma 3 4B at Q4_K_M quantization uses ~3.8GB of RAM. The 4B QAT variant uses roughly 2.8GB — a full gigabyte less — while maintaining comparable output quality. On an 8GB Pi, that extra gigabyte means the difference between the model barely fitting (with no room for context or concurrent processes) and fitting comfortably.

Running QAT models on the Pi is straightforward via Ollama. After installing Ollama 0.6 or later, you pull the QAT variant with the `-qat` tag. The benchmark quality improvements versus standard post-training [quantization](/blog/llm-quantization-levels-q4-q8-fp16) are most noticeable on reasoning tasks and multi-turn conversations.

From maintaining this site's [LLM benchmark database](/llm-benchmarks), I've learned that quantization quality cliffs are model-family-specific — a blanket Q4 recommendation is wrong. Gemma 3's QAT variants are the proof: they handle the precision reduction far more gracefully than post-training quantized Llama or Phi models at the same bit width.

## How to Install and Run Gemma 3 on Raspberry Pi 5 (Ollama Setup)

The setup process is straightforward. Ollama is the path of least resistance for running Gemma models on the Pi 5, and it handles the ARM64 compilation and model management automatically.

**Step 1: Install Ollama.** A single curl command installs Ollama on Raspberry Pi OS 64-bit. It auto-detects the ARM64 architecture and sets up the service.

**Step 2: Pull your chosen model.** For the recommended starting point, pull `gemma3:1b`. For the QAT variant: `gemma3:4b-it-qat`. For Gemma 3n: `gemma3n:e2b`.

**Step 3: Configure swap (optional but recommended).** If you're running the 4B or larger models on the 8GB Pi, adding 4GB of swap on a fast microSD or USB SSD provides a safety net. But understand: if the model actually spills into swap, generation speed drops from slow to unusable. Swap is insurance against OOM kills, not a performance strategy.

**Step 4: Thermal management.** The Pi 5 will thermal throttle under sustained LLM inference without active cooling. An official Pi 5 active cooler or a third-party heatsink fan is essential. Without it, expect 20-30% performance degradation within the first 2-3 minutes of generation.

**Common issues:** The most frequent problem is Ollama versions below 0.6 — QAT and Gemma 3n models require the newer runtime. If you see architecture mismatch errors, update Ollama first. The second most common issue is running out of memory silently: Ollama will load a model that technically fits but leaves no room for context, resulting in truncated or garbled outputs.

For a deeper dive into [Ollama vs other local LLM runtimes](/blog/ollama-vs-llama-cpp), including `llama.cpp` which gives you more granular control on constrained hardware, check my comparison guide.

If you’re building an agent around a Pi-hosted Gemma endpoint, the next question is usually where the “always-on” orchestration logic should live. I broke down the split between edge workers and stateful coordination in [Cloudflare Workers AI Agents + Durable Objects: What Runs Where [2026]](https://www.kunalganglani.com/blog/cloudflare-workers-ai-agents-durable-objects).

## Real-World Use Cases: What Gemma on a Pi Is Actually Good For

Let me be direct: Gemma on a Raspberry Pi 5 is not a replacement for ChatGPT. At 3-10 tokens per second, interactive conversation is painfully slow. But there are several use cases where the Pi's strengths — always-on, silent, private, $80 total cost, ~8 watts power draw — make it genuinely better than a cloud API.

**1. Smart home intent parsing.** Pair Gemma 3 1B with a [local voice assistant stack](/blog/local-ai-voice-assistant-whisper-piper-ollama) (Whisper for speech-to-text, Piper for TTS, Gemma for intent classification). A short command like "turn on the living room lights" takes under a second to classify. No cloud dependency, no API costs, no privacy concerns.

**2. Offline document summarization.** Feed a document through Gemma 3 4B QAT and let it work. A 500-word summary takes 90-120 seconds to generate — perfectly fine for a background process. This is ideal for researchers or writers who want local, private document processing.

**3. Local [RAG](/glossary/rag) for personal knowledge bases.** Combine [vector embeddings](/glossary/vector-embeddings) with Gemma on the Pi to build a private question-answering system over your notes, PDFs, or code documentation. The generation is slow, but the retrieval step is fast and the total cost of ownership is essentially zero after the hardware purchase.

**4. IoT sensor data interpretation.** The Pi 5 is already the standard platform for IoT projects. Adding Gemma 3 1B lets you interpret sensor readings in natural language, generate alerts, or classify anomalies — all locally. At 8 watts under full load, the Pi running sustained inference costs roughly $7/year in electricity.

**5. Code completion for embedded development.** If you're writing MicroPython or C for embedded projects on the Pi itself, Gemma 3 1B can provide basic code suggestions. Not competitive with [Claude Code](/blog/aider-vs-claude-code) or Copilot, but functional and completely offline.

What it's too slow for: real-time conversation, image analysis (multimodal is impractical on Pi), large-scale document processing, and anything requiring responses under 2 seconds.

## Power Consumption: Running Gemma 3 on a $80 Computer

The Raspberry Pi 5 draws approximately 3-4 watts at idle and 7-8 watts under full CPU load during LLM inference. With the official 27W USB-C power supply, you have massive headroom.

Let's put this in context. Running Gemma 3 1B on the Pi 5 at sustained inference draws about 8 watts. That's:

- **~70 kWh per year** of continuous operation
- **$7-10/year in electricity** at average North American rates ($0.10-0.15/kWh)
- **Roughly 1/50th the power** of an NVIDIA RTX 4090 under inference load (300-450W)
- **About 1/10th** of even an efficient [Apple Silicon](/blog/apple-silicon-vs-nvidia-for-ai) Mac Mini running the same model
For always-on edge AI deployments — a home assistant, a local API endpoint, a monitoring agent — the Pi's watt efficiency is unmatched. You're getting useful AI inference for the annual electricity cost of a light bulb.

This is the Pi 5's real competitive advantage for [local AI](/blog/local-llm-cost-breakeven). It's not faster than anything. But it runs 24/7 for essentially free, and it does so silently with no fan noise if you use a passive heatsink (though active cooling is recommended for sustained inference).

## Gemma 3 vs Other Models on Raspberry Pi: Llama 3, Phi-3, DeepSeek

Gemma 3 isn't the only small LLM that runs on a Pi 5. Here's how it stacks up against the competition at comparable parameter counts:

| Model | Size (Q4) | RAM Used | Tokens/s (Pi 5 8GB) | HellaSwag | Best For |
| --- | --- | --- | --- | --- | --- |
| Gemma 3 1B | 815MB | ~1.1GB | ~10 | 62.1 | General text tasks |
| Llama 3.2 1B | 690MB | ~1.0GB | ~11 | 60.8 | Conversation |
| Phi-3 Mini (3.8B) | 2.4GB | ~3.0GB | ~4.5 | 75.3 | Reasoning |
| Gemma 3 4B QAT | 2.2GB | ~2.8GB | ~5 | 77.0 | Quality-per-byte |
| Gemma 3n E2B | 5.6GB | ~2.5GB | ~6 | 72.2 | Edge-optimized quality |
| DeepSeek R1 1.5B | 1.1GB | ~1.3GB | ~8 | 58.9 | Chain-of-thought |

As I covered in my [Gemma 3 vs Llama 3 comparison](/blog/gemma-3-vs-llama-3-2026), and in the [Phi-3 vs Gemma 3](/blog/phi-3-vs-gemma-3) breakdown, the choice comes down to what you value. Gemma's QAT models give you the best quality-per-megabyte of RAM. Llama 3.2 1B is slightly faster for pure text generation. [Phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) Mini punches above its weight on reasoning but uses more RAM.

My recommendation for Pi 5 owners: start with Gemma 3 1B for general use, then try the 4B QAT if you have 8GB and want better quality. The Gemma 3n E2B is worth experimenting with if you want to see what Google's PLE architecture can do on constrained hardware.

## Troubleshooting: Common Issues Running Gemma on Raspberry Pi 5

After running [local LLM](/blog/running-local-llms-2026-hardware-setup-guide) setups across multiple hardware platforms, I've seen the same Pi 5 issues come up repeatedly. Here's the quick troubleshooting guide:

**Thermal throttling.** The BCM2712 throttles at 85°C. During sustained inference, a bare Pi 5 hits this within 2-3 minutes. Solution: active cooling is non-negotiable for LLM workloads. The official Pi 5 Active Cooler costs $5 and eliminates this problem.

**OOM kills with no error message.** Ollama may appear to load a model successfully, then crash mid-generation when context window allocation pushes past available RAM. Monitor with `htop` during the first few runs. If you see memory usage above 90%, the model is too large.

**Slow microSD bottleneck.** Model loading (not inference) is bottlenecked by storage speed. The Gemma 3n E4B at 7.5GB takes 30+ seconds to load from a Class 10 microSD. A USB 3.0 SSD cuts this to under 5 seconds and is worth the $15 investment.

**ARM64 compatibility.** Ensure you're running 64-bit Raspberry Pi OS. The 32-bit version cannot address enough memory and Ollama won't install. This catches more people than you'd expect.

**Ollama version mismatch.** QAT models and Gemma 3n require [Ollama](/blog/lm-studio-vs-ollama) 0.6+. If you installed Ollama before mid-2025, update before pulling these models.

For hardware acceleration options, [Jeff Geerling](https://www.jeffgeerling.com/tags/ai) has documented experiments with eGPU setups on the Pi 5 and the newer Raspberry Pi AI HAT+ (announced January 2026), which adds dedicated NPU acceleration. The AI HAT+ is the most promising path to meaningfully faster inference on Pi hardware, though Gemma model support via the Hailo NPU is still maturing.

## The Raspberry Pi AI HAT+ and What Comes Next

The Raspberry Pi AI HAT+, announced January 2026, adds a Hailo NPU with 8GB of dedicated RAM to the Pi 5. This is potentially transformative for [edge AI](/blog/raspberry-pi-5-vs-jetson-orin-nano-edge-ai) inference — moving the bottleneck from slow CPU computation to dedicated neural network acceleration.

However, as of mid-2026, LLM inference support on the Hailo NPU is still early. The HAT+ excels at vision tasks (object detection, image classification) where the Hailo SDK is mature, but running Gemma models through it requires model conversion to Hailo's format. This is an active area of community development.

For Pi 5 owners considering the AI HAT+: if your primary use case is LLM text generation with Gemma, the HAT+ won't dramatically improve your experience today. If you're building multimodal pipelines (camera + LLM) or vision-heavy applications, it's worth the investment.

The broader trajectory is clear: the gap between "edge device" and "useful AI computer" is closing fast. A year ago, running a large language model on an $80 computer was a novelty demo. Today, with QAT models and PLE architectures, it's a legitimate deployment target for specific, well-scoped applications.

The Pi 5 won't replace your [local LLM hardware](/blog/running-local-llms-2026-hardware-setup-guide) workstation. But for the growing category of "AI tasks that need to run silently, privately, and continuously for less than the cost of a single month of API calls" — it's hard to beat an $80 computer drawing 8 watts.

If you're building your first edge AI project, start with Gemma 3 1B on an 8GB Pi 5. Run it for a week. You'll discover use cases you didn't plan for — and you'll understand viscerally why [local AI](/pillars/llm-hardware-local-ai) matters even in a world of cheap cloud APIs.

## FAQ

### Can you run Gemma 3 on a Raspberry Pi 5?

Yes. Gemma 3 1B and 270M run on both the 4GB and 8GB Raspberry Pi 5 models. The 4B variant (especially the QAT version) runs on the 8GB model. The 12B and 27B models do not fit in the Pi 5's RAM and are not practical to run, even with swap.

### What is the best Gemma model size for Raspberry Pi 5?

For the 8GB Pi 5, the Gemma 3 4B QAT offers the best balance of quality and speed. For the 4GB Pi 5, the Gemma 3 1B is the only practical choice with reasonable quality. The Gemma 3n E2B is an excellent alternative that delivers 5B-model quality in just 2GB of effective RAM.

### How fast is Gemma 3 on Raspberry Pi 5 in tokens per second?

The Gemma 3 1B generates approximately 10 tokens per second on the Pi 5 8GB. The 4B QAT model produces about 5 tokens per second. The smallest 270M variant reaches roughly 22 tokens per second. These are CPU-only inference speeds since the Pi 5 has no GPU acceleration.

### What is Gemma 3n and how does it differ from Gemma 3?

Gemma 3n is a mobile-first variant of Gemma released in mid-2025 that uses Per-Layer Embeddings (PLE) to dramatically reduce RAM usage. Its 5B-parameter E2B model runs with just 2GB of effective RAM, while the 8B-parameter E4B needs only 3GB. It delivers better quality than models with similar memory footprints, making it ideal for edge devices like the Raspberry Pi 5.

### Does Raspberry Pi 5 have enough RAM for a local LLM?

The 8GB Raspberry Pi 5 has enough RAM to run small LLMs like Gemma 3 1B, 4B QAT, and Gemma 3n E2B effectively. The 4GB model is more limited, supporting only the 1B and smaller variants. For serious local AI work, the 8GB model is strongly recommended — the $20 price difference unlocks access to significantly smarter models.

### How much power does the Raspberry Pi 5 use when running a local LLM?

The Pi 5 draws approximately 7-8 watts under full CPU load during LLM inference. Running continuously for a full year, that translates to roughly 70 kWh and about $7-10 in electricity costs at average North American rates. This makes it one of the most power-efficient platforms for always-on AI inference.