Artificial Intelligence

Machine Learning

Large Language Models

Mixture Of Experts

Explore recommended reads, summarized by AI

Technical Architecture

Explorations of how Mixture of Experts systems are designed and built.

DeepSeek-V3 Architecture

MoE Fundamentals and Implementation

Popular this week

Pankaj

Colibri Runs a 744B Model on 25 GB of RAM. The Real Breakthrough Is Weight Streaming

In

Towards AI

by

Silasyl

Inside the Architectures of Open-Source LLMs: Attention, MoE, Hybrid Layers and Quantization

In

Towards AI

by

Artha Mukherjee

A GPU-Poor’s Guide to Local LLM Inference in 2026

Nayeem Islam

The What, Why, and How of Mixture of Experts (MoE)

Tahir

Kimi K3 AI Model Architecture Breakdown

Latest stories

Carl Johan Rehn

Demystifying Top-k Routing: Why Hard Selection Is Not Just a Heuristic

Raghu Juluri

Colibrì Is Not an AI Project. It’s a Storage Engine.

In

Towards AI

by

Silasyl

Inside the Architectures of Open-Source LLMs: Attention, MoE, Hybrid Layers and Quantization

Abhinav Dobhal

Colibri Runs 744B-Parameter MoE Models on the PC You Already Own by Streaming Experts From NVMe

Billy Toth

Why All-to-Alls are Important for MoEs

See more stories

Who to follow

Popular this week

Pankaj

Colibri Runs a 744B Model on 25 GB of RAM. The Real Breakthrough Is Weight Streaming

In

Towards AI

by

Silasyl

Inside the Architectures of Open-Source LLMs: Attention, MoE, Hybrid Layers and Quantization

In

Towards AI

by

Artha Mukherjee

A GPU-Poor’s Guide to Local LLM Inference in 2026

Nayeem Islam

The What, Why, and How of Mixture of Experts (MoE)

Tahir

Kimi K3 AI Model Architecture Breakdown

Latest stories

Carl Johan Rehn

Demystifying Top-k Routing: Why Hard Selection Is Not Just a Heuristic

Raghu Juluri

Colibrì Is Not an AI Project. It’s a Storage Engine.

In

Towards AI

by

Silasyl

Inside the Architectures of Open-Source LLMs: Attention, MoE, Hybrid Layers and Quantization

Abhinav Dobhal

Colibri Runs 744B-Parameter MoE Models on the PC You Already Own by Streaming Experts From NVMe

Billy Toth

Why All-to-Alls are Important for MoEs

See more stories

Who to follow