Arun Taneja·4d agoLLM Inference on a GPU, Explained from ScratchA beginner’s tour of what really happens inside a GPU when a language model answers you, one idea at a time.
Waizkhan·Nov 2A Deep Dive into Two Philosophies of AI Inference OptimizationA few weeks ago, I thought I had a fairly simple picture of AI inference optimization: quantize models, write faster kernels, and use the…
InOpen IntelligencebyDr. Fadi Shaar·Oct 24Uzu vs. llama.cpp: How One Inference Engine Reached 92 Tokens per Second on Apple SiliconA reported benchmark on a base M5 MacBook Pro puts Uzu far ahead of MLX and llama.cpp on Qwen3.5–9B, raising questions about what optimized…
Shivank Chaudhary·Oct 178x Faster LLM Cold Starts on Kubernetes With NVIDIA Dynamo Snapshot, your guide to scale to 0 and…Testing NVIDIA Dynamo Snapshot on H200s: the numbers, what broke, and where it stops helping.
