AIwithMaha·Sep 21How vLLM Serves Thousands of Requests: PagedAttention and Continuous BatchingThis post is for beginners. If you’ve read my KV cache post, you already know half of what you need. If you haven’t, don’t worry, I’ll…
AIwithMaha·Sep 11FP32, FP16, BF16, and QuantizationHow many bits does a number need? That question is the whole idea behind fp32, fp16, bf16, and quantization. More bits make a number more…
AIwithMaha·Aug 17KV Cache From Scratch: Prefill and Decodellms write one word at a time so to pick each new word, the model looks back at everything written so far. So the model recomputes a huge…
AIwithMaha·Jun 10Tiled Matrix Multiplication in CUDAThis post is for beginners. If you already know more advanced GPU techniques, you can skip it. But if you are new to GPU programming, start…
AIwithMaha·May 13Stop Waiting to Feel MotivatedMost people are waiting to feel motivated before they start. They want to learn something new, build a habit, or finish a project. But they…
AIwithMaha·Apr 16Top 5 Mistakes Developers Make When Building LLM AppsYou have built a few LLM projects maybe a chatbot for a college assignment, a summarizer for a hackathon, or a side project you pushed to…
AIwithMaha·Apr 11Stop Using To-Do Lists. Put Everything on Your Calendar InsteadWe all have a to-do list. Some of us have three.And every night, we look at that list and feel the same thing, guilt. Half the things…
AIwithMaha·Apr 11Anthropic Just Made It Way Easier to Build AI AgentsBuilding an AI agent sounds exciting. But anyone who has actually tried to ship one knows the truth most of the work has nothing to do with…
AIwithMaha·Apr 120 CUDA Concepts Explained SimplyYou train a model, move it to device = “cuda” , and suddenly everything runs faster. It works. But most of us don’t really know what…A response icon1A response icon1
AIwithMaha·Mar 26Making Transformers Faster: GPU Memory Optimization for Matrix MultiplicationEvery time a large language model generates a token, thousands of matrix multiplications execute on the GPU. The difference between a model…