Rohit Jindal·Oct 24CUDA Matrix Multiplication — Part 5In the previous blog, we learned how to introduce our first CUDA optimization using shared memory to reduce repeated accesses to global…A response icon1
Aniket Maheshwari·Oct 21Naive attention from scratch — the memory wall FlashAttention removesStandard attention has a memory problem, and the cleanest way to understand FlashAttention is to first build the version that has it.A response icon2
The Opinion Board·Oct 7Classical DSA Is Killing Your GPU Performance and Nobody Is Talking About ItThe data structures your computer science degree worshipped are architectural catastrophes on modern GPU hardware — and the industry is…
Debasish Mohanty·Oct 6Why Attention Is Slow: Profiling a CUDA Kernel from 4.66 ms to 2.57 ms on a Tesla T4What shared-memory tiling fixed, what it didn’t, and what an LLM inference engineer should take from it.
Inazhar labsbyazhar·Oct 3GPU Kernel Engineering From First PrinciplesWhat really happens between one line of Python and thousands of GPU threads
