Deepak Upman·May 24Normalization & Activation Functions in Modern Transformer Architectures — Part 1Intuitive Overview
Vishal Lakshmi Narayanan·Dec 5, 2025How Modern LLMs Understand Context: Inside RMSNorm, SwiGLU, RoPE & Next-Gen AttentionWe often talk about large language models as if they understand. We say they follow context. We say they remember previous conversation…
InData And BeyondbyML Point·Oct 15, 2025Deconstructing Attention: Investigating Design Principles for Effective Language Modeling (Paper…Attention mechanisms remain the backbone of modern Transformer-based language models, enabling them to capture complex relationships…
Al'amin ai·Sep 18, 2025Building Qwen3 from Scratch: A Deep Dive into Modern Transformer ArchitectureHow I built a state-of-the-art language model from the ground up, implementing cutting-edge techniques like GQA, RMSNorm, SwiGLU, and the…
InFoundation Models Deep DivebyM·Jun 18, 2025RMSNorm: Fueling the Next Generation of LLMsIn the noisy world of AI breakthroughs, RMSNorm is the quiet engine that keeps the revolution running — proof that elegance and efficiency…
