Deepak Upman·May 24Normalization & Activation Functions in Modern Transformer Architectures — Part 1Intuitive Overview
Vishal Lakshmi Narayanan·Dec 5, 2025How Modern LLMs Understand Context: Inside RMSNorm, SwiGLU, RoPE & Next-Gen AttentionWe often talk about large language models as if they understand. We say they follow context. We say they remember previous conversation…
InData And BeyondbyML Point·Oct 15, 2025Deconstructing Attention: Investigating Design Principles for Effective Language Modeling (Paper…Attention mechanisms remain the backbone of modern Transformer-based language models, enabling them to capture complex relationships…
Al'amin ai·Sep 18, 2025Building Qwen3 from Scratch: A Deep Dive into Modern Transformer ArchitectureHow I built a state-of-the-art language model from the ground up, implementing cutting-edge techniques like GQA, RMSNorm, SwiGLU, and the…
InFoundation Models Deep DivebyM·Jun 18, 2025RMSNorm: Fueling the Next Generation of LLMsIn the noisy world of AI breakthroughs, RMSNorm is the quiet engine that keeps the revolution running — proof that elegance and efficiency…
Hugman Sangkeun Jung·Dec 9, 2024[Hands-On] Mastering LLama — LLama1 바닥부터 구현하기(2/3)라마1 구축하기: 계층 정규화, RMS 정규화, 순방향 신경망에 대한 심층 분석
Hugman Sangkeun Jung·Dec 9, 2024[Hands-On] Mastering LLama — Implementing LLama1 from Scratch (2/3)Building LLama1: Deep Dive into Layer Normalization, RMSNorm, and Feed-Forward NetworksA response icon1A response icon1
VectorWorks Academy·Nov 27, 2024Ace AI Interview Series 8— What is the Common Normalization Method in LLM Training?For the complete list of AI interview prep topics, check out the Table of Contents for this comprehensive AI interview prep series …
Hugman Sangkeun Jung·Oct 29, 2024Mastering LLama — Layer Normalization 과 RMSNorm 이해하기LLama 아키텍처의 RMSNorm: 고급 정규화 기법 이해하기A response icon1A response icon1
Hugman Sangkeun Jung·Oct 29, 2024Mastering LLama — LayerNormalization and RMSNormUnderstanding RMSNorm: Advanced Normalization in LLama Architecture
![[Hands-On] Mastering LLama — LLama1 바닥부터 구현하기(2/3)](https://miro.medium.com/v2/resize:fill:160:106/1*mByyRwFnt3NiFFIdYSN1OA.png)
![[Hands-On] Mastering LLama — LLama1 바닥부터 구현하기(2/3)](https://miro.medium.com/v2/resize:fill:320:214/1*mByyRwFnt3NiFFIdYSN1OA.png)
