Shajia Siddiqa·Jul 17Rotary Positional Embeddings (RoPE): The Mathematical Trick Behind Modern LLMsHow models like LLaMA, Mistral, Gemma, and Qwen understand the order of words without learning positional embeddings.
Ahtesham Ul Haq·Jul 11Positional Encodings in LLMs. The full story (from integer counting to RoPE)I am writing this article b/c positional encoding is one of those topics which looks very scary if you directly open the “Attention is All…
YouShin kim·Apr 16[arXiv] — TriAttention: Efficient Long Reasoning with Trigonometric KV Compression본 보고서는 대규모 언어 모델(LLM)의 긴 문맥 추론 과정에서 발생하는 KV 캐시(Key-Value Cache) 메모리 병목 현상을 해결하기 위해 제안된 TriAttention 방법론에 대한 기술적 분석을 담고 있습니다. 해당 연구는 기존의 사후…
YouShin kim·Apr 14[arXiv] — TriAttention: Efficient Long Reasoning with Trigonometric KV Compression본 보고서는 대규모 언어 모델(LLM)의 긴 문맥 추론 과정에서 발생하는 KV 캐시(Key-Value Cache) 메모리 병목 현상을 해결하기 위해 제안된 TriAttention 방법론에 대한 기술적 분석을 담고 있습니다. 해당 연구는 기존의 사후…
Mustafa Genc·Mar 20Transformers & LLMs — Part 13: KV Cache, PagedAttention, and MLAThe Engineering Behind Fast LLM Inference
Mustafa Genc·Feb 9Transformers & LLMs — Part 5: Rotary Positional EmbeddingsMain Source: Stanford CME295: Transformers & LLMs | Autumn 2025
Anas Limem·Nov 16, 2025Positional Encoding Explained: How Transformers Understand Word OrderHave you ever wondered how a Large Language Model (LLM) knows the order of words after it turns them into vectors?A response icon1A response icon1
Farasatul Adnan·Jul 4, 2025Understanding DeepSeek Architecture: Rotary Positional Embeddings (RoPE)*** I wrote this blog for my own edification***
InGoPenAIbyArXiv In-depth Analysis·Feb 28, 2025NeoBERT: Advancing Encoder Models in NLP, A Next-Generation BERTIntroduction
Hugman Sangkeun Jung·Dec 10, 2024[Hands-On] Mastering LLama — LLama1 바닥부터 구현하기(3/3)바닥부터 만드는 라마: Multihead attention, Multi-Layer 및 구현과 검증
![[Hands-On] Mastering LLama — LLama1 바닥부터 구현하기(3/3)](https://miro.medium.com/v2/resize:fill:160:106/1*WJOS4f_Or6xH8bD3bZqIjA.png)
![[Hands-On] Mastering LLama — LLama1 바닥부터 구현하기(3/3)](https://miro.medium.com/v2/resize:fill:320:214/1*WJOS4f_Or6xH8bD3bZqIjA.png)