Alex Xiaoli Shen·Nov 17, 2025Hands-On Transformer Deep Dive: Part 4— YaRN (Yet another RoPE extensioN)This is Part 4 of the “Hands-on Transformer Deep Dive” series. We’ll dive deep into context length extension with RoPE, esp. YaRN.
Alex Xiaoli Shen·Sep 15, 2025Hands-On Transformer Deep Dive: Part 3— Positional Encoding, RoPE & YaRNThis is Part 3 of the “Hands-on Transformer Deep Dive” series. We dive deep into position encoding, implement Sinusoidal Encoding and…
Alex Xiaoli Shen·Aug 5, 2025Hands-On Transformer Deep Dive: Part 2 — Multi-head Attention Variants with CodeWhat exactly are the multi-head attention variants MHA, MQA, GQA, and MLA? Follow along to understand and implement them all.
Alex Xiaoli Shen·Aug 1, 2025Hands-On Transformer Deep Dive: Part 1 — Masked Attention Explained & ImplementedIn this “Hands-On Transformer Deep Dive” series, we go step-by-step through the algorithms and components of modern Transformers, with…A response icon3A response icon3
Alex Xiaoli Shen·Jul 10, 2023A Fullstack Text-to-Image Generative AI App You Can Build in an AfternoonAs a fun project for a hot summer weekend, I built a minimal fullstack text-to-image Generative AI web app with the Stable Diffusion model…
IncodeburstbyAlex Xiaoli Shen·Dec 10, 2020Migrate PostgreSQL on AWS from RDS to AuroraThis is a story about rescuing a not-so-old system from the legacy architecture design which no longer suits its global requirements.
Alex Xiaoli Shen·Apr 26, 2020The DNS Lookup Journey…Or one answer to the famous “what happens when you type an URL in your browser address bar and hit the Enter key” question.A response icon1A response icon1
Alex Xiaoli Shen·Jan 3, 2019My Docker CheatsheetAfter lots of fumbling and rummages I’ve finally compiled my own Docker cheatsheet.
Alex Xiaoli Shen·Nov 10, 2018A Simple Friendly Story of Go PointersScary stories have been told about the pointers in C/C++. The pointers in Go, however, are just simple and friendly: two usages of the…
Alex Xiaoli Shen·Nov 8, 2018Build a Create-React-App (ejected) for stagingIn my work, there are environment variables predefined by the DevOps team which must obey certain company-wide rules. The implication: no…