Baicen Xiao·Sep 27, 2025Hands-On LLM Alignment: Coding GRPO from Scratch, Step by StepHands-on GRPO from scratch: vLLM + HF Transformers to finetune Qwen2.5-Math-1.5B. Train and roll-out split on 2x24G GPUs for efficiency.A response icon1A response icon1
Baicen Xiao·Aug 23, 2025Avoiding Amnesia: Some Practical Guides to Mitigate Catastrophic Forgetting in LLMs Post-trainingPost-training is how we turn base LLMs into specialists — but it often degrades previously learned skills. Here are methods to mitigate.
Baicen Xiao·Feb 13, 2025Why is RLHF necessary when we can do SFT in tuning LLMs?SFT makes LLMs follow instructions, while RLHF refines quality, safety, and alignment with human preferences at potentially lower cost.
Baicen Xiao·Aug 18, 2024RLHF vs. DPO: Choosing the Method for LLMs Alignment TuningComparison of DPO vs RLHF for LLM alignment tuning: how DPO simplifies RLHF in LLMs alignment, plus pros and cons of both methods.
Baicen Xiao·Jul 1, 2024Strategies for Balancing Multiple Loss Functions in Deep LearningThis article introduces methods for balancing multiple losses (objectives) in deep learning with some PyTorch codes for better…A response icon3A response icon3
Baicen Xiao·Apr 28, 2024Understand Classifier Guidance and Classifier-free Guidance in diffusion models via Python…We introduce conditional controls in diffusion models in generative AI, which involves classifier guidance and classifier-free guidance.A response icon1A response icon1
Baicen Xiao·Mar 10, 2024An Overview of Large Multi-modal Models (LMMs): Part 1We will walk through some of the most prominent large multi-modal models including CLIP, ALBEF, COCA, VLMo, BLIP, and BEITv3.A response icon1A response icon1
Baicen Xiao·Feb 26, 2024Hands-on: Mixture of Experts with TransformersIntroduce a simple way of creating the Mixture of Experts (MoE) framework with TransformersA response icon2A response icon2
Baicen Xiao·Jan 14, 2024Some basic knowledge of LLM: Parameters and Memory EstimationHow to estimate memory usage for LLMA response icon2A response icon2
Baicen Xiao·Jan 7, 20249. How to make CoreML model updatable.The goal of this article is to provide hands-on experience on how to make CoreML models updatable on devices.