Member-only story
From Post-Hoc Refusal Vectors to Native Control Planes in Foundation Model
MIT CSAIL research shows RLHF vulnerabilities can be broken in 16 steps. See why agentic AI alignment requires LLM pre-training safety controls.
TL;DR: Post-training alignment is the process of using Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) to steer a language model toward safe, helpful behavior after its base training is complete. While the industry relies on this stage as the primary safety boundary for agentic AI, research presented by Professor Dylan Hadfield-Menell at the MIT CSAIL Agentic AI Summit demonstrates that anything added during post-training can be cheaply and rapidly undone. Across recent studies, expert red-teamers bypassed frontier GPT-5 safeguards in six hours, single-vector linear algebra (“abliteration”) stripped refusal circuits without gradient updates, and just 16 gradient steps on 128 examples recovered supposedly “unlearned” hazardous knowledge. Even benign fine-tuning on narrow tasks like insecure code (
chmod 777) triggers broad "emergent misalignment" because post-training merely selects a fragile persona inside a pre-trained world simulator rather than removing underlying capabilities. Furthermore, ecological audits of medical fine-tunes reveal chaotic…
