Dr. Narayan Srinivasa·Jan 29Stable Steps in a Chaotic World: Decoding Proximal Policy OptimizationReinforcement learning is full of beauty and frustration. At its core is a simple promise: an agent learns by acting, failing, adapting —…
Dr. Narayan Srinivasa·Jan 19Credit with Critique: Why Actor–Critic Changes EverythingREINFORCE is a beautiful idea. Strip reinforcement learning down to its essence, and you arrive at something almost disarmingly simple…
Dr. Narayan Srinivasa·Jan 9Credit Without Critics: How REINFORCE Teaches Policies What to DoIn reinforcement learning, progress often comes from asking the same question in slightly different ways — each time under fewer…
Dr. Narayan Srinivasa·Dec 27, 2025The Deadly Triad: Why Off-Policy Learning Fails Under Approximation1. When Tables Break, Assumptions Break Too
Dr. Narayan Srinivasa·Dec 19, 2025When Tables Break: Reinforcement Learning Beyond GridworldsWhy Function Approximation Becomes Inevitable in Real-World RL
Dr. Narayan Srinivasa·Dec 7, 2025Breaking the Tyranny of the Timestep: How n-Step TD Transforms RL Prediction and ControlIn the last two articles of this series, we explored an elegant idea: reinforcement learning (RL) doesn’t need a model of the world to…
Dr. Narayan Srinivasa·Dec 4, 2025No Model, No Problem — Now with Real-Time Control: On-Policy SARSA and Off-Policy Q-LearningHow TD Control Teaches Agents Not Just to Learn, but to Act in Real-Time
Dr. Narayan Srinivasa·Nov 28, 2025Where Monte Carlo Meets Dynamic Programming: The Magic of Temporal-Difference LearningIf Monte Carlo (MC) methods are the patient storytellers of reinforcement learning — waiting until an episode ends before revealing the…
Dr. Narayan Srinivasa·Nov 21, 2025No Model, No Problem, Part II: Off-Policy Methods in Reinforcement LearningThis is Part II of my exploration of Monte Carlo methods in reinforcement learning. If you missed Part I, where we covered on-policy…
Dr. Narayan Srinivasa·Nov 19, 2025No Model, No Problem, Part I: On-Policy Monte Carlo in Reinforcement LearningThis is the fourth article in my Reinforcement Learning series. Catch up here: