Unit 2

Deep Reinforcement Learning

Move from tables to neural function approximation, studying DQN, policy gradients, PPO, exploration, safety, and generalization.

Chapter 1

Value-Based Deep RL

Extend value-based learning with neural networks, then examine the stabilizing tricks and failure modes that define DQN-style agents.

Chapter 2

Policy Gradients and PPO

Optimize policies directly, connect policy gradients to actor-critic methods, and study why PPO became a practical default for many RL systems.

Chapter 3

Safety, Generalization, and Exploration

Study how agents discover useful behavior, fail under distribution shift, and require evaluation beyond training return.