Chapter 2

Policy Gradients and PPO

Optimize policies directly, connect policy gradients to actor-critic methods, and study why PPO became a practical default for many RL systems.

• 1. Policies as Trainable Objects→
• 2. REINFORCE Intuition→
• 3. Actor-Critic Methods→
• 4. TRPO→
• 5. PPO→
• 6. SAC→
• 7. Chapter Debrief→