Chapter 1

Human-in-the-Loop RL

Study how humans and AI raters shape RL systems through preference data, constitutions, learned reward models, disagreement-aware training, and safety evaluation.

• 1. The Human Feedback Loop→
• 2. The RLHF Pipeline→
• 3. Bradley-Terry Reward Model→
• 4. Disagreement is Signal→
• 5. RLAIF and Constitutional AI→
• 6. Evaluation and Safety Audits→