Intended Goals vs. Specified Rewards
Reward design starts with a gap. The intended goal is what the designer wants. The specified reward is the scalar feedback the agent actually receives. Reinforcement learning optimizes the specified reward.
This is why reward design is a core modeling choice. If the reward measures the wrong thing, a capable agent may become very good at the wrong behavior. A cleaning robot rewarded for moving objects may move clean objects forever. A content system rewarded for short-term clicks may learn sensationalism while usefulness suffers.
Reward specification
High reward can diverge from success
Keep the intended goal fixed: clean the room. Change only the specified reward and inspect the behavior it encourages.
Intended goal
Leave the room cleaner than it started.
Specified reward
+1 each time any object is moved
The robot moves clean objects around because every movement pays.
Total reward
high
Actual cleanliness
low
The key diagnostic question is: what behavior would score well under this reward even if the real goal failed? If that behavior exists, the reward is a loose proxy for the intended goal.
Good reward design often requires extra metrics beyond the training reward. These metrics act like smoke detectors: they tell you when reward is rising while the actual task stalls.
Reward Is Feedback
The agent optimizes the feedback channel you built. Misaligned feedback creates misaligned behavior.
A cleaning robot gets every time it moves an object. What is the most likely reward-design problem?