Generalization Failures
High training return can hide a narrow solution. An agent may memorize a layout, exploit one seed, depend on exact physics, or learn a shortcut that only works because the reward function has a quirk.
Generalization asks a stronger question: did the policy learn a reusable strategy? For a door-key environment, that might mean navigating to the key, opening the door, and reaching the goal across new layouts, sizes, starts, and dynamics.
Evaluation design
What would convince you it learned the task?
Toggle held-out conditions for a trained agent. Each one asks a different generalization question.
robustness evidence
64%
This split can catch common generalization failures while staying interpretable.
Good evaluation splits isolate what changed. A held-out seed split asks whether the agent survives ordinary randomness. A held-out layout split asks whether it learned the task structure. A dynamics perturbation asks whether the policy is robust to different transition behavior.
Training diversity can help, but more training steps alone are not the same as better generalization. Replaying the same narrow experience stream longer can make brittle behavior more confident.
Longer Training Is Not a Robustness Strategy
If the environment is too narrow, extra optimization can deepen overfitting. Robust RL usually needs diverse starts, held-out layouts, domain randomization, curriculum design, or explicit stress tests.
Which evaluation best tests whether a gridworld agent learned a reusable strategy rather than one training instance?