Exploration vs. Exploitation

A Q-table starts with guesses. If the agent always chooses the action with the highest current Q-value, an early lucky or unlucky sample can lock behavior into a bad pattern. The agent may never try the action that would have been better.

Exploitation uses what the agent currently believes is best. Exploration tries actions to gather information. RL agents need both: exploiting too early can trap learning, but exploring forever ignores useful evidence.

A common compromise is an $\epsilon$-greedy policy:

$$a_t=\begin{cases}\text{random action}, & \text{with probability } \epsilon \\ \arg\max_a Q(s_t,a), & \text{with probability } 1-\epsilon.\end{cases}$$

When $\epsilon=0$, the agent is fully greedy. When $\epsilon=1$, every action is random. Most practical settings use something in between, often with $\epsilon$ changing over training.