I have implemented Q learning on a grid of size (n x n) with a single reward of 100 in the middle. The agent learns for 1000 epochs to reach the goal by the following agency: He chooses with probability 0.8 the move with the highest state-action-value and chooses a random move by 0.2. After a move the state-action value is updated by the Q learning rule.
Now I did the following experiment: All fields next to the goal got a reward of -100 except the neighbour at the bottom. After learning for 1000 epochs the agent clearly avoids going the top way and arrives at the goal from the bottom most frequently.
After learning set the reward of the bottom neighbour to -100 and the top neighbour back to 0 and start learning again for 1000 epochs while sticking with the state action value map. It's actually horrible! The agent needs very long to find the goal (on a 9x9 grid up to 3 minutes). After checking the paths I've seen that the agent spends a lot of time bouncing between two states like (0,0)->(1,0)->(0,0)->(1,0)...
It is hard for me to imagine if this behaviour makes any sense. Has someone experience with a situation like this?