A neural active inference model of perceptual-motor learning
People intercepting a moving target usually steer by what they see right now, but some situations require planning ahead. In an earlier human study, participants controlled their speed along a straight path to catch a target that usually sped up partway through each trial, and they learned to speed up before the change happened. This paper asks whether active inference, in which an agent picks actions that minimize expected free energy (a mix of progress toward preferred outcomes and information gained), produces the same anticipation.
The agent is one neural network with two output heads. One head estimates the expected free energy of each pedal position and the agent picks the best one; the other predicts the next observation, and its prediction error drives exploration. A prior function turns each observation into a score of how close the agent is to the speed it needs to intercept, which supplies the goal-seeking signal without a hand-crafted reward.
Main results
- With both the goal-seeking and the exploration terms, the agent reached about a 90% interception success rate, while a DQN baseline trained on the sparse end-of-episode reward averaged 22% at test time.
- With a discount factor of 0.99 the agent sped up before the target changed speed, as human participants did; with a discount factor of 0 it only reacted.
- Making the simulated vehicle less responsive lowered success but moved the agent's timing closer to the target's actual time to contact.
- Sharing one network between the two heads used 66.8% of the parameters of two separate networks.