Choice history effects in mice and humans improve reward harvesting efficiency

In foraging for food and other resources, animals have to make repeated choices about where best to search. Experimental studies of such decisions typically work with the simplifying assumption that animals’ past decisions should have no impact on the distribution of actual resources, which remains fixed. Many such studies have nevertheless found that animals’ current decisions do often depend on their past decisions, and interpret this as a bias or deviation from optimal behaviour. After all, such behaviour typically degrades the rate of obtaining rewards in these simplified contexts.
But animals actually living in nature face fluctuating conditions, complex environments, and the constant need to forage for sources of food in competition with other animals. Doing this successfully demands flexibility. An animal might just recall which patches have yielded the best foraging results in the recent past, and forage there again. Yet food can become depleted in over-foraged patches, and grow more abundant in other relatively untouched patches. Successful foraging means taking such factors into account. Among other things, this implies that animals’ current choices may well depend on the history of their past choices. Animals should keep in memory not only which food patches provided the best food quality, but also which food patches they have visited recently.
In a recent paper, LML External Fellow Oliver Hulme and colleagues examine this issue in experiments in which they let the reward probabilities change as a function of an animal’s past choices. In this case, the probability of earning a reward becomes a function of the number of trials that have elapsed since that option was last chosen. In this case, the best strategy for maximizing the reward rate over all of the available options depends on the history of how recently each option has been chosen in the past. To maximize their reward rate, optimal agents should choose the options with the highest set reward probability and switch when the reward probability of the other unchosen options overtakes that of the initial option.
In their experiments, Hulme and colleagues evaluated whether the animals – both humans and mice – did follow optimal foraging principles in this more complex and variable setting. They found that both mice and humans did base their decisions on past choice history, and thereby achieved a superior reward harvesting efficiency. As they note, this behaviour also accords well with other published data from monkeys. This evidence provides an explanatory account of choice history effects as being well attuned to the task, and not reflecting any underlying behavioural bias.
A pre-print of the paper is available at https://www.researchgate.net/publication/355053445_Choice_history_effects_in_mice_and_humans_improve_reward_harvesting_efficiency