HMN 2026: How The timing of rewards plays a key role in learning,

For almost a century, psychology and neuroscience researchers have been trying to understand the processes via which humans and other animals acquire new skills or learn to deal with specific situations. One well-known and widely documented type of learning occurs when specific events are followed by rewards or punishment.

In these scenarios, the brain learns to associate specific signals or cues with a good or bad outcome. Past findings have shown that the neurotransmitter dopamine plays a central role in this process, by providing a teaching signal to indicate the anticipation of a positive experience and increase motivation to pursue rewards.

Most psychological theories introduced to date suggest that reward-based learning is a gradual process, and every time one experiences a reward after a specific action or stimulus, the brain slightly “adjusts” its expectations for future rewards. This process was so far believed to occur irrespective of the time that passes between one reward and the next.

Researchers at University of California–San Francisco and University of California–Berkeley recently carried out a mouse study challenging this assumption, suggesting that the strength of reward-based learning also depends on the timing between rewards and not just on how many times a mouse is rewarded after the same stimulus. Their paper, published in Nature Neuroscience, could reshape existing models of learning, decision-making and potentially even addiction.

“How does the brain learn that a cue predicts a reward?” said Vijay Mohan K. Namboodiri, senior author of the paper, speaking to Medical Xpress. “We had previously published a paper proposing that instead of directly learning predictions, animals learn whether cues systematically precede rewards. Briefly, our idea was that instead of learning the future effects of possible cues, animals would instead look back in their memory to identify the causes of meaningful effects, i.e., learn cause-effect relationships by looking backwards in time from effects.”

Shortly after they published this previous paper, Namboodiri and his colleagues realized that their findings might also suggest that animals learn associations between cues and rewards faster when experimental trials are spaced out over time. This would entail that for a fixed time, the animals learn from rewards, irrespective of how many cue-reward pairings they experienced.

“In our first attempt to test this prediction, Dr. Burke ran a few animals with a 10-minute separation between cue-reward pairings,” said Namboodiri. “These animals learned much faster than our usual animals, where we use one-minute separation between cue-reward pairings. This early result showed us that there is something interesting there. We thereafter set out to test whether there is a rule governing learning rate control and whether the learning rate scales proportionally with the time between cue-reward experiences.”

Exploring the impact of timing in reward-based learning

The main objective of the team’s recent study was to explore the possibility that reward-based learning is driven by the time between one reward and the next, as opposed to by the number of cue-reward pairings experienced by animals. To do this, they carried out a series of experiments involving adult mice.

The mice were trained on simple Pavlovian conditioning tasks. These are tasks in which an animal hears a brief tone and then receives a reward (i.e., some sugar water). In their experiments, Namboodiri and his colleagues carefully manipulated the number of cue-reward pairings that the animals experienced and how far apart they were in time.

“Over the same total training duration (for example, one hour), we compared animals that experienced many closely spaced cue–reward pairings (e.g., 120) with animals that experienced very few, widely spaced pairings (e.g., six),” explained Namboodiri. “Behaviorally, we quantified learning by measuring conditioned responses to the cue—specifically, how quickly animals started to lick on a tube in anticipation of the upcoming sugar solution.”

Conventional psychological models suggest that more cue-reward pairings within the same time frame will result in stronger learning, which entails mice looking for the sugar water as soon as they hear the familiar sound. Surprisingly, however, the researchers found that after a fixed period, all mice had learned the association well, regardless of whether they had experienced a few or several pairings.

“The field had known that spreading pairings out in time speeds up learning per pairing, but it was still assumed that the final level of learning depended on the total number of pairings,” said Namboodiri. “Our experiments showed that instead, total learning is determined by time, not count.”

To further test their hypothesis, the researchers carried out a series of additional tests in which they manipulated the intervals between cues, rewards, and cue-reward pairings.

“This allowed us to quantify how learning changed as a function of these temporal parameters,” said Namboodiri. “Across these experiments, the results were consistently explained by a simple mathematical rule. The amount of learning from each cue–reward experience is proportional to the average time between rewards.”

Namboodiri and his colleagues finally tried to shed light on the biological and neuroscientific underpinnings of the time-dependent learning mechanism they uncovered. To do this, they measured dopamine signals in a brain region associated with reward-learning and pleasure-seeking behavior called the nucleus accumbens.

The team monitored these dopamine signals using genetically encoded fluorescent dopamine sensors. They tracked them while the mice were exposed to cues and as they gradually learned to associate these cues with rewards.

“We found that dopamine signals followed the same learning rule: The rate of change in dopamine cue responses depended on the average time between rewards, not on the raw number of cue–reward pairings,” said Namboodiri. “This parallel between behavior and dopamine activity shows that the brain’s reward system implements a time-based learning rule, revealing a simple biological underpinning for how animals learn from rewards.”

Revising conventional models of learning

The recent findings gathered by Namboodiri and his colleagues could have important implications for neuroscience and psychology research. If validated in further experiments, they could redefine the current understanding of reward-based learning, informing the creation of new theoretical models.

In the future, this study could potentially also inspire the creation of new approaches to train artificial intelligence (AI) agents, which take the timing between rewards into account to learn from a few experiences. Finally, it could help to shed more light on some of the underpinnings of addiction and substance-use disorders (SUDs), as well as other mental health disorders associated with unhelpful reward-seeking behaviors.

“Our work could help to better understand diseases that involve dopamine-mediated learning, such as addiction, and provide important rules for optimizing learning for both humans and artificial intelligence systems,” added Namboodiri. “Our current follow-up studies are aimed at understanding where in the brain the duration is between rewards calculated to drive this learning rate control, and whether this rule extends to drug rewards or other forms of learning.”

The content is provided for information purposes only.