HMN 2026: How AI turns mouse movements into language-like tokens, uncovering autism-related social behavior patterns

An artificial intelligence model capable of reading and interpreting animal behavior like language has been developed by researchers at KAIST. The team created an AI model that learns behavioral data in a manner similar to natural language and was able to independently identify social behavioral deficits in an autism mouse model, opening a new avenue for interpretable neuroscience.

A research team led by Professor Dae-Soo Kim from the Department of Brain and Cognitive Sciences has developed an AI model that interprets animal movements as a form of behavioral language. The study is published in the International Journal of Computer Vision (IJCV).

The researchers transformed skeletal movements of mice into tokens, analogous to words in natural language, and trained a transformer-based model to learn behavioral meaning. The resulting model, named BehaVERT, successfully identified core social behavioral abnormalities in an autism mouse model without being provided any prior biological knowledge.

The study introduces a novel AI framework for analyzing animal behavior through language-based representations. Beyond simple behavior classification, the model demonstrates the ability to uncover biologically meaningful patterns and may serve as a foundation for next-generation behavioral foundation models applicable to drug discovery, psychiatric research and behavioral genetics.

Behavior as a sequence of tokens

Inspired by the idea that animal behavior may possess structures similar to language, the researchers represented the positions of a mouse’s nose, ears, spine, limbs and tail as behavioral tokens and trained a BERT-based transformer architecture.

As a result, BehaVERT learned not only to classify behaviors but also to understand their contextual meaning over time, much like language models infer meaning from sequences of words.

The model achieved state-of-the-art performance across five international benchmark data sets covering social interaction, multi-animal behavior, three-dimensional motion analysis and autism-related behavioral assessment.

Importantly, BehaVERT also provides interpretability, allowing researchers to visualize which behavioral cues influenced its decisions.

Autism-related cues emerged on their own

In experiments distinguishing Shank3B knockout autism-model mice from healthy controls, the AI consistently focused on oral-oral contact behavior. This finding aligns with previous biological studies showing that autism-model mice exhibit deficits in social interaction despite maintaining normal approach behavior.

In other words, the AI independently rediscovered a key biological characteristic solely from behavioral observations, without explicit biological instruction.

A semantic map of behavior

The researchers further found that the model’s internal representation space organized behavioral features such as mobility, attention and social engagement into structured patterns. This suggests that animal behavior, much like language, may possess an underlying semantic structure.

The study also highlights an unusual interdisciplinary achievement. The first author, Dr. Seungjae Shin, and other members of the research team were trained primarily in biology rather than AI. By independently learning transformer architectures and deep learning techniques, they designed specialized models and training strategies tailored for behavioral analysis.

Kim’s laboratory has long pursued AI-driven behavioral analysis and previously developed AVATAR, a technology that reconstructs rodent behavior in virtual environments, leading to the founding of Actnova Inc.

“The project began with a simple question: Could animal movements contain a structure similar to language?” said Dr. Seungjae Shin, the first author of the study.

Learning across species without labels

The team also adopted a self-supervised learning framework that enables AI to learn directly from behavioral data without manual annotations. Furthermore, a model trained on rat behavior successfully transferred to mouse behavior analysis, demonstrating the feasibility of a behavioral foundation model applicable across species.

“BehaVERT goes beyond behavior classification and enables the interpretation of behavioral meaning,” said Professor Dae-Soo Kim. “We expect it to become a key research tool for discovering new insights in drug development, psychiatric disorders, behavioral genetics and many other areas of life sciences.”

This content is provided for information purposes only.