The BrainVisionCenter Research Institute and Competence Center (BVC), in collaboration with the HUN-REN Institute of Experimental Medicine, has developed a unique virtual reality (VR) headset optimized for mice. This innovation opens new horizons in the study of brain function and the development of vision-restoring brain-computer interfaces. The device, named Moculus, is the first to enable the realistic simulation of natural vision in experimental animals. While previously it took experimental animals 5–9 days to learn to differentiate between two images, with the new device, this learning process can now be 100 times faster. This means that studying the mechanisms of vision is now possible within an hour.

Their newly developed virtual reality system, combined with rapid 3D imaging, has enabled rodents to perform fast visual learning tasks in as little as one day. Moculus provides a more realistic virtual presence for mice in the VR world through a lifelike, stereoscopic display, significantly accelerating the learning process.

Over the past 20–30 years, neuroscientists, pharmaceutical companies, and corporations have developed numerous virtual reality tools to study the vision of experimental animals. However, these tools all relied on two-dimensional projections to represent virtual spaces, assuming that experimental animals, like humans, could understand and reconstruct the surrounding 3D reality from two-dimensional projections, such as the flat image of a television screen. Recent research, however, has revealed that this assumption is incorrect. For rodents, two-dimensional projections do not provide a realistic experience, which distorts the results.

This was illustrated with a simple yet convincing example: mice ran across a virtual abyss displayed on traditional VR screens without hesitation. In contrast, they almost immediately stopped and even retreated backward when the abyss was presented using the newly developed Moculus system, as they perceived it to be realistic.

In other words, the Moculus project demonstrated that mice can only perceive the world in three dimensions if the virtual reality is precisely calculated and projected in a way tailored to their vision, replicating how they see reality. This is because, unlike humans, mice lack sufficient capacity for abstract visual thinking.

The Moculus VR system includes a specialized treadmill that records and transmits data on the mouse’s movements, two screens, and an optical imaging system designed to match them. This system provides a field of view wider than 180 degrees, which is essential for mice, allowing them to interact naturally with the virtual environment. Meanwhile, researchers can use two-photon microscopy to precisely map the brain activity patterns of the mice. Mapping these patterns offers insights into how animals learn and the neural mechanisms that regulate decision-making. These studies not only enhance our understanding of fundamental brain functions but also contribute to the development of therapeutic solutions for neurological disorders, such as vision impairments.

The essence of learning lies in the fact that these rich neuronal representations compete with one another. During learning, a kind of “competitive process” takes place in the brain among spatiotemporal neuronal representations. The goal of this competition is to encode behaviorally relevant information, such as rewards or punishments. Researchers have demonstrated that feedback derived from rewards and punishments serves as the critical information that continuously teaches and reprograms the functioning of the entire neural network at the cellular level. This process ultimately determines the “winning” representation, which dominates the encoding, while the coding of other previously competing information decreases and returns to baseline activity.

According to the current state of the literature, neuronal activity in the visual cortex increases by only about 10% as a result of learning. However, the most recent studies report even more surprising findings, including cases of 0% change in activity or even decreases in activity. It is important to note that in these earlier studies, training mice took at least 5–9 days, providing ample time for various processes to reorganize brain activity patterns.

The ultra-fast learning capability provided by Moculus has, for the first time, enabled the capture of a snapshot of brain activity before reorganization processes have begun. The results revealed that the functioning of vision differs significantly from previously established textbook data. The brain can temporarily activate nearly all neurons in the visual cortex to perform visual tasks, thereby maximizing its computational capacity to ensure the richest possible representation of the specific visual components.

The study of mice using VR headsets opens new possibilities in neurobiological research and can contribute to a more comprehensive understanding of the human brain’s functioning. The project’s most significant achievement is that the new tool generates spatiotemporal brain activity patterns that encode specific visual elements of the environment with unparalleled precision and depth. This capability allows vision restoration devices based on 3D acousto-optical microscopy to reactivate neuronal activity more accurately than ever before, enabling the creation of much more precise artificial vision.

The mouse VR system has generated significant interest in the market for neuroscience research tools, as no similar device is currently available. One of its major advantages is its compact, modular design, which allows it to be easily integrated with various electrophysiological or imaging equipment, such as two-photon microscopes.

The project is the result of a collaboration between the BrainVisionCenter, founded in 2021 by Botond Roska, Balázs Rózsa (Director), and the Ministry of Innovation and Technology, and the HUN-REN Institute of Experimental Medicine. The research was supported by grants ERC-682426 (VISIONby3DSTIM), GINOP-2016, NKP-2017, VKE-2018, GINOP-2021-00061, PM/20453-15/2020, KFI-2018-097, AMPLITUDE, GYORSÍTÓSÁV-2021-04, GINOP_PLUSZ-00143, GYORSÍTÓSÁV-2022-064, NL-2022-012, KK-2022-05, ED-2021-00190, ED-2022-00208, and Gergely Szalay’s NKFIH/143650 research fellowship.

Gergely Dobos, Balázs Rózsa, and Gergely Szalay are the inventors of the PCT/HU2020/050029 patent.

Publication: Moculus: an immersive virtual reality system for mice incorporating stereo vision

DOI: 10.1038/s41592-024-02554-6

The BrainVisionCenter Research Institute and Competence Center (BVC), in collaboration with the HUN-REN Institute of Experimental Medicine (HUN-REN KOKI), has developed a virtual reality (VR) headset optimized for mice. This groundbreaking device creates new avenues for studying brain function and advancing brain-computer interfaces aimed at restoring vision. The device, named “Moculus,” provides experimental animals with a highly realistic simulation of natural vision, enabling up to 100 times faster learning. The innovation aligns with the mission of the BVC, founded by Botond Roska and Balázs Rózsa, which focuses on developing vision-restoring therapies and treatments for central nervous system diseases. The development is the work of Linda Judák, Gergely Szalay, Gergely Dobos, and Balázs Rózsa, who examined the plasticity of the mouse visual cortex during rapid learning. The significance of their research is underscored by its publication in one of the world’s most prestigious scientific journals, Nature Methods.

The cortical representation of behavior and perception is one of the key areas of scientific research, as it forms the foundation for a deeper understanding of brain mechanisms and the discovery and development of related therapeutic strategies. To understand these phenomena at the level of individual cells, the most appropriate method is to study the brain activity of mice using real-time 3D imaging. However, during such studies, it is particularly important to keep the mouse’s head completely stable, as movement can compromise the accuracy of the results. Researchers typically achieve this by fixing the mouse’s head and employing virtual reality systems.

Over the past 20–30 years, neuroscientists, pharmaceutical companies, and corporations have developed numerous virtual reality tools to study the vision of experimental animals. However, these tools typically used two-dimensional projections to represent virtual spaces, assuming that experimental animals, like humans, are capable of reconstructing the surrounding 3D reality from two-dimensional images, such as humans do for the flat image of a television screen. Researchers from HUN-REN KOKI, BVC, the Institute of Molecular and Clinical Ophthalmology Basel, and Pázmány Péter University, however, have demonstrated that this assumption is incorrect. For rodents, two-dimensional projections do not provide a realistic experience, which can distort behavior results. This was illustrated by Gergely Dobos and his colleagues with a simple yet striking example: mice crossed a virtual abyss displayed on traditional VR screens without hesitation. In contrast, they immediately stopped and even retreated backward when the abyss was presented using the Moculus system.

“The project has proven that mice can only perceive the world in three dimensions if virtual reality is projected in a realistic way tailored to their vision. Unlike humans, mice lack sufficient capacity for abstract visual thinking, making it essential for what they see to faithfully reflect reality,” explained dr. Gergely Szalay, lead researcher at HUN-REN KOKI and BVC.

The Moculus VR system includes a specialized treadmill that records and transmits data on the mouse’s movements, two screens, and a matching optical imaging system. The latter provides the mice with a field of vision wider than 180 degrees, enabling them to interact naturally with the virtual environment. Meanwhile, researchers can map the mouse’s brain activity patterns using two-photon microscopy, which allows them to gain deeper insights into how animals learn and the neural mechanisms that govern decision-making. These studies not only advance our understanding of fundamental brain functions but also contribute to the development of therapeutic solutions for neurological disorders, such as vision impairment.

“Rodents’ visual learning abilities are surprisingly advanced. Contrary to previous assumptions, they can acquire new visual information in as little as one day, or sometimes just 30 minutes. This means rodents learn over 100 times faster with Moculus than with earlier virtual reality systems, which required 5–9 days of training. By reducing the difficulties and artifacts associated with lengthy training sessions, Moculus is revolutionizing the study of visual learning mechanisms, enabling discoveries even during a single short training session,” explained Dr. Linda Judák, lead researcher at HUN-REN KOKI and BVC. One of the device’s greatest advantage is its ability to identify complex brain activity patterns associated with learning, including so-called anticipatory neuronal signals that appear even before the presentation of visual stimuli. Dr. Linda Judák also added that during their research, they observed that neurons gradually become engaged in the learning process, with their activity intensifying particularly during the critical period preceding behavioral decisions. Using the device, they discovered new, previously unknown neural network mechanisms that emerge during visual learning.

According to current scientific literature, the activity of neurons in the visual cortex increases by approximately 10% as a result of learning. Moreover, the latest research has produced surprising findings: in some cases, a 0% change in activity or even a decrease in activity has been observed. It is important to note that in these earlier studies, training mice took 5–9 days, providing sufficient time for memory consolidation processes to reorganize brain activity patterns. Thanks to the ultra-fast learning opportunity provided by Moculus, researchers were able, for the first time, to capture a snapshot of brain activity before these reorganization processes began, directly observing the effects of learning. The results indicate that the functioning of the visual cortex differs significantly from the previously accepted textbook data. The brain is capable of activating nearly all neurons in the visual cortex for a short time to perform visual tasks, thereby maximizing its computational capacity to achieve a richer representation of visual components.

The essence of learning lies in the competition between these rich neuronal representations. During the learning process, a kind of “competitive process” occurs in the brain among spatiotemporal neuronal representations, aiming to encode behaviorally relevant information, such as positive or negative reinforcement. It has been demonstrated that the feedback derived from these signals is the critical information that teaches and reprograms the functioning of neural networks at the cellular level. This process determines the “winning” representation that dominates the encoding, while the coding of other information returns to baseline activity,” explained Dr. Balázs Rózsa, Director of BVC and Group Leader at HUN-REN KOKI and Pázmány Péter Catholic University.
He emphasized that the most significant achievement of the project is the new tool, which generates spatiotemporal brain activity patterns that encode specific visual elements of our environment with orders of magnitude greater precision and depth. This enables vision restoration devices based on 3D acousto-optical microscopy to reactivate neuronal activity with unprecedented accuracy, thereby creating much more precise artificial vision than ever before.

“Three years ago, at the end of 2021, during the establishment of the BrainVisionCenter, Botond Roska and I identified as our main mission, the basic research in vision restoration, the development of specialized research tools necessary for this, and cortical vision restoration related research. Moculus represents an important milestone in these efforts, as it allows the cortical role of the genetic engineering methods developed by Botond and his team to be tested more effectively than with previous methods,” emphasized the director.

The device has generated significant interest in the field of neuroscience research tools, as no similar solution is currently available on the market. One of its major advantages is its compact, modular design, which makes it easily adaptable to any electrophysiological or imaging equipment, such as two-photon microscopey system.