"We used two kinds of data to build NeuroMechFly," says Professor Pavan Ramdya at EPFL's School of Life Sciences. "First, we took a real fly and performed a CT scan to build a morphologically realistic biomechanical model. The second source of data were the real limb movements of the fly, obtained using pose estimation software that we've developed in the last couple of years that allow us to precisely track the movements of the animal."

Ramdya's group, working with the group of Professor Auke Ijspeert at EPFL's Biorobotics Laboratory, has published a paper in Nature Methods showcasing the first ever accurate "digital twin" of the fly Drosophila melanogaster, dubbed "NeuroMechFly".

Time flies
Drosophila is the most commonly used insect in the life sciences and a long-term focus of Ramdya's own research, who has been working on digitally tracking and modeling this animal for years. In 2019, his group published DeepFly3D, a deep-learning based motion-capture software that uses multiple camera views to quantify the movements of Drosophila in three-dimensional space.

Continuing with deep-learning, in 2021 Ramdya's team published LiftPose3D, a method for reconstructing 3D animal poses from 2D images taken from a single camera. These kinds of breakthroughs have provided the exploding fields of neuroscience and animal-inspired robotics with tools whose usefulness cannot be overstated.

In many ways, NeuroMechFly represents a culmination of all those efforts. Constrained by morphological and kinematic data from these previous studies, the model features independent computational parts that simulate different parts of the insect's body. This includes a biomechanical exoskeleton with articulating body parts, such as head, legs, wings, abdominal segments, proboscis, antennae, halteres (organs that help the fly measure its own orientation while flying), and neural network "controllers" with a motor output.

Why build a digital twin of Drosophila?
"How do we know when we've understood a system?" says Ramdya. "One way is to be able to recreate it. We might try to build a robotic fly, but it's much faster and easier to build a simulated animal. So one of the major motivations behind this work is to start building a model that integrates what we know about the fly's nervous system and biomechanics to test if it is enough to explain its behavior."

"When we do experiments, we are often motivated by hypotheses," he adds. "Until now, we've relied upon intuition and logic to formulate hypotheses and predictions. But as neuroscience becomes increasingly complicated, we rely more on models that can bring together many intertwined components, play them out, and predict what might happen if you made a tweak here or there."

The testbed
NeuroMechFly offers a highly valuable testbed for studies that advance biomechanics and biorobotics, but only in so far as it accurately represents the real animal in a digital environment. Verifying this was one of the researchers' main concerns. "We performed validation experiments which demonstrate that we can closely replicate the behaviors of the real animal," says Ramdya.

The researchers first made 3D measurements of real walking and grooming flies. They then replayed those behaviors using NeuroMechFly's biomechanical exoskeleton inside a physics-based simulation environment.

As they show in the paper, the model can actually predict various movement parameters that are otherwise unmeasured, such as the legs' torques and contact reaction forces with the ground. Finally, they were able to use NeuroMechFly's full neuromechanical capabilities to discover neural network and muscle parameters that allow the fly to "run" in ways that are optimized for both speed and stability.

"These case studies built our confidence in the model," says Ramdya. "But we are most interested in when the simulation fails to replicate animal behavior, pointing out ways to improve the model." Thus, NeuroMechFly represents a powerful testbed for building an understanding of how behaviors emerge from interactions between complex neuromechanical systems and their physical surroundings.

A community effort
Ramdya stresses that NeuroMechFly has been and will continue to be a community project. As such, the software is open source and freely available for scientists to use and modify. "We built a tool, not just for us, but also for others. Therefore, we made it open source and modular, and provide guidelines on how to use and modify it."

"More and more, progress in science depends on a community effort," he adds. It's important for the community to use the model and improve it. But one of the things NeuroMechFly already does is to raise the bar. Before, because models were not very realistic, we didn't ask how they could be directly informed by data. Here we've shown how you can do that; you can take this model, replay behaviors, and infer meaningful information. So this, I think, is a big step forward."

Research Report:NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster.

Related Links
Swiss Federal Institute of Technology Lausanne
All about the robots on Earth and beyond!

Tweet

Thanks for being here; We need your help. The SpaceDaily news network continues to grow but revenues have never been harder to maintain. With the rise of Ad Blockers, and Facebook - our traditional revenue sources via quality network advertising continues to decline. And unlike so many other news sites, we don't have a paywall - with those annoying usernames and passwords. Our news coverage takes time and effort to publish 365 days a year. If you find our news sites informative and useful then please consider becoming a regular supporter or for now make a one off contribution.
SpaceDaily Contributor $5 Billed Once credit card or paypal		SpaceDaily Monthly Supporter $5 Billed Monthly paypal only

Shaping the future of photonic sensing: Advanced Navigation acquires Vai Photonics
Sydney, Australia (SPX) May 06, 2022
Advanced Navigation, one of the world's most ambitious innovators in AI robotics, and navigation technology has announced the acquisition of Vai Photonics, a spin-out from The Australian National University (ANU) developing patented photonic sensors for precision navigation. Vai Photonics share a similar vision to provide technology to drive the autonomy revolution and will join Advanced Navigation to commercialise their research into exciting autonomous and robotic applications across land, air, ... read more