Earth Protect Blog

SCIENCE & NATURE

Scientists are using machine learning to eavesdrop on various animals

By Emily Anthes

© The New York Times Co.

The naked mole rat may not be much to look at, but it has much to say. The wrinkled, whiskered rodents, which live, as many ants do, in large, underground colonies, have an elaborate vocal repertoire. They whistle, trill and twitter; grunt, hiccup and hiss.

And when two of the voluble rats meet in a dark tunnel, they exchange a standard salutation. “They’ll make a soft chirp, and then a repeating soft chirp,” said Alison Barker, a neuroscientist at the Max Planck Institute for Brain Research in Germany. “They have a little conversation.”

Hidden in this everyday exchange is a wealth of social information, Barker and her colleagues discovered when they used machine-learning algorithms to analyze 36,000 soft chirps recorded in seven mole rat colonies.

Not only did each mole rat have its own vocal signature, but each colony had its own distinct dialect, which was passed down, culturally, over generations. During times of social instability — as in the weeks after a colony’s queen was violently deposed — these cohesive dialects fell apart. When a new queen began her reign, a new dialect appeared to take hold.

“The greeting call, which I thought was going to be pretty basic, turned out to be incredibly complicated,” said Barker, who is now studying the many other sounds the rodents make. “Machine-learning kind of transformed my research.”

Machine-learning systems, which use algorithms to detect patterns in large collections of data, have excelled at analyzing human language, giving rise to voice assistants that recognize speech, transcription software that converts speech to text and digital tools that translate between human languages.

In recent years, scientists have begun deploying this technology to decode animal communication, using machine-learning algorithms to identify when squeaking mice are stressed or why fruit bats are shouting. Even more ambitious projects are underway — to create a comprehensive catalog of crow calls, map the syntax of sperm whales and even to build technologies that allow humans to talk back.

“Let’s try to find a Google Translate for animals,” said Diana Reiss, an expert on dolphin cognition and communication at Hunter College and cofounder of Interspecies Internet, a think tank devoted to facilitating crossspecies communication.

The field is young and many projects are still in their infancy; humanity is not on the verge of having a Rosetta Stone for whale songs or the ability to chew the fat with cats. But the work is already revealing that animal communication is far more complex than it sounds to the human ear, and the chatter is providing a richer view of the world beyond our own species.

“I find it really intriguing that machines might help us to feel closer to animate life, that artificial intelligences might help us to notice biological intelligences,” said Tom Mustill, a wildlife and science filmmaker and the author of the forthcoming book, “How to Speak Whale.” “This is like we’ve invented a telescope — a new tool that allows us to perceive what was already there but we couldn’t see before.”

Advanced eavesdropping Studies of animal communication are not new, but machine-learning algorithms can spot subtle patterns that might elude human listeners. For instance, scientists have shown that these programs can tell apart the voices of individual animals, distinguish between sounds that animals make in different circumstances and break their vocalizations down into smaller parts, a crucial step in deciphering meaning.

“One of the things that’s really great about animal sound is that there are still so many mysteries and that those mysteries are things which we can apply computation to,” said Dan Stowell, an expert in machine listening at Tilburg University and Naturalis Biodiversity Center in the Netherlands.

Several years ago, researchers at the University of Washington used machine learning to develop software, called Deep-Squeak, that can automatically detect, analyze and categorize the ultrasonic vocalizations of rodents.

DeepSqueak has been repurposed for other species, including lemurs and whales, while other teams have developed their own systems for automatically detecting when clucking chickens or squealing pigs are in distress.

Decoding meaning

Decoding the meaning of animal calls also requires large amounts of data about the context surrounding each squeak and squawk.

To learn more about the vocalizations of Egyptian fruit bats, researchers used video cameras and microphones to record groups of the animals for 75 days. Then they reviewed the recordings, painstakingly noting several important details, such as which bat was vocalizing and in what context, for each of nearly 15,000 calls.

The bats are pugilistic, frequently quarreling in their crowded colonies, and the vast majority of their vocalizations are aggressive.

“Basically, they’re pushing each other,” said Yossi Yovel, a neuroecologist at Tel Aviv University in Israel who led the research. “Imagine a big stadium and everybody wants to find a seat.”

But a machine-learning system could distinguish, with 61% accuracy, between aggressive calls made in four different contexts, determining whether a particular call had been emitted during a fight related to food, mating, perching position or sleep. That’s not a perfect performance, Yovel noted, but it is significantly better than the 25% accuracy associated with random guessing.

Whale tales

Other major projects are underway. Project CETI — short for the Cetacean Translation Initiative — is bringing together machine-learning experts, marine biologists, roboticists, linguists and cryptographers, among others, at more than a dozen institutions to decode the communication of sperm whales, which emit bursts of clicks that are organized into Morse code-like sequences called codas.

The team is planning to install its “core whale-listening stations,” each of which includes 28 underwater microphones, off the coast of Dominica this fall. It plans to use robotic fish to record audio and video of the whales, as well as small acoustic tags to record the vocalizations and movements of individual animals.

Then, the researchers will try to decipher the syntax and semantics of whale communication and probe bigger scientific questions about sperm whale behavior and cognition, such as how large groups coordinate their actions and how whale calves learn to communicate.

“Every which way we turn there’s another question,” said David Gruber, a marine biologist at Baruch College in New York City who leads Project CETI. “If there was a big event that happened a week ago, how would we know that they’re still communicating about it? Do whales do mathematics?”

The Earth Species Project, a California-based nonprofit, is also partnering with biologists to pilot an assortment of machinelearning approaches with whales and other species.

For instance, it is working with marine biologists to determine whether machine-learning algorithms can automatically identify what behaviors baleen whales are engaging in, based on movement data collected by tracking tags.

Talking back

The Earth Species Project has also teamed up with Michelle Fournet, a marine acoustic ecologist at the University of New Hampshire, who has been trying to decipher humpback whale communication by playing prerecorded whale calls through underwater speakers and observing how the whales respond.

Now, Earth Species scientists are using algorithms to generate novel humpback whale vocalizations — that is, “new calls that don’t exist but sound like they could,” Fournet said. “I can’t say how cool it is to imagine something from nature that isn’t there and then to listen to it.”

Playing these new calls to wild whales could help scientists test hypotheses about the function of certain vocalizations, she said.

Given enough data about how whales converse with each other, machine-learning systems should be able to generate plausible responses to specific whale calls and play them back in real time, experts said. That means that scientists could, in essence, use whale chatbots to “converse” with the marine mammals even before they fully understand what the whales are saying.

These machine-mediated conversations could help researchers refine their models, and improve their understanding of whale communication. “At some point, it might be a real dialogue,” said Michael Bronstein, a machine-learning expert at the University of Oxford and part of Project CETI.

He added, “As a scientist, this is probably the craziest project I have ever participated in.”

Learning to listen

The prospect of ongoing, two-way dialogue with other species remains unknown. But true conversation will require a number of “prerequisites,” including matching intelligence types, compatible sensory systems and, crucially, a shared desire to chat, said Natalie Uomini, an expert on cognitive evolution at the Max Planck Institute for Evolutionary Anthropology.

These experiments may also raise ethical issues, experts acknowledge. “If you find patterns in animals that allow you to understand their communication, that opens the door to manipulating their communications,” Mustill said.

But the technology could also be deployed for the benefit of animals, helping experts monitor the welfare of both wild and domestic fauna. Scientists also said that they hoped that by providing new insight into animal lives, this research might prompt a broader societal shift. Many pointed to the galvanizing effect of the 1970 album “Songs of the Humpback Whale,” which featured recordings of otherworldly whale calls and has been widely credited with helping to spark the global Save the Whales movement.

Biologist Roger Payne, who produced that album, is now part of Project CETI. And many scientists said they hoped these new, high-tech efforts to understand the vocalizations of whales — and crows and bats and even naked mole rats — will be similarly transformative, providing new ways to connect with and understand the creatures with whom we share the planet.

“It’s not what the whales are saying that matters to me,” Gruber said. “It’s the fact that we’re listening.”