Subtitles section Play video Print subtitles Here, off Canada's Pacific coast, researchers are hoping to make a long-held dream come true. Behavioral biologists and IT experts have teamed up to create programs aimed at deciphering acoustic signals from animals. Could artificial intelligence help to identify patterns in the sounds made by marine mammals? Will we soon start to understand what it is that whales are talking about? The coastal waters between Alaska in the U.S. and Canada's Vancouver Island are where a group of killer whales or “orcas” spend their summers. A team of scientists from the DEEP AL research expedition are preparing to embark. “DEEP AL” stands for “Deep Learning Applied to Animal Linguistics.” Computer scientist Elmar Nöth from Germany's University of Erlangen has spent years working on automatic recognition for human speech patterns. Can the same methods be adapted to animal languages? Underwater microphones embedded in tubes serve as the expedition's ears. For three summers, teams of computer scientists and biologists have set out to record orca calls and document whale behavior. Rachael Cheng from the Leibniz Institute for Zoo and Wildlife Research in Berlin is looking for patterns between behavior and animal vocalizations that could help to decipher whale language. I assume they would exchange information. They may have a very, very different system which may not fit into our prototype of a language. Around 300 orcas are identified as "Northern Residents" — as they spend each summer along the coast of Alaska and northern British Columbia. They live in small family groups and are very communicative. Do different families use different dialects? And is it possible to discern the meaning of individual calls? Here the researchers lower the highly sensitive microphones into the water. The eight hydrophones can record sounds up to a frequency of 100 kilohertz. That's far higher pitched than what is audible to the human ear. Thanks to the network of hydrophones and acoustic triangulation, the researchers will later be able to calculate the positions of the whales. To avoid disturbing the whales with engine noise, the research trimaran is equipped with an electric motor. While visibility is usually limited underwater, sound waves are transmitted over considerable distances. That's why a communication system with loud calls is clearly beneficial. They're very tight together, with frequent direction change. Circling. And body twisting. Plus a lot of echolocation. They are socializing, like you will see. They bump into each other and rub, and frequent twisting, and then jump onto each other. It sounds like they talk about the plan “What are we going to do next?” Orcas only spend about five percent of their time at the surface, which makes systematic observation quite difficult. The research team uses drones to document the behavior of the animals. Diving expeditions with whales are prohibited in Canada. The scientists are looking for the smallest meaningful units of communication. Which whale is calling, and which one answers? Are some sounds repeated more than others? Biologist Elysanne Durand examines the recordings. Each call comprises a series of brief pulses which sound like melodic curves to human ears. Each shift in sound could be meaningful. While recording the whale calls, the researchers also document the behavior of the animals. So there's four individuals in this group here that are circling, there were four back there with the male, there is one I can see out of the corner of my eye coming towards us, and the mom and calf. So that's 8, 10, 11 individuals minimum. The more data available, the easier it is to "train" deep-learning programs to decipher whale language. It's therefore a major advantage for the researchers that over 20,000 hours of orca calls have already been collected — more than for any other animal species. Whale researcher Jared Towers has been tasked by the Canadian government with observing different orca populations. Back in the 1970s, scientists here began documenting individual animals as well as their group structures. A108 is right underneath the boat here, here she is, she is gone off ahead. Jared Towers has no problem telling the orcas apart. This scratch here on A108 has persisted for at least a couple of years now. So the way that we identify the individuals no matter what population they belong to is by appearance and you get used to looking for certain features on an individual killer whale. The dorsal fins and the patches around the fins have different shapes — enabling scientists to catalog each of the Northern Residents. Every family forms a lifelong bond. What we are looking at with all these families is an adult female leading the group, and the fathers of their offspring don't play much of a role in that family group. Jared Towers works for a federal institution that safeguards Canada's waters. His former boss, John Ford, was the first to distinguish between the disparate calls of the whales. Ford's research revolutionized our understanding of the communication system used by the resident orcas. We are listening to calls of A-Clan whales. That is the exciting part of underwater listening. You are getting a window into their life that you would never see. John Ford discovered that the whales use about 50 different calls. Different families prefer different types of calls. These were named “dialects” and used to help identify individual families. When they are making the various stereotyped signals, it's simply to keep in touch with everybody in the group — they exchange them, they are constantly monitoring each other's location. They know where they are because of their very directional hearing, and they can monitor the behavioral state, the excitement level, the arousal state of all the other animals in the kin group. Northern Resident orcas visit pebble beaches daily. You can hear them rubbing right now. They're making socializing sounds and you can hear the pebbles getting pushed around. And they just rub all sides of their body. Just in the shallow part of the beach. Only a few orca groups worldwide engage in this sort of body rub. This behavior is not genetic, rather it's a tradition passed on within families of the Northern Residents, just like their language. Back on the German-Canadian research boat, it's a challenge to locate each family among the 300 individuals that make up the Northern Resident population. The orcas are constantly on the move in an expanse of water the size of Belgium. The expedition covers an area from Vancouver Island to the southern tip of Alaska. Seagulls indicate where schools of salmon might be. And this is where orcas often hunt too. Calls from the “A-Clan” can be heard on the underwater microphones. The team tries to determine the position of the whales. They compare the calls with the catalog compiled by John Ford, but they encounter discrepancies. The calls of class "N9" are used by different whale families. Calls from the same class should be almost identical, but these differ in length, melody and harmonics. Human analysis so far has amounted to only a rough classification. Looking at the spectrogram I am very sure that we can achieve something that rivals the human performance. Two whale families approach. What calls are they exchanging? Here is that call. OK, let's just record this. After I hear the call the back group surfaced. Then I spotted the front group turning around. The researchers are interested in which group is calling, which one answers, and which sounds they're using. Look here, I4 is approaching the boat. You can hear lots of calls. That's N23 from the GI clan. There is a lot of variation also — and interestingly, here is the A23 family. The calls look very different. That's how people differentiate different matrilines. If we have a lot of calls, we can try to train a classifier. The programmers use algorithms or “classifiers” to automatically analyze millions of whale calls in order to compare recurring sound patterns with recurring behavioral patterns. This same method is used to decipher the meaning of individual words in foreign languages. For Elmar Nöth and his team, it's no easy feat to automatically filter