Virtual reality harnesses brain’s ability to interpret sound

Tuesday, November 3, 1998

Virtual reality harnesses brain’s ability to interpret sound

RESEARCH: By placing sound in new arrangements, project helps
workers better recognize information

By Matt Grace

Daily Bruin Contributor

A pair of UCLA scientists is taking advantage of the fact that
your brain is listening, even though sometimes you’re not.

By positioning sounds in a virtual audio reality, the
researchers can optimize a person’s ability to interpret and
distinguish multiple sources of information.

If all goes well, their project could spill over into the
high-stress world of air traffic controllers, airplane pilots and
surgeons, where photo-flash decisions must be made in response to a
messy buzz of speech and beeps.

"The brain is programmed to notice novelty," said Mary Shafer,
senior research engineer in flying qualities at NASA-Dryden. So if
information can be presented in novel way, a person will take
notice.

Case in point: when someone says a person’s name in a
conversation across the room, the brain of the person called
designates priority to it, and their attention shifts to the
direction in which the cue came. Virtual audio essentially mimics
the brain’s ability to sort sounds in relation to space.

"We’re doing something your brain already does," Shafer said.
"What we’re trying to do is push everyone away except for the
person that (the test flight controller) needs to listen to."

By filtering an audio signal through a computer, scientists can
position the sound into different "virtual" arrangements around the
head, said Michael Orosz, a graduate student in the department of
computer science.

"This simulates the effect of having the sound come from the
right or left side of the body."

And it is the sensation of hearing the sound from different
points in space that makes distinguishing the sounds much
easier.

The research focuses on two ways people can distinguish sounds.
First, the ear can locate a sound by "phase difference," or the
delay in sound being detected between one ear and the other, and
second, by a difference in volume.

The immediate goal of the project is to improve the interface
between NASA-1 (the name for the test flight controller at
NASA-Dryden) and the channels of information he or she must
monitor, using "virtual audio."

NASA-1 is in charge of the overall test program and is
responsible for listening to and sorting through a collage of
speech and audio cues.

"In the case of an emergency, this information may well
overwhelm NASA-1," said Walter Karplus, professor of computer
science at UCLA and principal investigator in this research
project.

If something goes wrong, NASA-1 must react instantly ­
whether it be to scrub the experiments on the flight, order the
airplane to return to base or, in extreme situations, order the
pilot to eject, Karplus said.

Unfortunately, the sounds heard by NASA-1 travel on a single
channel, making the task of sorting the signals, and determining
which ones are important at the time, all the more difficult.

"In this system, NASA-1 wears stereophonic earphones, and a
computer is used to process each speech channel and present its
contents to NASA-1 in such a way that each speech source appears to
emanate from a different point in space," Karplus said.

"Using virtual audio, the possibility that NASA-1 (could) be
confused or misled in listening to many simultaneous space signals
is minimized," continued Karplus.

"Sound sources are located and processed so as to draw NASA-1’s
attention to the most urgent messages."

An important cue, such as an engine trouble warning, can be
highlighted by making it easier to hear. Less immediate signals
­ which could become important later on ­ can be moved
into the background without being eliminated.

But there’s a problem.

Sounds coming from either directly in front or directly behind
the head hit both ears at the same time, eliminating the brain’s
ability to distinguish between the two cues ­ phase and
intensity ­ that are used to filter noise, Orosz said.

If more than four or five different sounds are coming in,
"you’re going to have to put some (signals) in front and some in
back," Orosz said. Otherwise, the sounds would have to be placed
close together, effectively counteracting the virtual audio.

"In the auditory world, if you have sounds coming two or three
degrees apart in space, you can’t distinguish them," Orosz
said.

Currently, Orosz and a team of cognitive psychologists from
Purdue University are exploring ways to optimize the comprehension
of large volumes of information and to make rapid decisions under
pressure.

In addition, a software language is being developed that will
allow users to customize it.

"The science itself is non-specific," Orosz said. "It’s not just
cutting-edge technology, it’s very flexible and applicable."

This makes a transition for virtual audio into the real world,
not only feasible but highly probable.

Although most of the projects are developed specifically for
NASA, fallout from government contracts often leads to commercial
projects.

"For example, the (Federal Aviation Administration) has
announced that within the next few years the managing of air
traffic around airports will become the responsibility of pilots
and not air traffic controllers," Karplus said.

"This will place new loads upon the pilot of each plane, and the
method developed at UCLA may play a part."

Virtual audio could also be used by fire chiefs in emergency
situations, where they are required to handle communication with
multiple sources. Similarly, Karplus envisions surgeons working
with virtual audio.

Medical procedures that involve collaboration between a number
of clinicians will require the head surgeon to process many voices
coming to him at once.

Virtual audio would prevent them from confusing and cancelling
each other out and help to maintain vigilance, Karplus said.

"There may be someone who does not talk very much, but the
doctor still needs to know he’s there."

CHARLES KUO

Walter Karplus works on a stereo system that will allow people
like air traffic controllers to process large amounts of sound
data.

Comments, feedback, problems?

© 1998 ASUCLA Communications Board[Home]