There is now a new version of the Auditory Toolbox. It
contains Matlab
functions to implement many different kinds of auditory models. The
toolbox
includes code for Lyon's passive longwave model, Patterson's gammatone
filterbank, Meddis' hair cell model, Seneff's auditory model,
correlograms
and several common representations from the speech-recognition world
(including
MFCC, LPC and spectrograms). This code has been tested on Macintosh,
Windows,
and Unix machines using Matlab 5.2.
Note: This toolbox was originally published as Apple Computer Technical Report #45. The old technical report ( PDF PDF and Postscript ) and old code ( Unix TAR and Macintosh BinHex ) are available for historical reasons.
(Version 2.0)
My primary scientific goal is to understand how our
brains perceive
sound.
My role in this research area is a modeler, I build models that explain
the neurophysiological and psychoacoustic data. Hopefully these models
will help other researchers understand the mechanisms involved and
result
in better experiments. My latest work in this area is titled
"Connecting
Correlograms to Neurophysiology and Psychoacoustics" and was presented
at the XIth
International Symposium on Hearing in Grantham England from 1-6 August,
1997. Two correlograms, one computed using autocorrelation and other
other
computed using AIM, are shown on the left.
The
information in most auditory models flows exclusively bottom-up,
yet
there is increasing evidence that a great deluge of information is
flowing
down from the cortex. A paper I wrote for the 
I have written several papers describing how to convert
auditory
representations
into sounds. I have built models of the cochlea and central auditory
processing,
which I hope both explain auditory processing and will allow us to
build
auditory sound separation tools. These papers describe the process of
converting
sounds into cochleagrams and correlograms, and then converting these
representations
back into sounds. Unlike the printed versions of this work, the web
page
includes audio file examples. It includes better spectrogram inversion
techniques, a description of how to invert Lyon's passive cochlear
model,
and a description of correlogram inversion. This material was first
presented
as part of the Proceedings of the ATR Workshop on "A Biological
Framework
for Speech Perception and Production" published in September 1994.
A more refined version of this paper was an invited talk at the 
Pattern Playback is the term used by Frank
Cooper to describe his
successful
efforts to paint spectrogram on plastic and then convert them into
sound.
I wrote of Pattern Playback techniques, from Frank Cooper's efforts to
my own efforts with auditory model inversion, in a paper which was
published
at the 1995 IEEE International Conference on Systems, Man, and
Cybernetics.
My paper is titled "Pattern Playback from 1950 to 1995". The image at
the
left shows a portion of one of Cooper's spectrograms.
Chris
Bregler, Michele Covell, and I developed a technique we call
Video
Rewrite to automatically synthesize video of talking heads. This
technology
is cool because we use a purely data driven approach (concatenative
triphone
video synthesis) to create new video of a person speaking. Given new
audio,
we concatenate the best sequence of lip images and morph them into a
background
sequence. We can automatically create sequences like the Kennedy and
Johnson
scenes in the movie "Forrest Gump."
We studied how adults convey affective messages to
infants using
prosody.
We did not attempt to recognize the words, let alone to distill more
nebulous
concepts such as satire or irony. We analyzed speech with low-level
acoustic
features and discriminated approval, attentional bids, and prohibitions
from adults speaking to their infants. We built automatic classifiers
to
create a system, Baby Ears, that performs the task that comes so
naturally
to infants. The image on the left shows one of the decision surfaces
which
classifies approval, attention and prohibition utterances on the basis
of their pitch.
Eric
Scheirer and I worked on a system for discriminating between
speech
and music in an audio signal. This paper describes a large number of
features,
how they can be combined into a statistical framework, and the
resulting
performance on discriminating signals found on radio stations. The
results
are better then anybody else's results. (That comparison is not
necessarily
valid since there are no common testing databases. We did work hard to
make our test set representative.) This paper was published at the 1997
ICASSP in Munich. The image on the left shows clouds of our data.
Work we've done to morph between two sounds is described
in a paper at
the 1996 ICASSP. This work is new because it extends previous audio
morphing
work to include inharmonic sounds. This paper uses results from
Auditory
Scene Analysis to represent, match, warp, and then interpolate between
two sounds. The image on the left shows the smooth spectrogram, one of
two independent representations used when morphing audio signals.
I
have written Matlab m-functions that read and write QuickTime movies.
The WriteQTMovie code is more
general than previous solutions for creating movies in Matlab. It runs
on any platform that Matlab runs on. It also lets you add sound to the
movie. The ReadQTMovie code reads and parses JPEG compressed moves.
I
created a Hypercard stack to make it easier for people with a
Macintosh
and CDROM drive to interact with the Acoustical Society of America's 
I
wrote a program for the Macintosh 660/AV and 840/AV computers that uses
the DSP (AT&T3210) to monitor audio levels. VUMeters runs on any
Macintosh
with the AT&T DSP chip. Source and binaries are included.
Bill Stafford and I wrote TCPPlay
to allow us to play sounds from a
Unix
machine over the network to the Macintosh on our desks. This archive
includes
Macintosh and Unix source code and the Macintosh application. There are
other network audio solutions, but this works well on the Macintosh.
In
a past life, I worked on medical imaging. A book on tomographic imaging
(cross-sectional x-ray imaging) was published by IEEE Press: Avinash C.
Kak and Malcolm Slaney, Principles of Computerized Tomographic
Imaging,
(New York : IEEE Press, c1988). The software used to generate many of
the
tomographic images in this book is available. The parallel beam
reconstruction
on the left was generated with the commands
