A Joint Audio-Visual Approach to Audio Localization

Localization of audio sources is an important research problem, e.g.,
to facilitate noise reduction. In the recent years, the problem has
been tackled using distributed microphone arrays (DMA). A common
approach is to apply direction-of-arrival (DOA) estimation on
each array (denoted as nodes), and then map the DOA estimates to
a location. In practice, however, the individual nodes contain few
microphones, limiting the DOA estimation accuracy and, thereby,
also the localization performance. We investigate a new approach,
where range estimates are also obtained and utilized from each node,
e.g., using time-of-flight cameras. Moreover, we propose an optimal
method for weighting such DOA and range information for audio
localization. Our experiments on both synthetic and real data show
that there is a clear, potential advantage of using the joint audiovisual
localization framework.