Institut für Schallforschung der Österreichischen Akademie der Wissenschaften: A-1010 Wien; Liebiggasse 5. Tel. +43 1/4277-29500; Fax +43 1/4277-9296;

Institut für Schallforschung der Österreichischen Akademie der Wissenschaften: A-1010 Wien; Liebiggasse 5. Tel. +43 1/4277-29500; Fax +43 1/4277-9296; email: [email protected]; http://www.kfs.oeaw.ac.at

Masking, Maths, Marseille & More- An introduction to the phase vocoder, the phase, psychoacoustic masking and frame multipliers (an extension of the talk given at the HASSIP minischool 2003 in Marseille and at the LMA, CRNS 2004)

Peter Balazs

Acoustics Research Institute, Austrian Academy of Science, Vienna- http://www.kfs.oeaw.ac.at- [email protected]


• Phase Vocoder • Phase in STFT

• Masking • Frame Multiplier

CONTENT


SHORT TIME FOURIER TRANSFORMATION

The phase vocoder is an algorithm whose first part of analysis is just the discrete STFT.

Short Time Fourier transformation, STFT


INTERPRETATION OF THE PHASE VOCODER (STFT)

Fourier-transform

Windowed signal

mathematically equivalent

Filterbank

A channel of the filterbank

Gabor projection

Shifted atoms


Uncertainty:

FROM FOURIER TO STFT

Non-uniqueness:


overlap addoscillator bankprojection on dual atoms

SYNTHESIS

REDUNDANCYSTFT

PERCEPTUAL REDUNDANCY

masking

Basis or frame

SYNTHESIS AND REDUNDANCY

Gabor frame :


PHASE VOCODER

Low pass filter

Phase calculation

Filterbank

A channel of the filterbank


AUDITORY PERCEPTION

In the human ear the sound wave excitates the basilar membrane and activates the nerve cells. Nearer to the window they react to high, farther away to low frequencies. This correlation between frequency and point on the membrane is called tonotopy.

The human ear


AUDITORY PERCEPTION

Bark :

0

2

4

6

8

10

12

14

16

18

030

060

090

012

0015

0018

0021

0024

0027

0030

0033

0036

0039

00


AUDITORY MASKING

Spectrum and relevance threshold

Masking : the presence of one stimulus, the masker, decreases the response to another stimulus, the target.

Thresholding = Gabor Multiplier


http://www.mmk.ei.tum.de/~tal/demos/demos_content.html

Psychoakustik (Zwicker) - Examples


IMPLEMENTATION OF MASKING FILTER IN STX

output signal

Bark scale transformationspreading function

convolution

inverse bark scale transformation and

compensation

power spectrum [dB/Hz]

power spectrum [dB/Bark]

auditory filter

threshold spectrum [dB/Bark]

threshold spectrum [dB/Hz]

analysis section of the phase vocoder

synthesis section of the phase vocoder

phase spectrum

offset level [dB]

filter: remove spectral components lower than threshold

filtered power spectrum [dB/Hz]

input signal

For the analysis and re-synthesis the atoms PVANAand PVSYN are used.

IRRECKEL: Module for computation of relevance threshold and filtering



Original



Relevant



Irrelevant


APPLICATIONS

Sound / Data Compression : MP3

Sound Design

Background – Foreground Separation: over-masking

Contrast Increase


CURRENT IMPROVEMENT / DEVELOPMENT

• Improvements of the current algorithms .

• Newer psychoacoustical knowledge

• Description of hearing and masking in the context of Gabor or wavelets

•Time-Frequency Masking

• Eigenfunctions


TEMPORAL MASKING


TIME FREQUENCY MASKING

Acoustics Research Institute, Austrian Academy of Science

LMA, CNRS Marseille

LATP, CMI, Marseille



f

t

A

C

B

D

Laback / Balazs:





Laback / Balazs:

f

t

A



Laback / Balazs:

f

t

A



Laback / Balazs:

f

t

A



Basic:

Use simple modell for simultaneous and temporal masking

Test

Use excitation pattern (Heijden)

“Extended”:

Use newer psychoacoustical knowledge (Laback)

Use algorithm to speparate signal in tonal, transient and noise parts (Jaillet)

Test

Use STX spreading function


FILTER

Thresholding:

Gabor Multiplier: irregular grid (bark)

Time Frequency localization operator (continuous)

Wavelet Multiplier ?

Multiple Gabor Extension Multiplier ?

(Or something completely different:) Auditory Filter Multiplier ?

Gamma-tone Filter Multiplier?

Spectrum and relevance threshold


FRAME MULTIPLIER

Frame

Frame Multiplier:


FRAME MULTIPLIER


REFERENCES:

[1] S.S. Abeyseker, K.P. Padhi et al, Investigation Of Different Freqeuncy Estimation Techniques Using The Phase Vocoder , Circuits and Systems, 2001. ISCAS 2001. The 2001 IEEE International Symposium on, Volume: 2 ,(2001)[2] P.Balazs, H. Waubke, W.A. Deutsch, Phasenanalyse mit akustischen Anwendungsbeispielen, Proceedings DAGA 2003 - Fortschritte der Akustik, Aachen (2003)

[3] O. Christensen, An Introduction To Frames And Riesz Bases , Birkhäuser Boston (2003)[4] W.A. Deutsch, A. Noll, Simulation auditorischer Signaltrennung in komplexen musikalischen Signalen durch Übermaskierung, Fortschritte der Akustik 'DAGA'93 (1993) [5] M. Dolson, The Phase Vocoder: A Tutorial, Computer Musical Journal, Vol. 10. No. 4 (1986)[6] G. Eckel, Ein Modell der Mehrfachverdeckung für die Analyse musikalischer Schallsignale, PhD Thesis, Universität Wien (1989)[7] H. Fastl, Beschreibung dynamischer Hörempfindungen anhand von Mithörschwellen-Muster, HochschulVerlag Freiburg (1982)[8] H.G. Feichtinger, T. Stromer, Gabor Analysis and Algorithms, Theory and Applications, Birkhäuser Boston (1998)[9] H. G. Feichtinger, T. Strohmer, Advances in Gabor Analysis , Birkhäuser Bosten (2003)[10] S. A. Gelfand, Essentials of audiology, Thieme New York. (1997)[11] K. Gröchenig, Foundations of Time-Frequency Analysis, Birkhäuser Boston (2001)[12] B. Laback, Effekte der Simultanmaskierung auf die Musikperzeption bei sensorineuralen Hörschäden und ihre Anwendung für Signalverarbeitungsalgorithmen in Hörgeräten, PhD Thesis, Universität Wien (1998)[13] B. Moore, Perceptual Consequences of Cochlear Damage, Oxford Medical Publications (1995)[14] A. Noll, J. White, P. Balazs, W. A. Deutsch, STX - Intelligent Sound Processing, Programmer's Reference, http://www.kfs.oeaw.ac.at[15] Zwicker E, Fastl H, Psychoacoustics, Springer-Verlag, Berlin, Germany, 1990

Documents

Institut für Schallforschung der Österreichischen Akademie der Wissenschaften: A-1010 Wien; Liebiggasse 5. Tel. +43 1/4277-29500; Fax +43 1/4277-9296;