Алгоритмы быстрого вычисления разреженного преобразования Фурье (Sparse FFT), осень 2015: Сигналы с разреженным спектром Фурье: определения и баз

Embed Size (px)

Text of Алгоритмы быстрого вычисления разреженного...

  • Sparse Fourier Transform(lecture 1)

    Michael Kapralov1

    1IBM Watson EPFL

    St. Petersburg CS ClubNovember 2015

    1 / 73

  • Given x Cn, compute the Discrete Fourier Transform (DFT) ofx :

    xi =1n

    j[n]

    xj ij ,

    where = e2i/n is the n-th root of unity.

    Assume that n is a power of 2.

    compression schemes(JPEG, MPEG)

    signal processingdata analysis

    imaging (MRI, NMR)

    2 / 73

  • Given x Cn, compute the Discrete Fourier Transform (DFT) ofx :

    xi =1n

    j[n]

    xj ij ,

    where = e2i/n is the n-th root of unity.

    Assume that n is a power of 2.

    compression schemes(JPEG, MPEG)

    signal processingdata analysis

    imaging (MRI, NMR)

    2 / 73

  • Given x Cn, compute the Discrete Fourier Transform (DFT) ofx :

    xi =1n

    j[n]

    xj ij ,

    where = e2i/n is the n-th root of unity.

    Assume that n is a power of 2.

    compression schemes(JPEG, MPEG)

    signal processingdata analysis

    imaging (MRI, NMR)

    2 / 73

  • DFT has numerous applications:

    3 / 73

  • Fast Fourier Transform (FFT)

    Computes Discrete Fourier Transform (DFT) of a length nsignal in O(n logn) time

    Cooley and Tukey, 1964

    Gauss, 1805

    Code=FFTW (Fastest Fourier Transform in the West)

    4 / 73

  • Fast Fourier Transform (FFT)

    Computes Discrete Fourier Transform (DFT) of a length nsignal in O(n logn) time

    Cooley and Tukey, 1964

    Gauss, 1805

    Code=FFTW (Fastest Fourier Transform in the West)

    4 / 73

  • Fast Fourier Transform (FFT)

    Computes Discrete Fourier Transform (DFT) of a length nsignal in O(n logn) time

    Cooley and Tukey, 1964

    Gauss, 1805

    Code=FFTW (Fastest Fourier Transform in the West)

    5 / 73

  • Fast Fourier Transform (FFT)

    Computes Discrete Fourier Transform (DFT) of a length nsignal in O(n logn) time

    Cooley and Tukey, 1964

    Gauss, 1805

    Code=FFTW (Fastest Fourier Transform in the West)

    6 / 73

  • Sparse FFT

    Say that x is k -sparse if x has k nonzero entries

    Say that x is approximately k -sparse if x is close to k -sparse insome norm (`2 for this lecture)

    -0.0004

    -0.0002

    0

    0.0002

    0.0004

    -1000 -500 0 500 1000

    time

    0

    0.2

    0.4

    0.6

    0.8

    1

    -1000 -500 0 500 1000

    frequency

    7 / 73

  • Sparse FFT

    Say that x is k -sparse if x has k nonzero entries

    Say that x is approximately k -sparse if x is close to k -sparse insome norm (`2 for this lecture)

    -0.0006

    -0.0004

    -0.0002

    0

    0.0002

    0.0004

    0.0006

    -1000 -500 0 500 1000

    time

    0

    0.2

    0.4

    0.6

    0.8

    1

    -1000 -500 0 500 1000

    frequency

    tail noisehead

    8 / 73

  • Sparse approximations

    JPEG=

    Given x , compute x , then keep top k coefficients only for k N

    Used in image and video compression schemes(e.g. JPEG, MPEG)

    9 / 73

  • Sparse approximations

    JPEG=

    Given x , compute x , then keep top k coefficients only for k N

    Used in image and video compression schemes(e.g. JPEG, MPEG)

    10 / 73

  • Computing approximation fast

    Basic approach:

    FFT computes x from x in O(n logn) time

    compute top k coefficients in O(n) time.

    Sparse FFT: directly computes k largest coefficients of x (approximately

    formal def later)

    Running time O(k log2 n) or faster

    Sublinear time!

    11 / 73

  • Computing approximation fast

    Basic approach:

    FFT computes x from x in O(n logn) time

    compute top k coefficients in O(n) time.

    Sparse FFT: directly computes k largest coefficients of x (approximately

    formal def later)

    Running time O(k log2 n) or faster

    Sublinear time!

    11 / 73

  • Sample complexity

    Sample complexity=number of samples accessed in timedomain.

    12 / 73

  • Sample complexity

    In medical imaging (MRI, NMR), one measures Fouriercoefficients x of imaged object x (which is often sparse)

    13 / 73

  • Sample complexity

    In medical imaging (MRI, NMR), one measures Fouriercoefficients x of imaged object x (which is often sparse)

    13 / 73

  • Sample complexity

    Measure x Cn, compute the Inverse Discrete FourierTransform (IDFT) of x :

    xi =

    j[n]xj ij .

    Given x Cn, compute the Discrete Fourier Transform (DFT) ofx :

    xi =1n

    j[n]

    xj ij .

    Sample complexity=number of samples accessed in timedomain.

    Governs the measurement complexity of imaging process.

    14 / 73

  • Sample complexity

    Measure x Cn, compute the Inverse Discrete FourierTransform (IDFT) of x :

    xi =

    j[n]xj ij .

    Given x Cn, compute the Discrete Fourier Transform (DFT) ofx :

    xi =1n

    j[n]

    xj ij .

    Sample complexity=number of samples accessed in timedomain.

    Governs the measurement complexity of imaging process.

    14 / 73

  • Sample complexity

    Measure x Cn, compute the Inverse Discrete FourierTransform (IDFT) of x :

    xi =

    j[n]xj ij .

    Given x Cn, compute the Discrete Fourier Transform (DFT)of x:

    xi =1n

    j[n]

    xj ij .

    Sample complexity=number of samples accessed in timedomain.

    Governs the measurement complexity of imaging process.

    15 / 73

  • Sample complexity

    Measure x Cn, compute the Inverse Discrete FourierTransform (IDFT) of x :

    xi =

    j[n]xj ij .

    Given x Cn, compute the Discrete Fourier Transform (DFT)of x:

    xi =1n

    j[n]

    xj ij .

    Sample complexity=number of samples accessed in timedomain.

    Governs the measurement complexity of imaging process.

    15 / 73

  • Given access to signal x in time domain, find best k -sparseapproximation to x approximately

    Minimize

    1. runtime

    2. number of samples

    16 / 73

  • Algorithms

    Randomization Approximation Hashing Sketching . . .

    Signal processing

    Fourier transform Hadamard transform Filters Compressive sensing . . .

    17 / 73

  • Lecture 1: summary of techniques fromGilbert-Guha-Indyk-Muthukrishnan-Strauss02, Akavia-Goldwasser-Safra03,

    Gilbert-Muthukrishnan-Strauss05, Iwen10, Akavia10,

    Hassanieh-Indyk-Katabi-Price12a, Hassanieh-Indyk-Katabi-Price12b

    Lecture 2: Algorithm with O(k logn) runtime (noiselesscase) Hassanieh-Indyk-Katabi-Price12b

    Lecture 3: Algorithm with O(k log2 n) runtime (noisy case)Hassanieh-Indyk-Katabi-Price12b

    Lecture 4: Algorithm with O(k logn) sample complexityIndyk-Kapralov-Price14, Indyk-Kapralov14

    18 / 73

  • Outline

    1. Computing Fourier transform of 1-sparse signals fast

    2. Sparsity k > 1: main ideas and challenges

    19 / 73

  • Outline

    1. Computing Fourier transform of 1-sparse signals fast

    2. Sparsity k > 1: main ideas and challenges

    20 / 73

  • Sparse Fourier Transform (k = 1)Warmup: x is exactly 1-sparse: xf = 0 when f 6= f for some f

    -0.0004

    -0.0002

    0

    0.0002

    0.0004

    -1000 -500 0 500 1000

    time

    0

    0.2

    0.4

    0.6

    0.8

    1

    -1000 -500 0 500 1000

    frequency

    f

    Note: signal is a pure frequency

    Given: access to x

    Need: find f and xf

    21 / 73

  • Two-point samplingInput signal x is a pure frequency, so xj = a f j

    +noise

    Sample x0,x1

    We have

    x0 = a

    +noise

    x1 = a f

    +noise

    So

    x1/x0 =f

    +noise

    Can read frequency from theangle!

    f

    unit circle

    x0 = ax1 = a f

    Pro: constant time algorithmCon: depends heavily on the signal being pure

    22 / 73

  • Two-point samplingInput signal x is a pure frequency, so xj = a f j

    +noise

    Sample x0,x1

    We have

    x0 = a

    +noise

    x1 = a f

    +noise

    So

    x1/x0 =f

    +noise

    Can read frequency from theangle!

    f

    unit circle

    x0 = ax1 = a f

    Pro: constant time algorithmCon: depends heavily on the signal being pure

    22 / 73

  • Two-point samplingInput signal x is a pure frequency, so xj = a f j

    +noise

    Sample x0,x1

    We have

    x0 = a

    +noise

    x1 = a f

    +noise

    So

    x1/x0 =f

    +noise

    Can read frequency from theangle!

    f

    unit circle

    x0 = ax1 = a f

    Pro: constant time algorithmCon: depends heavily on the signal being pure

    23 / 73

  • Two-point samplingInput signal x is a pure frequency, so xj = a f j

    +noise

    Sample x0,x1

    We have

    x0 = a

    +noise

    x1 = a f

    +noise

    So

    x1/x0 =f

    +noise

    Can read frequency from theangle!

    f

    unit circle

    x0 = ax1 = a f

    Pro: constant time algorithmCon: depends heavily on the signal being pure

    24 / 73

  • Two-point samplingInput signal x is a pure frequency, so xj = a f j

    +noise

    Sample x0,x1

    We have

    x0 = a

    +noise

    x1 = a f

    +noise

    So

    x1/x0 =f

    +noise

    Can read frequency from theangle!

    f

    unit circle

    x0 = ax1 = a f

    Pro: constant time algorithmCon: depends heavily on the signal being pure

    25 / 73

  • Two-point samplingInput signal x is a pure frequency, so xj = a f j

    +noise

    Sample x0,x1

    We have

    x0 = a

    +noise

    x1 = a f

    +noise

    So

    x1/x0 =f

    +noise

    Can read frequency fr