Federico Tessmann, Ferienakademie TUM Efficiant polynomial interpolation algorithms

Federico Tessmann, Ferienakademie TUM

Efficiant polynomial interpolation algorithms


Overview

Introduction to Vandermonde Matrices and its utilities

Univariate Interpolation

Multivariate Interpolation


Properties of Vandermonde Matrices

Easy to ensure that they are non-singular

Systems of linear equations whose coefficients form Vandermonde matrices are easy to solve exactly


The Vandermonde Matrix

1

122

111

1

1

1

nnn

n

n

n

kk

kk

kk

V


Generalized Vandermonde

n

n

n

en

en

en

eee

eee

n

kkk

kkk

kkk

V

21

21

21

222

111

neee 21where


Determinant of a Vandermonde

121

1

12

21

212

1

122

111

1

1

001

1

1

1

det

kkkkk

kkkkk

kk

kk

kk

V

nn

nnn

nn

nnn

n

n

n



njjn

nn

nnn

nn

kkV

kkkkk

kkkkk

111

121

1

12

21

212 )(det

1

1

001

nj

jnj

jnnj

jnn kkkkVkkVV1

12

221

11 )()(det)(detdet

nji

ijn kkV1

)(det



nji

ijn kkV1

)(det

The Vandermonde matrix

is non-singular the ki are distinct


The previous result can not be applyed for generalized Vandermonde matrices

Example

wich is 0 also when

ee

e

e

kkk

kV 12

2

12

1

1det

11

2

e

k

k


Non-singularity of generalized Vandermonde matrices

Proposition 1:

If the ki are distinct positiv real numbers

=> the matrix is non-zero


The inverse of a Vandermonde matrix

)()()(

)()()(

)()()(

*

1

1

1

21

22221

11211

21

22221

11211

1

122

111

nnnn

n

n

nnnn

n

n

nnn

n

n

kPkPkP

kPkPkP

kPkPkP

ppp

ppp

ppp

kk

kk

kk

12321)( n

injijijjij kpkpkppkP


The inverse of a Vandermonde matrix

)()()(

)()()(

)()()(

21

22221

11211

nnnn

n

n

n

kPkPkP

kPkPkP

kPkPkP

E

ni

ji ij

ij kk

kZZP

1

)(


Solving a Vandermonde system of equations

nnn

nnn

nn

nn

wxkxkxkx

wxkxkxkx

wxkxkxkx

13

221

21

232

2221

11

132

1211



Tnxxx ),,( 1

T

nwww ),,( 1

wxVn

wVxVV nnn

11

wVx n

1



nnnnn

n

n

n w

w

w

ppp

ppp

ppp

x

x

x

1

1

21

22221

11211

2

1

niniii wpwpwpx 2211

ni

ni

i wZZPcoefwZZPcoefx )),(()),(( 11

11

11)( n

njjj ZppZP


The algorithm to solve the system

ni

ikZZQ1

)()(

ii kZ

ZQZQ

)()(

)(

)()(

ii

ii kQ

ZQZP


The algorithm to solve the systemThe computation of the xi is arranged as follows:

),(

),(

),(

),(

),(

),(

1

1

0

111

111

011

2

1

nnn

nn

nn

nn ZPcoefw

ZPcoefw

ZPcoefw

ZPcoefw

ZPcoefw

ZPcoefw

x

x

x

Calculate each vector and add it to the accumulating X


Analysis of the algorithm

1. By calculating the vectors one after the other we only need to compute one Pi(Z) at the time

2. Each Pi(Z) only needs O(n) time and since we have n polinoms to compute, the complexity is O(n2) and the space needed is O(n)

3. Because the inverse of the transposed matrix is the transpose of the inverse of the matrix, the algorithm only need a little adjustment to solve a transposed Vandermonde system of equations

On the Appendix there is an example of this alorithm taken from Zippel


Univariate Interpolation

Lagrange Interpolation

Newton Interpolation

Abstract Interpolation



nnZpZppZP 10)(

nzzz ,,, 10

Giving are a set of distinct evaluation points with its correspondating functional values

ii wzP )(

The goal is to find the polinome



nn

nnnn

nn

nn

wzpzpzpp

wzpzpzpp

wzpzpzpp

2210

112

12110

002

02010

ni

ni

i wZZPcoefwZZPcoefp )),(()),(( 00

This is a Vandermonde system where



n

nn

nn

nn

nn

ZwZZPcoefwZZPcoef

ZwZZPcoefwZZPcoef

wZZPcoefwZZPcoefZP

))),(()),(((

))),(()),(((

)),(()),(()(

00

10

10

00

00

nn wZPwZPZP )()()( 00



nn wZPwZPZP )()()( 00

nj

j

ji

ni ij

i wzz

zZZP

0 0

)(


Newton Interpolation

f(a)=f(x)(mod (x-a))


The Chinese remainder algorithm over ZZ

)(mod

)(mod

22

11

akx

akx

)(mod)mod()( 21121

1121 aaaaakkkx

1),gcd(mod 2121

1 aaaa


Chinese remainder with polinoms

When given and nzz ,,1 ii wzf Then we change it to the following situation:Given

Compute

)(mod)( ii pwXf ii zxp

))(mod()( 10 nn

n ppaxaxf


Newton Interpolation algorithm

1. Let f(x)=0, q(x)=1

2. Loop for n times doing following:1. f(x)=f(x)+q(ki)-1q(x)(wi-f(ki))

2. q(x)=(x-ki)q(x)


Newton´s interpolation formula

Let

Newton´s interpolation formula claims that there exist constants such that

In fact, and is the solution of

)( ii kff

))(()()( 212110 kxkxkxxf

110 f

i

)( 12112 kkff


Newton´s interpolation formula

)()(

)()())(()()(

11

111

1)(

lll

llll

ll

kkkk

kxkxkffxfxf

)()( 112

121

)2( kXkk

fffxf

)()( 111 llllll kkkkff

Then

And more generally

Solving the givesi


Multivariate Interpolation

Dense Interpolation

Probabilistic Sparse Interpolation

Deterministic Sparse Interpolation without degree bounds


Multivariate dense InterpolationWe are given a black box with a degree bound „d“ for

the polinom P(xi,..,xn)

So we can assume that P has the form

),,(),,(),,( 21201 ndd

nn XXPXXXPXXP

P

),,,(

),,,(

),,,(

0201

0201

02010

n

ndP

nP

xxXP

xxx

xxx


Multivariate dense Interpolation

So we get the values of

which are the coeficients found by interpolating P on X1

By doing this procedure we compute recursively P(X1,...,Xk,x(k+1)0,...,xn0)

),,( 020 ni XxP


Multivariate dense Interpolation

),,,,(

),,,(

),,,(

),,,(

),,,(

),,,(

),,,(

03021

021

021

0210

0201

0201

02010

n

nd

nddP

ndP

n

ndP

nP

xxXXP

xxXP

xxx

xxx

xxXP

xxx

xxx


The complexity of the dense interpolationLet I(d) be the complexity of interpolating d+1 values to produce a

univariate plynomial of degree „d“ and Nk the complexity for the first k variables

)()1(

)()1()1(

)()1()1(

)(

1

11

12

1

dIdkN

dIdNdN

dIdNdN

dIN

kk

kkk



Formal Presentation

Example

Analysis


Probabilistic Sparse InterpolationAssume we want to dermine P(X1,..., Xn) which is an element of L[X]

where L is a field of cardinal q and the degree of each Xi is bounded by „d“ and there are no more than T non-zero monomials

TeT

en XcXcXXP

1

11 ),,(

),,(),,,,,(

,,,,,,

),,(

:

111

11

1

nknkk

kkk

T

XXPskelxxXXskelP

eeeskelPeeePskel

eeskelP

Def



),,(),,,,,(,

),,(

111

1

nknkk

n

XXPskelxxXXskelPk

xx

Def:

is a precise evaluation point if:



Tkk enkTk

enkkn XXXcXXXcXXP

),,(),,(),,( 1111

1

q

dTnn

q

dT

q

dTn

2

)1()1(

q

dTkn )(

The probability by wich is an imprecise evaluation point:

For each k we can write

It is an imprecise evaluation point if one of the cik = 0And the probability that this happends is no more than

),,( 1 nxx



Given is a k-1 tuple The probability that

is 0 if we are we are working on a field of characteristic 0 or at least

When working on a field of q elements the

probability is bounded by

),,( 11 kyyy

jiee eeyy ji

,

dkp )32( 1

q

dTT

2

)1(



So the following probability is then one that underlines the Probabilistic Sparse Interpolation

q

ntdT

q

dTnn

q

dTT

2

)2(

2

)1(

2

)1( 2


Probabilistic Sparse InterpolationAssume we want to dermine P(X1,..., Xn) which is an element of L[X] where L is a field of cardinal q

and the degree of each Xi is bounded by „d“ and there are no more than T non-zero monomials

As in the dense interpolation we Interpolate TeT

en XcXcXXP

1

11 ),,(

),,(),,(),,( 21201 ndd

nn XXPXXXPXXP

),,,(

),,,(

),,,(

0201

0201

02010

n

ndP

nP

xxXP

xxx

xxx



q

dTknkn

2

))(1(1

TeT

enkk XpXpxxXXP

010,00,11

1),,,,(

kk eePskel

,,1

At the kth stage the first computation gives us:

We then assume that

The probability of that being the right skeleton is

We then pick a (k-1) tuple

And we set up the following transposed Vandermonde system of linearecuations

),,( 11 kyyy



T

T

T

eTTj

eTjkj

Tk

T

eTj

ejkjk

eTj

ejkjk

Tjjkj

ypypxyyP

ypypxyyP

ypypxyyP

ppxP

1

1

1

111

221

21

21

111

1

),,,(

),,,(

),,,(

),1,,1(

So each of the can be computed using O(n2) and we can avoid computing the other interpolations

ijp



The probability that the Vandermonde system of equation is non-singular is bounded by

q

dTT

2

)1(1



So we get

for each k

Then we solve

trough the dense interpolation

We then expand it and we get

And we are ready to compute the (k+1)th stage

TeTj

ejkjk XpXpxXXP

11,11 ),,(

)( kii XPp

TeT

enkk XpXpxxXXP

01000)1(,1

0),,,,(


Probabilistic Sparse InterpolationExample

Lets assume we are given a Black Box representing the following polinom

ZYXYZXYZXZXZYXP 554525),,(

532

41

50

17165

15

052

050

545

3

012

010

215

0

005

000

)(

),,(

),,(

),,(

),,(),,(

),,(

YdXYdYdXd

cXcXc

zyx

zyx

cXcXc

zyx

zyx

cXcXc

zyx

zyx

P

P

P

P

P

P



Given are a bound on the number of non-zero terms „T“ and the number of variables „n“

We want to compute

By choosing a distinct prime for each Xi then the quantities will all be distinct.

Let

Then we get:

TeT

e XcXcXP

11)(

1ei Xm

jj

TTj vmcmc 11

),,( 1j

njj xxp

)( j

j pPv



111

11

111

11

01

tt

TTt

TT

T

vmcmc

vmcmc

vcc

The rank of the system of equations is exactly the number of non-zero monomials in P

This could be easily done by taking the first T equations and computing their rank which requires O(T3)



Let

and consider

so

consider also

01 )()()( qZqmZmZZQ ttt

iti

itt

ij

tjj qmcmcmQc )()(

111

1

00110 qvqvv ttt

iti

kitt

kij

kj

tjj qmcmcmQmc )()(

111

1



Then we get the following Toeplitz system of linear equations

1212201

1101

1100

tttt

ttt

ttt

vqvqv

vqvqv

vqvqv



1

122

111

2

1

112

11

21

221

21

110

1

1

1

00

00

00111

ttt

t

t

ttt

tt

t

ttt

t

t

mm

mm

mm

c

c

c

mmm

mmm

vvv

vvv

vvv

So the system is non-singular if the mi are distinct



1212201

1101

1100

tttt

ttt

ttt

vqvqv

vqvqv

vqvqv

•So the system can be solved by Gaussian elimination O(t3)

•So then we get Q(Z)

•In order to find the mi we just need to find the zeroes of Q which are in fact positive integers making this procedure much easier

•Knowing the mi, the ei can easily be determined by factoring each of the mi, which is in fact very easy because the possible divisors are the first n primes allready known

•By knowing the mi it is allso easy to compute the ci just by solving the Vandermonde system, formed by the first t equations


The End

Questions?

Bibliography: Richard Zippel


Appendix

pseudo-codes for the interpolation algorithms


Code for Vandermonde Matrices

);(

}

}

);),((][][:][

{1:

]);[(/)(:)(

]);[),((:)(

{1:

];0;...;0[][:[]

]);[(])1[(:)(

]){[],[(

1

xreturn

ZZPcoefiwjxjx

dontojfor

ikQZQZP

ikZZQntPolyQuotieZQ

dontoifor

nnewArrayx

nkZkZZQ

nwnkrmondeSolveVande

ji

iii

i


Code for Lagrange Interpolation

);(

}

);(][:

]);[(/)(:)(

]);[),((:)(

{1:

;0:

]);[(])1[(:)(

]){[],[(

freturn

ZPiwff

ikQZQZP

ikZZQntPolyQuotieZQ

dontoifor

ialnewPolinomf

nkZkZZQ

nwnknterpolatioLagrangeIn

i

iii

i


Code for Newton Interpolation

);(

}

;])[(:

]));[(][()(])[(:

{1:

;0:

;1:

]){[],[(

1

freturn

qikXq

ikfiwXqikqff

dontoifor

ialnewPolinomf

ialnewPolinomq

nwnkrpolationNewtonInte

Documents

Federico Tessmann, Ferienakademie TUM Efficiant polynomial interpolation algorithms