Prévision consommation électrique par processus à valeurs fonctionnelles

Prévision d’un processus à valeurs fonctionnelles.Application à la consommation d’électricité.

Jaïro Cugliari

Groupe select, INRIA Futurs

16 décembre 2011

Directeurs de thèse: Anestis ANTONIADIS (Univ. Joseph Fourier)Jean-Michel POGGI (Univ. Paris Descartes)

Encadrement industriel: Xavier BROSSAT (EDF R&D) .

MotivationKernel-wavelet functional model

Clustering functional data using waveletsConditional Autoregression Hilbertian process

Concluding remarks

Outline

1 Motivation

2 Kernel-wavelet functional model

3 Clustering functional data using wavelets

4 Conditional Autoregression Hilbertian process

5 Concluding remarks

GT Prévision | Déc 2011 | J. Cugliari Prévision d’un processus à valeurs fonctionnelles.



Concluding remarks

Functional time seriesElectricity demand data

Outline

1 Motivation








Concluding remarks


FD as slices of a continuous process [Bosq, (1990)]

The prediction problem

Suppose one observes a square integrable continuous-time stochasticprocess X = (X(t), t ∈ R) over the interval [0,T ], T > 0;We want to predict X all over the segment [T ,T + δ], δ > 0Divide the interval into n subintervals of equal size δ.Consider the functional-valued discrete time stochastic processZ = (Zk , k ∈ N), where N = 1, 2, . . ., defined by

Xt

tT T + δ0




Concluding remarks





Xt

tT T + δ0




Concluding remarks





Xt

t1δ 2δ 3δ 4δ 5δ 6δ0 T + δ

Z1(t) Z2(t) Z5(t)

Z3(t) Z4(t) Z6(t)

Zk (t) = X(t + (k − 1)δ)

k ∈ N ∀t ∈ [0, δ)




Concluding remarks





Xt

t1δ 2δ 3δ 4δ 5δ 6δ0 T + δ

Z1(t) Z2(t) Z5(t)

Z3(t) Z4(t) Z6(t)

Zk (t) = X(t + (k − 1)δ)

k ∈ N ∀t ∈ [0, δ)

If X contents a δ−seasonal component, Z is particularly fruitful.




Concluding remarks


Prediction of functional time seriesLet (Zk , k ∈ Z) be a stationary sequence of H-valued r.v. Given Z1, . . . ,Zn wewant to predict the future value of Zn+1.

A predictor of Zn+1 using Z1,Z2, . . . ,Zn is

Zn+1 = E[Zn+1|Zn,Zn−1, . . . ,Z1].

Autoregressive Hilbertian process of order 1The arh(1) centred process states that at each k,

Zk = ρ(Zk−1) + εk (1)

where ρ is a compact linear operator and εkk∈Z is an H−valued strong whitenoise.Under mild conditions, equation (1) has a unique solution which is a strictlystationary process with innovation εkk∈Z. [Bosq, (1991)]When Z is a zero-mean arh(1) process, the best predictor of Zn+1 givenZ1, . . . ,Zn−1 is:

Zn+1 = ρ(Zn).




Concluding remarks


Prediction of functional time seriesLet (Zk , k ∈ Z) be a stationary sequence of H-valued r.v. Given Z1, . . . ,Zn wewant to predict the future value of Zn+1.

A predictor of Zn+1 using Z1,Z2, . . . ,Zn is

Zn+1 = E[Zn+1|Zn,Zn−1, . . . ,Z1].

Autoregressive Hilbertian process of order 1The arh(1) centred process states that at each k,

Zk = ρ(Zk−1) + εk (1)

where ρ is a compact linear operator and εkk∈Z is an H−valued strong whitenoise.Under mild conditions, equation (1) has a unique solution which is a strictlystationary process with innovation εkk∈Z. [Bosq, (1991)]When Z is a zero-mean arh(1) process, the best predictor of Zn+1 givenZ1, . . . ,Zn−1 is:

Zn+1 = ρ(Zn).




Concluding remarks


Electricity demand dataSome salient features

(a) Long term trend. (b) Annual and week cycles.

(c) Daily pattern. (d) Demand (in Gw/h) as a function oftemperature (in C)




Concluding remarks


Electricity demand forecast

Short-term electricity demand forecast in literature

Time series analysis: sarima(x), Kalman filter [Dordonnat et al. (2009)]

Machine learning. [Devaine et al. (2010)]

Similarity search based methods. [Poggi (1994), Antoniadis et al. (2006)]

Regression: edf modelisation scheme [Bruhns et al. (2005)] , gam [Pierrot andGoude (2011)]

New challenges

Market liberalization: may produce variations on clients’ perimeter thatrisk to induce nonstationarities on the signal.Development of smart grids and smart meters.

But, almost all the models rely on a monoscale representation of the data




Concluding remarks


Electricity demand forecast

Short-term electricity demand forecast in literature

Time series analysis: sarima(x), Kalman filter [Dordonnat et al. (2009)]

Machine learning. [Devaine et al. (2010)]

Similarity search based methods. [Poggi (1994), Antoniadis et al. (2006)]

Regression: edf modelisation scheme [Bruhns et al. (2005)] , gam [Pierrot andGoude (2011)]

New challenges

Market liberalization: may produce variations on clients’ perimeter thatrisk to induce nonstationarities on the signal.Development of smart grids and smart meters.

But, almost all the models rely on a monoscale representation of the data




Concluding remarks

WaveletsPrediction algorithmCorrections to handle nonstationarity

Outline

1 Motivation








Concluding remarks


Kernel regression for functional time series [Antoniadis et al. (2008)]

For a more general class of process

The regression function E[Zn+1|Zn,Zn−1, . . . ,Z1] can be estimated by anonparametric approach.Key idea: similar futures correspond to similar pasts.

The resulting predictor Zn+1(t) of Zn+1

is obtained by a kernel regression of Zn over the history Zn−1, . . . ,Z1.is a weighted mean of futures of past segments. Weights increase withsimilarity between last observed segment n and past segmentsm = 1, . . . , n,

Zn+1(t) =

n−1∑m=1

wn,mZm+1(t)




Concluding remarks


Kernel regression for functional time series [Antoniadis et al. (2008)]

For a more general class of process

The regression function E[Zn+1|Zn,Zn−1, . . . ,Z1] can be estimated by anonparametric approach.Key idea: similar futures correspond to similar pasts.

The resulting predictor Zn+1(t) of Zn+1

is obtained by a kernel regression of Zn over the history Zn−1, . . . ,Z1.is a weighted mean of futures of past segments. Weights increase withsimilarity between last observed segment n and past segmentsm = 1, . . . , n,

Zn+1(t) =

n−1∑m=1

wn,mZm+1(t)


Wavelets to cope with fd

domain-transform techniquefor hierarchical decomposingfinite energy signalsdescription in terms of abroad trend (approximationpart), plus a set of localizedchanges kept in the detailsparts.

Discrete Wavelet TransformIf z ∈ L2([0, 1]) we can write it as

z(t) =

2j0−1∑k=0

cj0,kφj0,k (t) +

∞∑j=j0

2j−1∑k=0

dj,kψj,k (t),

where cj,k =< g , φj,k >, dj,k =< g , ϕj,k > are the scale coefficients andwavelet coefficients respectively, and the functions φ et ϕ are associated to aorthogonal mra of L2([0, 1]).



Concluding remarks


Approximation and details

In practice, we don’t dispose of the whole trajectory but only with a(possibly noisy) sampling at 2J points, for some integer J .Each approximated segment Zi,J (t) is broken up into two terms:

a smooth approximation Si (t) (lower freqs)a set of details Di (t) (higher freqs)

Zi,J (t) =

2j0−1∑k=0

c(i)j0,kφj0,k (t)︸︷︷︸Si (t)

+

J−1∑j=j0

2j−1∑k=0

d (i)j,kψj,k (t)︸︷︷︸

Di (t)

The parameter j0 controls the separation. We set j0 = 0.

zJ (t) = c0φ0,0(t) +

J−1∑j=0

2j−1∑k=0

dj,kψj,k (t).




Concluding remarks


A two step prediction algorithm

Step I: Dissimilarity between segmentsSearch the past for segments that are similar to the last one.For two observed series of length 2J say Zm and Zl we set for each scale j ≥ j0:

distj (Zm,Zl ) =

(2j−1∑k=0

(d (m)j,k − d (l)

j,k )2

)1/2

Then, we aggregate over the scales taking into account the number ofcoefficients at each scale

D(Zm,Zl ) =

J−1∑j=j0

2−j/2distj (Zm,Zl )




Concluding remarks


A two step prediction algorithm

Step 2: Kernel regressionObtain the prediction of the scale coefficients at the finest resolutionΞn+1 = c(n+1)

J,k : k = 0, 1, . . . , 2J − 1 for Zn+1

Ξn+1 =

n−1∑m=1

wm,nΞm+1

wm,n =K(D(Zn,Zm)

hn

)∑n−1m=1 K

(D(Zn,Zm)hn

)Finally, the prediction of Zn+1 can be written

Zn+1(t) =

2J−1∑k=0

c(n+1)J,k φJ,k (t)




Concluding remarks


Corrections proposed to handle nonstationarity

On mean level

base Sn+1(t) =∑n−1

m=1 wm,nSm+1(t)

prst Sn+1(t) = Sn(t)

diff Sn+1(t) = Sn(t) +∑n−1

m=2 wm,n∆(Sm)(t)

Figure: Daily prediction error (in mapex100).




Concluding remarks



On mean level


m=1 wm,nSm+1(t)



m=2 wm,n∆(Sm)(t)





Concluding remarks



On mean level


m=1 wm,nSm+1(t)



m=2 wm,n∆(Sm)(t)





Concluding remarks


Corrections proposed to handle nonstationarityOn groups by post-treatment

Define new weights and renormalize.

wm,n =

ww,m if gr(m) = gr(n)0 otherwise

gr(n) is the group ofthe n-th segment.

1 Deterministic groups: Calendar or Calendar transitions.2 Groups learned from data via clustering analysis. (e.g. temperature curves)3 Cross deterministic with clustering groups (e.g. calendar-temperature

transitions).





Concluding remarks


Corrections proposed to handle nonstationarityOn groups by post-treatment

Define new weights and renormalize.

wm,n =

ww,m if gr(m) = gr(n)0 otherwise

gr(n) is the group ofthe n-th segment.

1 Deterministic groups: Calendar or Calendar transitions.2 Groups learned from data via clustering analysis. (e.g. temperature curves)3 Cross deterministic with clustering groups (e.g. calendar-temperature

transitions).



Predict of 10 September 2005 (Saturday)

SimilIndex date SimilIndex

2004-09-10 0.4552003-09-05 0.1412002-09-06 0.0832004-09-03 0.0702003-09-19 0.0682000-09-08 0.0582000-09-15 0.0191999-09-10 0.017

similar past similar future



Concluding remarks

Clustering via feature extraction

Outline

1 Motivation








Concluding remarks


Aim

Segmentation of X may not suffices torender reasonable the stationaryhypothesis.If a grouping effect exists, we mayconsidered stationary within each group.Conditionally on the grouping, functionaltime series prediction methods can beapplied.We propose a clustering procedure thatdiscover the groups from a bunch ofcurves.

We use wavelet transforms to take into accountthe fact that curves may present non stationarypatters.

Two strategies to clusterfunctional time series:

1 Feature extraction(summary measures of thecurves).

2 Direct similarity betweencurves.




Concluding remarks


Aim

Segmentation of X may not suffices torender reasonable the stationaryhypothesis.If a grouping effect exists, we mayconsidered stationary within each group.Conditionally on the grouping, functionaltime series prediction methods can beapplied.We propose a clustering procedure thatdiscover the groups from a bunch ofcurves.

We use wavelet transforms to take into accountthe fact that curves may present non stationarypatters.

Two strategies to clusterfunctional time series:

1 Feature extraction(summary measures of thecurves).

2 Direct similarity betweencurves.




Concluding remarks


Energy decomposition of the DWT

Energy conservation of the signal

‖z‖2 ≈ ‖zJ‖22 = c20,0 +

J−1∑j=0

2j−1∑k=0

d2j,k = c2

0,0 +

J−1∑j=0

‖dj‖22.

For each j = 0, 1, . . . , J − 1, we compute the absolute and relativecontribution representations by

contj = ||dj||2︸︷︷︸AC

and relj =||dj||2∑j ||dj||2︸︷︷︸

RC

.

They quantify the relative importance of the scales to the global dynamic.RC normalizes the energy of each signal to 1.




Concluding remarks


Schema of procedure

0. Data preprocessing. Approximate sample paths of z1(t), . . . , zn(t)1. Feature extraction. Compute either of the energetic components using absolute

contribution (AC) or relative contribution (RC).2. Feature selection. Screen irrelevant variables. [Steinley & Brusco (’06)]

3. Determine the number of clusters. Detecting significant jumps in the transformeddistortion curve. [Sugar & James (’03)]

4. Clustering. Obtain the K clusters using PAM algorithm.




Concluding remarks


Electricity demand data

Feature extraction:- Significant scales are associated to mid-frequencies.- The retained scales parametrize the represented cycles of 1.5, 3 and6 hours (AC) and to the cycles of 30 minutes, 1.5 and 3 hours (RC).

Number of clusters:8 for AC and 5 for RC.

For instance, we found a class of segments that can be recognized assummer Monday




Concluding remarks


Towards a dissimilarity based onwavelet coherence

Distance based on wavelet-correlation between two time series.Can be used to measure relationship between two (possibly nonstationary) time series, i.e. temperature and load.The strength of the relationship is hierarchically decomposed on scales(≈frequencies), without loose of time location.


(a) Curves (b) Calendar

Figure: Curves membership of the clustering using wer based dissimilarity (a)and the corresponding calendar positioning (b).



Concluding remarks

carh: Conditional Autoregressive Hilbertian ModelSome results

Outline

1 Motivation








Concluding remarks


carh process

Let (Z ,V ) = (Zk ,Vk ), k ∈ Z be stationary sequences of H ×Rd− valued r.v.defined over (Ω,F ,P).

We will focus on the behaviour of Z conditioned to V .(Z ,V ) is a carh(1) if it is stationary and and such that,

Zk = a + ρVk (Zk−1 − a) + εk , k ∈ Z, (2)

where for each v ∈ Rd , av = Ev [Z0|V ], εkk∈Z is an H−white noiseindependent of V , and ρVk k∈Z is a sequence of linear compact operators.




Concluding remarks


Simulation and predictionWe extend the simulation strategies for arh processes [Guillas & Damon(2000)] to the simple case of an carh process with d = 1 and V is a i.i.d.sequence of Beta(β1, β2) rv.Numerical experience: prediction of the electricity demand using thetemperature as exogenous information

Figure: Prediction of one simulated curve of an carh process (full line).GT Prévision | Déc 2011 | J. Cugliari Prévision d’un processus à valeurs fonctionnelles.



Concluding remarks

Outline

1 Motivation






Concluding remarks

We study the problem of prediction of E[Zn+1|Zn,Vn+1] for H−valued rv.Stationary assumptions are held conditionally on an exogenous rv V .First, V is multiclass and prediction is done by kwf.When the states of V are unknown, clustering is used to discover them.Last, carh uses V ∈ Rd and arh ideas.

The contribution of wavelets

To exploit information from past data that was observed under a differentregime from the actual one.Appropriate tool to detect useful similarities between nonstationarypatterns of rough trajectories.Allow fast computations: online prediction version of kwf.

The contribution of the kernel regression on fts

Accurate alternative for heavy parametric model.Interpretation ability through the study of past similar behaviours.Allow fast computations: online prediction version of kwf.Special attention must be paid to transitions.



Concluding remarks

Some referencesA. Antoniadis, X. Brossat, J. Cugliari, and J.-M. Poggi.Clustering functional data using wavelets.arXiv:1101.4744, 2011.

A. Antoniadis, E. Paparoditis, and T. Sapatinas.A functional wavelet-kernel approach for time series prediction.Journal of the Royal Statistical Society, Series B, Methodological, 68(5):837, 2006.

D. Bosq.Linear processes in function spaces: Theory and applications.Springer-Verlag, New York, 2000.

A. Mas.Estimation d’opérateurs de corrélation de processus fonctionnels: lois limites, tests,déviations modérées.PhD thesis, Université Paris 6, 2000.

J.-M. Poggi.Prévision nonprametrique de la consommation électrique.Rev. Statistiqué Appliquée, XLII(4):93–98, 1994.

www.math.u-psud.fr/[email protected]


www.math.u-psud.fr/~cugliari

[email protected]

Technology

Prévision consommation électrique par processus à valeurs fonctionnelles