1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理

1

物理フラクチュオマティクス論Physical Fluctuomatics

応用確率過程論Applied Stochastic Process

第 5 回グラフィカルモデルによる確率的情報処理 5th Probabilistic information processing by means of

graphical model

東北大学大学院情報科学研究科応用情報科学専攻田中和之 (Kazuyuki Tanaka)[email protected]

http://www.smapip.is.tohoku.ac.jp/~kazu/

物理フラクチュオマティクス論 ( 東北大 )

2

今回の講義の講義ノート

田中和之著：確率モデルによる画像処理技術入門，

森北出版， 2006 ．

物理フラクチュオマティクス論 ( 東北大 )

物理フラクチュオマティクス論物理フラクチュオマティクス論 (( 東北大東北大 )) 33

ContentContentss

1.1. IntroductionIntroduction

2.2. Probabilistic Image ProcessingProbabilistic Image Processing

3.3. Gaussian Graphical ModelGaussian Graphical Model

4.4. Statistical Performance Statistical Performance AnalysisAnalysis

5.5. Concluding RemarksConcluding Remarks


ContentContentss






物理フラクチュオマティクス論 ( 東北大 ) 5

Markov Random Fields for Image Processing

S. Geman and D. Geman (1986): IEEE Transactions on PAMIS. Geman and D. Geman (1986): IEEE Transactions on PAMIImage Processing for Image Processing for Markov Random Fields (MRF)Markov Random Fields (MRF) (Simulated Annealing, Line Fields)(Simulated Annealing, Line Fields)

J. Zhang (1992): IEEE Transactions on Signal ProcessingJ. Zhang (1992): IEEE Transactions on Signal ProcessingImage Processing in EM algorithm for Image Processing in EM algorithm for Markov Markov Random Fields (MRF)Random Fields (MRF) (Mean Field Methods) (Mean Field Methods)

Markov Random Fields are One of Probabilistic Methods for Image processing.


Markov Random Fields for Image Processing

In Markov Random Fields, we have to consider not only the states with high probabilities but also ones with low probabilities.In Markov Random Fields, we have to estimate not only the image but also hyperparameters in the probabilistic model.We have to perform the calculations of statistical quantities repeatedly.

Hyperparameter Estimation

Statistical Quantities

Estimation of Image

We can calculate statistical quantities by adopting the Gaussian graphical model as a prior probabilistic model and by using Gaussian integral formulas.


Purpose of My TalkReview of formulation of probabilistic model for image processing by means of conventional statistical schemes.Review of probabilistic image processing by using Gaussian graphical model (Gaussian Markov Random Fields) as the most basic example.

K. Tanaka: Statistical-Mechanical Approach K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. to Image Processing (Topical Review), J. Phys. A: Math. Gen., vol.35, pp.R81-R150, Phys. A: Math. Gen., vol.35, pp.R81-R150, 2002.2002.

Section 2 and Section 4 are summarized in the present talk.


ContentContentss







Image Representation in Computer Vision

Digital image is defined on the set of points arranged on a square lattice.The elements of such a digital array are called pixels.We have to treat more than 100,000 pixels even in the digital cameras and the mobile phones.

xx

y y

)1,1( )1,2( )1,3(

)2,1( )2,2( )2,3(

)3,1( )3,2( )3,3(

),( yxPixels 200,307480640


Image Representation in Computer Vision

Pixels 65536256256

x

yAt each point, the intensity of light is represented as an integer number or a real number in the digital image data.A monochrome digital image is then expressed as a two-dimensional light intensity function and the value is proportional to the brightness of the image at the pixel.

yxfyx ,),(

0, yxf 255, yxf


Noise Reduction by Conventional FiltersNoise Reduction by Conventional Filters

173

110218100

120219202

190202192

Average

192 202 190

202 219 120

100 218 110

192 202 190

202 173 120

100 218 110

It is expected that probabilistic algorithms for image processing can be constructed from such aspects in the conventional signal processing.

Markov Random Fields Probabilistic Image ProcessingAlgorithm

Smoothing Filters

The function of a linear filter is to take the sum of the product of the mask coefficients and the intensities of the pixels.


Bayes Formula and Bayesian Network

Posterior Probability

}Pr{

}Pr{}|Pr{}|Pr{

B

AABBA

Bayes Rule

Prior Probability

Event A is given as the observed data.Event B corresponds to the original information to estimate. Thus the Bayes formula can be applied to the estimation of the original information from the given data.

A

B

Bayesian Network

Data-Generating Process


Image Restoration by Probabilistic Model

Original Image

Degraded Image

Transmission

Noise

Likelihood Marginal

PriorLikelihood

Posterior

}ageDegradedImPr{

}Image OriginalPr{}Image Original|Image DegradedPr{

}Image Degraded|Image OriginalPr{

Assumption 1: The degraded image is randomly generated from the original image by according to the degradation process. Assumption 2: The original image is randomly generated by according to the prior probability.

Bayes Formula


Image Restoration by Probabilistic Model

Degraded

Image

i

fi: Light Intensity of Pixel iin Original Image

),( iii yxr

Position Vector

of Pixel i

gi: Light Intensity of Pixel iin Degraded Image

i

Original

Image

The original images and degraded images are represented by f = {fi} and g = {gi}, respectively.


Probabilistic Modeling of Image Restoration

PriorLikelihood

Posterior



Viii fg ),(}|Pr{

}Image Original|Image DegradedPr{

fFgG

fg

Random Fieldsfi

gi

fi

gi

or

Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels.


Probabilistic Modeling of Image Restoration

PriorLikelihood

Posterior



Eij

ji ff ),(}Pr{

}Image OriginalPr{

fF

f

Random Fields

Assumption 2: The original image is generated according to a prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels.

i j

Product over All the Nearest Neighbour Pairs of Pixels


Prior Probability for Binary Image

== >p p p

2

1p

2

1i j Probability of Neigbouring Pixel

Eij

ji ff ),(}Pr{ fFi j

It is important how we should assume the function (fi,fj) in the prior probability.

)0,1()1,0()0,0()1,1(

We assume that every nearest-neighbour pair of pixels take the same state of each other in the prior probability.

1,0if


Prior Probability for Binary Image

Prior probability prefers to the configuration with the least number of red lines.

Which state should the center pixel be taken when the states of neighbouring pixels are fixed to the white states?

？

>

== >p p p

2

1p

2

1i j Probability of Nearest Neigbour Pair of Pixels


Prior Probability for Binary ImagePrior Probability for Binary Image

Which state should the center pixel be taken when the states of neighbouring pixels are fixed as this figure?

？ - ？== >

p p

> >=

Prior probability prefers to the configuration with the least number of red lines.


What happens for the case of large umber of pixels?

p 0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0

lnp

Disordered StateCritical Point

(Large fluctuation)

small p large p

Covariance between the nearest neghbour pairs of pixels

Sampling by Marko chain Monte Carlo

Ordered State

Patterns with both ordered statesand disordered states are often generated near the critical point.


Pattern near Critical Point of Prior Probability

0.0

0.2

0.4

0.6

0.8

1.0

0.0 0.2 0.4 0.6 0.8 1.0ln p

similar

small p large p

Covariance between the nearest neghbour pairs of pixels

We regard that patterns generated near the critical point are similar to the local patterns in real world images.


ContentContentss







Bayesian Image Analysis by Gaussian Graphical Model

Ejiji ff

Z },{

2

PR 2

1exp

)(

1|Pr

fF

0005.0 0030.00001.0

Patterns are generated by MCMC.

Markov Chain Monte Carlo Method

PriorProbability

,if

E:Set of all the nearest-neighbour pairs of pixels

V:Set of all the pixels

||21},{

2PR 2

1exp)( V

Ejiji dzdzdzzzZ



2,0~ Nfg ii

Viii gf 2

22 2

1exp

2

1,Pr

fFgG

Histogram of Gaussian Random Numbers

,, ii gf

Degraded image is obtained by adding a white Gaussian noise to the original image.

Degradation Process is assumed to be the additive white Gaussian noise.

V: Set of all the pixels

Original Image f Gaussian Noise n

Degraded Image g


Bayesian Image Analysis

Eji

jiVi

ii ffgf},{

),(),(

Pr

PrPrPr

gG

fFfFgGgGfF

fg

fF Pr fFgG Pr gOriginalImage

Degraded Image

Prior Probability


Degradation Process

Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability.

E ： Set of all the nearest neighbour pairs of pixels

V ： Set of All the pixels


Estimation of Original Image

We have some choices to estimate the restored image from posterior probability.

In each choice, the computational time is generally exponential order of the number of pixels.

}|Pr{maxargˆ gG iiz

i zFfi

2}|Pr{minargˆ zgGzF dzf iiii

}|Pr{maxargˆ gGzFfz

Thresholded Posterior Mean (TPM) estimation

Maximum posterior marginal (MPM) estimation

Maximum A Posteriori (MAP) estimation

if

ii fF\

}|Pr{}|Pr{f

gGfFgG

(1)

(2)

(3)


Statistical Estimation of Hyperparameters

zzFzFgGgG d}|Pr{},|Pr{},|Pr{

},|Pr{max arg)ˆ,ˆ(

,

gG

f g

Marginalized with respect to F

}|Pr{ fF },|Pr{ fFgG gOriginal Image

Marginal Likelihood

Degraded ImageVy

x

},|Pr{ gG

Hyperparameters are determinedso as to maximize the marginal likelihood Pr{G=g|,} with respect to ,


Bayesian Image Analysis

,, ji gf

TT2

POS

},{

22

2POS

2

1))((

2

1exp

,,

1

2

1

2

1exp

,,

1

},|Pr{

}|Pr{},|Pr{},,|Pr{

fCfgfgfg

g

gG

fFfFgGgGfF

Z

ffgfZ Eji

jiVi

ii

A Posteriori Probability

Gaussian Graphical Model

||21

TT2POS

2

1))((

2

1exp,, VdzdzdzZ zCzgzgzg


Average of Posterior Probability

)0,00())()((2

1exp)(

||

||21T2

V

V ,,dzdzdz

mzCI mz mz

gCI

I

CI

IzCI

CI

Iz

CI

IzCI

CI

Izz

gGzFzF

2

||21

T

22

22

||21

T

22

22

||21

2

1exp

2

1exp

},,|Pr{,

V

V

V

dzdzdzgg

dzdzdzgg

dzdzdz

Gaussian Integral formula


gCI

I

zgGzFzf

2

},,|Pr{ˆ

d

Bayesian Image Analysis by Gaussian Graphical Model ,, ii gf

otherwise0

},{1

4

Eji

ji

ji C

Multi-Dimensional Gaussian Integral Formula

Ejiji

Viii ffgf

},{

22

2 2

1

2

1exp},,|Pr{

gGfF


Average of the posterior probability can be calculated by using the multi-dimensional Gauss integral Formula

|V|x|V| matrix

E:Set of all the nearest-neghbour pairs of pixels



Statistical Estimation of Hyperparameters

)()2(

),,(}|Pr{},|Pr{},|Pr{

PR2/||2

POS

Z

Zd

V

gzzFzFgGgG

f g

Marginalized with respect to F

}|Pr{ fF },|Pr{ fFgG gOriginal Image

Marginal Likelihood

Degraded ImageVy

x

},|Pr{ gG

||21

TT2

POS

2

1))((

2

1exp

,,

Vdzdzdz

Z

zCzgzgz

g

||21

TPR

2

1exp VdzdzdzZ zCz


Calculations of Partition Function

AmzA mz

det

)2()( )(

2

1exp

||

||21T

V

Vdzdzdz

T22

||2

||21

T

22

22

T2

||21T

2

T

22

22

POS

2

1exp

)det(

)2(

2

1exp

2

1exp

2

1

2

1exp

),,(

gCI

Cg

CI

gCI

IzCIg

CI

Iz

gCI

Cg

gCI

Cgg

CI

IzCIg

CI

Iz

g

V

V

V

dzdzdz

dzdzdz

Z

(A is a real symmetric and positive definite matrix.)Gaussian Integral formula

CCzz

det

)2(

2

1exp)(

||

||

||21 V

V

VPR dzdzdzZ


Exact expression of Marginal Likelihood in Gaussian Graphical Model

T

22 2

1exp

det2

det},|Pr{ g

CI

Cg

CI

CgG

V

gCI

If

2ˆ

Multi-dimensional Gauss integral formula

Vy

x

},|Pr{max argˆ,ˆ,

gG We can construct an exact EM algorithm.

)()2(

),,(}|Pr{},|Pr{},|Pr{

PR2/||2

POS

Z

Zd

V

gzzFzFgGgG



1T

22

2

11||

1

11

1Tr

||

1

g

CI

Cg

CI

C

ttVtt

t

Vt

T22

242

2

2

11

11

||

1

11

1Tr

||

1g

CI

Cg

CI

I

tt

tt

Vtt

t

Vt

0

0.0002

0.0004

0.0006

0.0008

0.001

0 20 40 60 80 100 t

t

g f̂

gCI

Im

2)()()(

ttt

2)(minarg)(ˆ tmztf ii

zi

i

Iteration Procedure in Gaussian Graphical Model

f̂

g


Image Restoration by Markov Random Field Model and Conventional Filters

2ˆ||

1MSE

Viii ff

V

MSE

Statistical Method 315

Lowpass Filter

(3x3) 388

(5x5) 413

Median Filter

(3x3) 486

(5x5) 445

(3x3) Lowpass(3x3) Lowpass (5x5) Median(5x5) MedianMRFMRF

Original ImageOriginal Image Degraded ImageDegraded Image

RestoredRestoredImageImage



ContentContentss







Performance Analysis

1y

x

2y

3y

4y

5y

1x̂

2x̂

3x̂

4x̂

5x̂

Pos

teri

or P

rob

abil

ity

Estimated ResultsObserved

Data

Sample Average of Mean Square Error

SignalA

dd

itiv

e W

hit

e G

auss

ian

Noi

se

5

1

2||ˆ||5

1][MSE

nnxx


Statistical Performance Analysis

ydxXyYxyhV

x

},|Pr{),,(1

),|(MSE2

),,( yh

g

y

Additive White Gaussian Noise

},|Pr{ xXyY

x

},,|Pr{ yYxX


Restored Image

Original Image Degraded Image

},|Pr{ xXyY

Additive White Gaussian Noise


Statistical Performance Analysis

ydxXyYxyV

ydxXyYxyhV

x

Pr1

Pr,,1

),|MSE(

212

2

CI

y

xdyxPxyh

12

)|(,,

CI

)2

1exp(

2

1exp,Pr

22

22

yx

yxxXyYVi

ii

otherwise,0

},{,1

,4

Eji

Vji

ji C


Statistical Performance Estimation for Gaussian Markov Random Fields

T22

242

22

2

22

||

22

242T

22

||

22

2T

22

||

22

2T

22

||

22T

22

||

2

2

2

T

2

2

2

22

||2

2

2

2

22

||2

22

22

||2

2

2

)(||

1

)(Tr

1

2

1exp

2

1

)(

1

2

1exp

2

1)(

)(

1

2

1exp

2

1

)()(

1

2

1exp

2

1)(

)()(

1

2

1exp

2

1)()(

1

2

1exp

2

1)(

1

2

1exp

2

1)(

1

2

1exp

2

11

},|Pr{),,(1

),|(MSE

xI

xVV

ydxyxxV

ydxyxyxV

ydxyxxyV

ydxyxyxyV

ydxyxxyxxyV

ydxyxxyV

ydxyxxxyV

ydyxxyV

ydxXyYxyhV

x

V

V

V

V

V

V

V

V

C

C

CI

I

CI

C

CI

C

CI

C

CI

I

CI

C

CI

I

CI

C

CI

I

CI

C

CI

I

CI

I

CI

I

CI

I

= 0

otherwise,0

},{,1

,4

Eji

Vji

ji C


Statistical Performance Estimation for Gaussian Markov Random Fields

xI

xVV

ydyxxyV

ydxXyYxyhV

x

V

22

242T

22

2

22

||

2

2

2

2

)(||

1

)(Tr

1

2

1exp

2

11

},|Pr{),,(1

),|(MSE

C

C

CI

I

CI

I

otherwise,0

},{,1

,4

Eji

Vji

ji C

0

200

400

600

0 0.001 0.002 0.003

=40

0

200

400

600

0 0.001 0.002 0.003

=40

),|(MSE x ),|(MSE x


ContentContentss







Summary

Formulation of probabilistic model for image processing by means of conventional statistical schemes has been summarized.

Probabilistic image processing by using Gaussian graphical model has been shown as the most basic example.

ReferenceReferencessReferenceReferencess

K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) .

K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A, 35 (2002).

A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing, Proceedings of IEEE, 90 (2002).

K. Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) .

K. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A, 35 (2002).

A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing, Proceedings of IEEE, 90 (2002).



Problem 5-1: Derive the expression of the posterior probability Pr{F=f|G=g,,} by using Bayes formulas Pr{F=f|G=g,,}=Pr{G=g|F=f,}Pr{F=f,}/Pr{G=g|,}. Here Pr{G=g|F=f,} and Pr{F=f,} are assumed to be as follows:

Viii gf 2

22 2

1exp

2

1,Pr

fFgG

Ejiji ff

Z },{

2

PR 2

1exp

)(

1|Pr

fF

||21},{

2PR 2

1exp)( V

Ejiji dzdzdzzzZ

Ejiji

Viii ffgf

Z },{

22

2POS 2

1

2

1exp

,,

1},,|Pr{

ggGfF

||21

},{

22

2PR 2

1

2

1exp V

Ejiji

Viii dzdzdzzzgzZ

[Answer]


Problem 5-2: Show the following equality.

TT2

},{

22

2

2

1))((

2

1

2

1

2

1

fCfgfgf

Eji

jiVi

ii ffgf

otherwise0

},{1

4

Eji

ji

ji C


Problem 5-3: Show the following equality.

T

2

T

22

22

TT2

2

1

2

1

2

1))((

2

1

gCI

Cg

gCI

IzCIg

CI

Iz

zCzgzgz


Problem 5-4: Show the following equalities by using the multi-dimensional Gaussian integral formulas.

T22

||2

||21},{

22

2POS

2

1exp

)det(

)2(

2

1

2

1exp,,

gCI

Cg

CI

g

V

VEji

jiVi

ii dzdzdzffgfZ

Cdet

)2(

2

1exp)(

||

||

||21},{

2

V

V

VEji

jiPR dzdzdzffZ


Problem 5-5: Derive the extremum conditions for the following marginal likelihood Pr{G=g} with respect to the hyperparameters and .

T

22 2

1exp

det2

det},|Pr{ g

CI

Cg

CI

CgG

V

T

22

2

||

1Tr

||

11g

CI

Cg

CI

C

VV

T

22

242

2

22

||

1Tr

||

1g

CI

Cg

CI

I

VV

[Answer]


Problem 5-6: Derive the extremum conditions for the following marginal likelihood Pr{G=g} with respect to the hyperparameters and .

T

22 2

1exp

det2

det},|Pr{ g

CI

Cg

CI

CgG

V

T

22

2

||

1Tr

||

11g

CI

Cg

CI

C

VV

T

22

242

2

22

||

1Tr

||

1g

CI

Cg

CI

I

VV

[Answer]


Problem 5-7: Make a program that generate a degraded image by the additive white Gaussian noise. Generate some degraded images from a given standard images by setting =10,20,30,40 numerically. Calculate the mean square error (MSE) between the original image and the degraded image.

Histogram of Gaussian Random Numbers Fi Gi~N(0,402)

Original Image Gaussian Noise Degraded Image

Sample Program: http://www.morikita.co.jp/soft/84661/

2||

1MSE

Viii gf

V

K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 .


Problem 5-8: Make a program of the following procedure in probabilistic image processing by using the Gaussian graphical model and the additive white Gaussian noise.

1T

22

2

11||

1

11

1Tr

||

1

g

CI

Cg

CI

C

ttVtt

t

Vt

T22

242

2

2

11

11

||

1

11

1Tr

||

1g

CI

Cg

CI

I

tt

tt

Vtt

t

Vt

gCI

Im

2)()()(

ttt

2)(minarg)(ˆ tmztf ii

zi

i

Algorithm: Repeat the following procedure until convergence

Sample Program: http://www.morikita.co.jp/soft/84661/

K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 .

Documents

1 物理フラクチュオマティクス論 Physical Fluctuomatics 応用確率過程論 Applied Stochastic Process 第 5 回グラフィカルモデルによる確率的情報処理