Embed Size (px)
Citation preview
ガウス分布の最尤推定問題を解くために微分システムを
作ってみた
2010/05/30
PRML読書会復習レーン#2
id:(t)yatsuta
PRML 本レーンで明かされた衝撃の事実
ゆとりは微分しない
> f <- expression(a*x^2+b*x+c)> D(f, "x")a * (2 * x) + b
Rで微分
Wolfram Alphaで微分
面白いのでHaskellで
作ってみました
ただし、標的はガウス関数のみ
（LTのお題）
参考文献
「微分」とは「式の変形」
・ddxsin x =cosx
・ddxexpx =expx
・ddxxn=nxn−1 など…
「式の変形」を表現するため「式」そのものを表現する
データ構造がほしい
●Cなら構造体＋共用体●C++/Java/Python…ならクラス
「式」をHaskellの型で表現
type VarName = String
data Expr = Num Double | Var VarName | Add [Expr] | Mult [Expr] | Minus Expr | Recip Expr | Pow Expr Expr | Exp Expr | Log Expr | Sqrt Expr | Sum VarName [VarName] Expr Expr Expr | Prod VarName [VarName] Expr Expr Expr deriving (Eq, Ord, Show)
Mult [Recip (Sqrt (Mult [Num 2, Var "PI", Var "sigma^2"])), Exp (Minus (Mult [Recip (Mult [Num 2, Var "sigma^2"]), (Pow (Add [Var "x", Minus (Var "mu")] ) (Num 2))]))]
例：ガウス関数
gaussian = Mult [Recip (Sqrt (Mult [Num 2, Var "PI", Var "sigma^2"])), Exp (Minus (Mult [Recip (Mult [Num 2, Var "sigma^2"]), (Pow (Add [Var "x", Minus (Var "mu")] ) (Num 2))]))]
1
22exp−
x−2
22
gaussian = Mult [Recip (Sqrt (Mult [Num 2, Var "PI", Var "sigma^2"])), Exp (Minus (Mult [Recip (Mult [Num 2, Var "sigma^2"]), (Pow (Add [Var "x", Minus (Var "mu")] ) (Num 2))]))]
1
22exp−
x−2
22
gaussian = Mult [Recip (Sqrt (Mult [Num 2, Var "PI", Var "sigma^2"])), Exp (Minus (Mult [Recip (Mult [Num 2, Var "sigma^2"]), (Pow (Add [Var "x", Minus (Var "mu")] ) (Num 2))]))]
1
22exp−
x−2
22
gaussian = Mult [Recip (Sqrt (Mult [Num 2, Var "PI", Var "sigma^2"])), Exp (Minus (Mult [Recip (Mult [Num 2, Var "sigma^2"]), (Pow (Add [Var "x", Minus (Var "mu")] ) (Num 2))]))]
1
22exp−
x−2
22
logLikelyhood = Log (Prod "n" ["x"] (Num 1) (Var "N") gaussian)
log∏n=1
N1
22exp−
xn−2
22
logLikelyhood = Log (Prod "n" ["x"] (Num 1) (Var "N") gaussian)
log∏n=1
N1
22exp−
xn−2
22
logLikelyhood = Log (Prod "n" ["x"] (Num 1) (Var "N") gaussian)
log∏n=1
N1
22exp−
xn−2
22
logLikelyhood = Log (Prod "n" ["x"] (Num 1) (Var "N") gaussian)
log∏n=1
N1
22exp−
xn−2
22
「式」を扱う関数を実装
●eval●simplify
●deriv●solve
●eval●simplify
●deriv●solve
eval その１
式の単純化
●expr + 0 = expr●expr * 0 = 0●expr * 1 = expr●Minus expr = -1 * expr●Recip expr = 1/x●その他、逆関数の相殺、約分など
eval その２
変数への代入
standardGaussian = eval gaussian [("PI", Num 3.141592), ("mu", Num 0), ("sigma^2", Num 1)]
> standardGaussian Mult [Num 0.3989423219002315, Exp (Mult [Num (-0.5), Pow (Var "x") (Num 2.0)])]
eval その３
数値計算
> eval standardGaussian [("x", Num 0)]Num 0.3989423219002315
> eval standardGaussian [("x", Num 1)]Num 0.24197074968943716
> eval standardGaussian [("x", Num 2)]Num 5.399097212943975e-2
> dnorm(c(0,1,2))[1] 0.39894228 0.24197072 0.05399097
Rでは…
●eval●simplify
●deriv●solve
simplify expr = eval expr []
●eval●simplify
●deriv●solve
「微分」とは「式の変形」↓
式変形の規則をHaskellで記述
dvdv
=1,∂ x∂ v
=0
deriv' (Var x) v | x == v = Num 1 | otherwise = Num 0
deriv' (Num _) _ = Num 0
dCdv
=0, C∈N
deriv' (Add exprs) v = Add [deriv' expr v | expr <- exprs]
ddv
fgh…=dfdv
dgdv
dhdv…
ddv
fgh…=dfdv
gh…fddv
gh…
deriv' (Mult [expr]) v = deriv' expr vderiv' (Mult (e:exprs)) v = Add [Mult [e, deriv' (Mult exprs) v], Mult [deriv' e v, Mult exprs]]
deriv' (Pow x n) v = Mult [Mult [n, Pow x (Add [n, Minus (Num 1)])], deriv' x v]
dxn
dv=nxn−1
dxdv
・ddvlog x =
1xdxdv
・ddvexpx =expx
dxdv
・ddv∑n=1
N
x =∑n=1
Ndxdv
deriv expr v = simplify (deriv' (simplify expr) v)
式変形(deriv')の前後で式を単純化●変形前はMinus、Recip、Prodなど、微分が定義されていない式を含む可能性●変形後は0の加算乗算、実行されていない数値演算などを含む可能性
> deriv logLikelyhood "mu"Mult [Add [Mult [Num (-1.0),Var "N",Var "mu"],Sum "n" ["x"] (Num 1.0) (Var "N") (Var "x")],Pow (Var "sigma^2") (Num (-1.0))]
−N∑n=1
N
xn1
2
●eval●simplify
●deriv●solve
> let dllh = deriv logLikelyhood "mu"> solve dllh (Num 0) "mu"Mult [Pow (Var "N") (Num (-1.0)),Sum "n" ["x"] (Num 1.0) (Var "N") (Var "x")]
N−1∑n=1
N
xn
ご清聴ありがとうございました
