Bayesian Network
By Zhang Liliang
Key Point Today
• Intro to Bayesian Network• Usage of Bayesian Network• Reasoning BN: D-separation
Bayesian Network Definition
Difficulty {easy, hard}Intelligence {low, high}Grade {A, B, C}SAT {low_mark, high_mark}Letter {No, Yes}
Bayes networks defines Joint Distribution in term of a graph over a collection of random variable.
P(D, I, G, S, L)= ?
Joint Distribution of BN
General Form:P(D, I, G, S, L) = P(L|D, I, G, S) *P(S|D, I, G) *P(G|D, I) *P(I|D) *P(D)
D
G
L
S
I
Can the formula be simplify?
Conditional IndependenceFor (sets of) random variables X,Y,ZX is conditional independence of Y given Z, Denotes as P (X ⊥ Y | Z), if:• – P(X, Y|Z) = P(X|Z) P(Y|Z)• – P(X|Y,Z) = P(X|Z)• – P(Y|X,Z) = P(Y|Z)
Joint Distribution of BN
General Form:P(D, I, G, S, L) = P(L|D, I, G, S) *P(S|D, I, G) *P(G|D, I) *P(I|D) *P(D)
D
G
L
S
I
In BN, it can be simplify as:P(D, I, G, S, L) = P(D) * P(I) * P(G|D, I) * P(S|I) * P(L|G)
Parameters: 2*2*3*2*2-1=47
Parameters: 1+1+8+2+3 = 15
Bayesian Network Definition(2)A Bayesian network is a directed acyclic graph(DAG) and a set of conditional probability distribution(CPD).
P(D, I, G, S, L) = P(D) * P(I) * P(G|D, I) * P(S|I) * P(L|G)
Usages of BN: Reasoning
3 kinds of reasoning:• Causal Reasoning• Evidential Reasoning• Intercausal Reasoning
D
G
L
S
I
Causal Reasoning
Evidential Reasoning
Intercausal Reasoning
How to reasoning?
• For certain cases, tractable - Full observed set of variable - just one variable unobserved• In general, intractable…(NP-complete)
How to deal with the problem?An intuitive solution: D-separation
Conditional Independence: RevisitedFor (sets of) random variables X,Y,ZX is conditional independence of Y given Z, Denotes as P (X ⊥ Y | Z), if:• – P(X, Y|Z) = P(X|Z) P(Y|Z)• – P(X|Y,Z) = P(X|Z)• – P(Y|X,Z) = P(X|Z)
D
G
L
S
I
Given an observation of G, is L is conditional independence of D?Given an observation of I, is G conditional independence of S ? Given an observation of G, is D conditional independence of I ?
A method may simplify the calculation when reasoning : to find out more variables which satisfied with conditional independence.
Three Easy Network about Conditional Independence
Tail to Tail Head to HeadHead to Tail
Head to Tail
(D L ) ? ⊥
(D L| G) ?⊥
D
G
L
S
INo
Yes
Tail to Tail
(G S) ?⊥
(G S| I) ?⊥
D
G
L
S
INo
Yes
Head to Head
(D I)?⊥
(D I| G) ?⊥
D
G
L
S
IYes
No
X and Y are conditionally independent given Z, if and only if X and Y are D-separated by Z
Suppose we have three sets of random variables: X, Y and Z
X and Y are D-separated by Z (and therefore conditionally independence, given Z) iff every path from any variable in X to any variable in Y is blocked
A path from variable A to variable B is blocked if it includes a node such that either1.arrows on the path meet either head-to-tail or tail-to-tail at the node and this node is in Z2.the arrows meet head-to-head at the node, and neither the node, nor any of its descendants, is in Z
Summary
• Bayesian Network = Directed Acyclic Graph + Conditional Probability Distribution
• Joint Distribution:• Three type of Reasoning BN: causal,
evidential, intercausal• Conditional Independence & D-separation
Reference
• Machine Learning, CMU• Probabilistic Graphical Models, Stanford, on
Coursera.• SamIam: a comprehensive tool, UCLA
Thanks