Embed Size (px)
DESCRIPTION
A New Bigram-PLSA Language Model for Speech Recognition. Mohammad Bahrani and Hossein Sameti. Department of Computer Engineering, Sharif University of Technology. EURASIP 2010. 報告者：郝柏翰. Outline. Introduction Review of the PLSA Model Combining Bigram and PLSA Models Experiments - PowerPoint PPT Presentation
Citation preview
A New Bigram-PLSA Language Model for Speech Recognition
Mohammad Bahrani and Hossein Sameti
報告者：郝柏翰2013/03/14
EURASIP 2010
Department of Computer Engineering, Sharif University of Technology
2
Outline
• Introduction
• Review of the PLSA Model
• Combining Bigram and PLSA Models
• Experiments
• Conclusion
3
Review of the PLSA Model
𝑃 (𝑤𝑖|𝑑 𝑗 )=∑𝑘
𝑃 (𝑤𝑖∨𝑧𝑘 )𝑃 (𝑧𝑘∨𝑑 𝑗)
• Bag-of-words
• Conditional independent
4
Combining Bigram and PLSA Models
1. Nie et al.’s Bigram-PLSA Model
2. Proposed Bigram-PLSA Model
we relax the assumption of independence between the latent topics and the context words and achieve a general form of the aspect model that considers the word history in the word document modeling.
l lijkil
iki
jik
zwwPdwzP
wdPwPwwdP
),|(),|(
)|()(),,(
5
Parameter Estimation Using the EM Algorithm
𝑃 (𝑧𝑙|𝑑𝑘 ,𝑤 𝑖 ,𝑤 𝑗 )=𝑃 ( 𝑧𝑙 ,𝑑𝑘 ,𝑤 𝑖 ,𝑤 𝑗)
∑𝑙′𝑃 (𝑧 𝑙′ ,𝑑𝑘 ,𝑤𝑖 ,𝑤 𝑗)
• E-step
),|()|( kilik dwzPwdP
6
Parameter Estimation Using the EM Algorithm
Let be the set of model parameters
apply Bayes’ rule
• M-step
7
Parameter Estimation Using the EM Algorithm
• Using Jensen’s inequality
8
Jensen’s inequality
)(1
)()('
k
ii
xP
xPxP
9
Parameter Estimation Using the EM Algorithm
• appropriate Lagrange multipliers
10
Comparison with Nie et al.’s Bigram-PLSA Model.
• The difference between our model and Nie et al.’s model is in the definition of the topic probability.
• we relax the assumption of independence between the latent topics and the context words and achieve a general form of the aspect model that considers the word history in the word-document modeling.
• The number of free parameters in our proposed model is
in Nie et al.’s model is
11
Experiments
12
Experiments
