a new bigram-plsa language model for speech recognition
DESCRIPTION
A New Bigram-PLSA Language Model for Speech Recognition. Mohammad Bahrani and Hossein Sameti. Department of Computer Engineering, Sharif University of Technology. EURASIP 2010. 報告者:郝柏翰. Outline. Introduction Review of the PLSA Model Combining Bigram and PLSA Models Experiments - PowerPoint PPT PresentationTRANSCRIPT
A New Bigram-PLSA Language Model for Speech Recognition
Mohammad Bahrani and Hossein Sameti
報告者:郝柏翰2013/03/14
EURASIP 2010
Department of Computer Engineering, Sharif University of Technology
2
Outline
• Introduction
• Review of the PLSA Model
• Combining Bigram and PLSA Models
• Experiments
• Conclusion
3
Review of the PLSA Model
𝑃 (𝑤𝑖|𝑑 𝑗 )=∑𝑘
𝑃 (𝑤𝑖∨𝑧𝑘 )𝑃 (𝑧𝑘∨𝑑 𝑗)
• Bag-of-words
• Conditional independent
4
Combining Bigram and PLSA Models
1. Nie et al.’s Bigram-PLSA Model
2. Proposed Bigram-PLSA Model
we relax the assumption of independence between the latent topics and the context words and achieve a general form of the aspect model that considers the word history in the word document modeling.
l lijkil
iki
jik
zwwPdwzP
wdPwPwwdP
),|(),|(
)|()(),,(
5
Parameter Estimation Using the EM Algorithm
𝑃 (𝑧𝑙|𝑑𝑘 ,𝑤 𝑖 ,𝑤 𝑗 )=𝑃 ( 𝑧𝑙 ,𝑑𝑘 ,𝑤 𝑖 ,𝑤 𝑗)
∑𝑙′𝑃 (𝑧 𝑙′ ,𝑑𝑘 ,𝑤𝑖 ,𝑤 𝑗)
• E-step
),|()|( kilik dwzPwdP
6
Parameter Estimation Using the EM Algorithm
Let be the set of model parameters
apply Bayes’ rule
• M-step
7
Parameter Estimation Using the EM Algorithm
• Using Jensen’s inequality
8
Jensen’s inequality
)(1
)()('
k
ii
xP
xPxP
9
Parameter Estimation Using the EM Algorithm
• appropriate Lagrange multipliers
10
Comparison with Nie et al.’s Bigram-PLSA Model.
• The difference between our model and Nie et al.’s model is in the definition of the topic probability.
• we relax the assumption of independence between the latent topics and the context words and achieve a general form of the aspect model that considers the word history in the word-document modeling.
• The number of free parameters in our proposed model is
in Nie et al.’s model is
11
Experiments
12
Experiments