sridhar raghavan
Post on 05-Jan-2016
179 Views
Preview:
DESCRIPTION
TRANSCRIPT
Sridhar Raghavan
Confidence Measure using Word Graphs
3/6
3/6 3/6
2/6
4/6
2/6
2/6
2/6
1/6
4/6
1/6
1/6
4/6
1/6
1/6
4/6
5/6Sil
Sil
This
is
atest
sentence
Sil
this
is
the
is
a
the
guest
a
quest
sentence
sense 1/6 1/6
Sil
Page 2 of 21Confidence measure using word posteriors
Abstract
Confidence measure using word posterior:
• There is a strong need for determining the confidence of a word hypothesis in a LVCSR system because conventional viterbi decoding just generates the overall one best sequence, but the performance of a speech recognition system is based on Word error rate and not sentence error rate.
• Word posterior probability in a hypothesis is a good estimate of the confidence.
• The word posteriors can be computed from a word graph where the links correspond to the words.
• A forward-backward algorithm is used to compute the link posteriors.
Page 3 of 21Confidence measure using word posteriors
Foundation
The equation for computing the posterior of the word is as follows [Wessel.F]:
w.succeeding sequences hypothesis wordall Denotes
w.preceeding sequences hypothesis wordall Denotes
T. to1 timefrom vector Acoustic
interest. of word theof timeend andStart ,
WordSingle
)|,,()|,,(
T1
11
e
a
ea
w w
Tea
Tea
w
w
x
tt
w
xwwwpxttwpa e
The idea here is to sum up the posterior probabilities of all those word hypothesis sequences that contain the word ‘w’ with same start and end times.
Page 4 of 21Confidence measure using word posteriors
Foundation: continued…
We cannot compute the above posterior directly, so we decompose it into likelihood and priors using Baye’s rule.
w w weaea
TT
ea
eaT
T
w weaea
T
a e
a e
wwwpwwwxpxp
wwwp
wwwxp
xp
wwwpwwwxp
),,().,,|()(
and
yprobabilit model Language ),,(
yprobabilit model Acoustic ),,|(
)(
),,().,,|(
11
1
1
1
The value in the numerator can be computed using the well known forward backward algorithm. The denominator term is simply the sum of the numerator for all words ‘w’ occuring in the same time instant ta to te.
Page 5 of 21Confidence measure using word posteriors
What is exactly a word posterior from a word graph?
A word posterior is a probability that is computed by considering a word’s acoustic score, language model score and its presence is a particular path through the word graph.
An example of a word graph is given below, note that the nodes are the start-stop times and the links are the words. The goal is to determine the link posterior probabilities. Every link holds an acoustic score and a language model probability.
quest
3/6
3/6 3/6
2/6
4/6
2/6
2/6
2/6
1/6
4/6
1/6
1/6
4/6
1/6
1/6
4/6
5/6Sil
Sil
This
is
atest
sentence
Sil
this
is
the
is
a
the
guest sentence
sense 1/6 1/6
Sil
Page 6 of 21Confidence measure using word posteriors
Example
Let us consider an example as shown below:
3/6
3/6 3/6
2/6
4/6
2/6
2/6
2/6
1/6
4/6
1/6
1/6
4/6
1/6
1/6
4/6
5/6Sil
Sil
This
is
atest
sentence
Sil
this
is
the
is
a
the
guest
a
quest
sentence
sense 1/6 1/6
Sil
The values on the links are the likelihoods.
Page 7 of 21Confidence measure using word posteriors
Forward-backward algorithm
Using forward-backward algorithm for determining the link probability.
The equations used to compute the alphas and betas for an HMM are as follows:
Computing alphas:
Step 1: Initialization: In a conventional HMM forward-backward algorithm we would perform the following –
i statein are given we X
data observed theof prob.Emission )(
i state of prob. Initial
1 )()(
1
1
11
Xb
NiXbi
i
i
ii
We need to use a slightly modified version of the above equation for processing a word graph. The emission probability will be the acoustic score and the initial probability is taken as 1 since we always begin with a silence.
Page 8 of 21Confidence measure using word posteriors
Forward-backward algorithm continue…
The α for the first node in the word graph is computed as follows:
1
)sil'' ofy probabilit * sil'' with starting ofy probabilit (Initial 1*11
Step 2: Induction
tj
11
Xn observatio theofy probabilitemmision )(b
yprobabilittion transi
1 ;2 )()()(
t
ij
tj
N
iijtt
X
a
NjTtXbaij
This step is the main reason we use forward-backward algorithm for computing such probabilities. The alpha values computed in the previous step is used to compute the alphas for the succeeding nodes.
Note: Unlike in HMMs where we move from left to right at fixed intervals of time, over here we move from one start time of a word to the next closest word’s start time.
Page 9 of 21Confidence measure using word posteriors
Forward-backward algorithm continue…
Let us see the computation of the alphas from node 2, the alpha for node 1 was computed in the previous step during initialization.
Node 2:
0.5
1*)6/3(*12
0.5025
)01.0*)6/3(*5.0()1*)6/3(*1(3
Node 3:
Node 4:
03-1.675E
)1*)6/2(*5025.0(4
The alpha calculation continues in this manner for all the remaining nodes
The forward backward calculation on word-graphs is similar to the calculations used on HMMs, but in word graphs the transition matrix is populated by the language model probabilities and the emission probability corresponds to the acoustic score.
1 3
4
2
3/6
3/6 3/6
2/6
4/6
Sil
Sil
this
is
α =1
α =0.5
α =0.5025
α=1.675E-03
Page 10 of 21Confidence measure using word posteriors
Forward-backward algorithm continue…
Once we compute the alphas using the forward algorithm we begin the beta computation using the backward algorithm.
The backward algorithm is similar to the forward algorithm, but we start from the last node and proceed from right to left.
Step 1 : Initialization
systems. ASRour ofboth in used valueinitial same
theis This 1. is node final at the of valueinitial the
hence and '1'usually isinstant final at the N The
N1 /1)(
iNiT
Step 2: Induction
node.current
thepreceedingjust nodes theof valuebeta The )(
score acoustic The )(b
score model Language a
Ni1 1....1;-T t)()()(
1
1j
ij
1 11
j
X
jXbai
t
t
N
j ttjijt
Page 11 of 21Confidence measure using word posteriors
Forward-backward algorithm continue…
Let us see the computation of the beta values from node 14 and backwards.
Node 14:
03-1.66E
1*01.0*)6/1(14
03-8.33E
.1*01.0*)6/5(13
Node 13:
Node 12:
05-5.555E
0333.8*01.0*)6/4(12
E
11
12
13 15
1/6
4/6
5/6
sentence
Sil
14
sentence
sense 1/6 1/6
Sil
β=1.66E-3
β=5.55E-3
β=0.833
β=0.1667
β=1
Page 12 of 21Confidence measure using word posteriors
Forward-backward algorithm continue…
Node 11:
05-1.666E
)0333.8*01.0*)6/1(( )03667.1*01.0*)6/1((12
EE
In a similar manner we obtain the beta values for all the nodes till node 1.
We can compute the probabilities on the links (between two nodes) as follows:
Let us call this link probability as Γ.
Therefore Γ(t-1,t) is computed as the product of α(t-1)*ß(t)*aij. These values give the un-normalized posterior probabilities of the word on the link considering all possible paths through the link.
Page 13 of 21Confidence measure using word posteriors
Word graph showing the computed alphas and betas
1 3
4
5
6
7 10
11
12
139
15
2
3/6
3/6 3/6
2/6
4/6
2/6
2/6
2/6
1/6
4/6
1/6
1/6
4/6
1/6
1/6
4/6
5/6Sil
Sil
This
is
atest
sentence
Sil
this
is
the
is
a
the
guest
quest14
sentence
sense 1/6 1/6
Silα =1β=2.8843E-14
8
α =0.5β=2.87E-14
α =0.5025β=5.740E-12
α=1.117E-5β=2.512E-7
α=1.675E-03β=1.536E-11
α=3.35E-3β=8.527E-10
α=1.675E-5β=4.61E-9
α=2.79E-8β=2.766E-6
α=1.861E-8β=2.766E-6
α=7.446E-8β=3.7E-5
α=7.751E-11β=1.66E-3
α=4.964E-10β=5.55E-3
α=3.438E-12β=0.833
α=1.2923E-13β=0.1667
α=2.88E-14β=1
Assumption here is that the probability of occurrence of any word is 0.01. i.e. we have 100 words in a loop grammar
This is the word graph with every node with its corresponding alpha and beta value.
Page 14 of 21Confidence measure using word posteriors
Link probabilities calculated from alphas and betas
Γ=4.649E-13
1 3
4
5
15
2
3/6
3/6 3/6
2/6
4/6
2/6
2/6
2/6
1/6
4/6
1/6
1/6
4/6
1/6
1/6
4/6
5/6Sil
Sil
This
is
atest
sentence
Sil
this
is
the
is
a
the
guest
quest14
sentence
sense 1/6 1/6
SilΓ=5.74E-12
Γ=2.87E-14
Γ=4.288E-12
Γ=7.71E-14
Γ=7.72E-14
Γ=1.549E-13
Γ=8.421E-12
Γ=4.649E-13
Γ=3.08E-13
Γ=4.136E1-12
Γ=3.08E-13
Γ=4.136E-12
Γ=4.136E-12
Γ=6.45E-13
Γ=1.292E-13Γ=1.292E-15
Γ=3.438E-14
The following word graph shows the links with their corresponding link posterior probabilities (not yet normalized).
6
7
8
9
10
11
12
13
Γ=2.87E-14
By choosing the links with the maximum posterior probability we can be certain that we have included most probable words in the final sequence.
Page 15 of 21Confidence measure using word posteriors
Using it on a real application
Using the algorithm on real application:
* Need to perform word spotting without using a language model i.e. we
can only use a loop grammar.
* In order to spot the word of interest we will construct a loop grammar
with just this one word.
* Now the final one best hypothesis will consist of a sequence of the
same word repeated N times. So, the challenge here is to determine
which of these N words actually corresponds to the word of interest.
* This is achieved by computing the link posterior probability and
selecting the one with the maximum value.
Page 16 of 21Confidence measure using word posteriors
1-best output from the word spotter
The recognizer puts out the following output :-
0000 0023 !SENT_START -1433.434204
0023 0081 BIG -4029.476440
0081 0176 BIG -6402.677246
0176 0237 BIG -4080.437500
0237 0266 !SENT_END -1861.777344
We have to determine which of the three instances of the word actually exists.
Page 17 of 21Confidence measure using word posteriors
0 1
2
3
4
5 6 7
sent_start
sent_end
-1433
-1095
-1888
-2875
-4029
-912
-1070
-1232
-6402
-1861
8-4056
Lattice from one of the utterances
For this example we have to spot the word “BIG” in an utterance that consists of three words (“BIG TIED GOD”). All the links in the output lattice contains the word “BIG”. The values on the links are the acoustic likelihoods in log domain. Hence a forward backward computation just involves addition of these numbers in a systematic manner.
Page 18 of 21Confidence measure using word posteriors
Alphas and betas for the lattice
0 1
2
3
4
5 6 7
sent_start
sent_end
-1433
-1095
-1888
-2875
-4029
-912
-1070
-1232
-6402
-1861
8-4056
α =0β=-67344
α =-1433β=-65911
α =-2528β=-15533
α =-6761β=-14621
α =-12139β=-13551
α =-18833β=-12319
α =-25235β=-5917
α =-29291β=-1861
α =-31152β=0
The initial probability at both the nodes is ‘1’. So, its logarithmic value is 0. The language model probability of the word is also ‘1’ since it is the only word in the loop grammar.
Page 19 of 21Confidence measure using word posteriors
Link posterior calculation
0 1
2
3
4
5 6 7
sent_start
sent_end
8
Γ=-67344
Γ=-18061Γ=-18061
Γ=-17942
Γ=-17859
Γ=-17781
Γ=-21382
Γ=-25690
Γ=-31152 Γ=-31152
Γ=-31152
It is observed that we can obtain a greater discrimination in confidence levels if we also multiply the final probability with the likelihood of the link other than the corresponding alphas and betas. In this example we add the likelihood since it is in log domain.
Page 20 of 21Confidence measure using word posteriors
Inference from the link posteriors
Link 1 to 5 corresponds to the first word time instance while 5 to 6 and 6 to 7 correspond to the second and third word instances respectively. It is very clear from the link posterior values that the first instance of the word “BIG” has a much higher probability than the other two.
Note: The part that is missing in this presentation is the normalization of these probabilities, this is needed to make comparison between various link posteriors.
Page 21 of 21Confidence measure using word posteriors
References:
• F. Wessel, R. Schlüter, K. Macherey, H. Ney. "Confidence Measures for Large Vocabulary Continuous Speech Recognition". IEEE Trans. on Speech and Audio Processing. Vol. 9, No. 3, pp. 288-298, March 2001
• Wessel, Macherey, and Schauter, "Using Word Probabilities as Confidence Measures, ICASSP'97
• G. Evermann and P.C. Woodland, “Large Vocabulary Decoding and Confidence Estimation using Word Posterior Probabilities in Proc. ICASSP 2000, pp. 2366-2369, Istanbul.
• X. Huang, A. Acero, and H.W. Hon, Spoken Language Processing - A Guide to Theory, Algorithm, and System Development, Prentice Hall, ISBN: 0-13-022616-5, 2001
• J. Deller, et. al., Discrete-Time Processing of Speech Signals, MacMillan Publishing Co., ISBN: 0-7803-5386-2, 2000
top related