Investigation on Mandarin Broadcast News Speech Recognition
Outline
The task Text training data and language modeling Acoustic training data and acoustic modeling Decoding structure Experimental results Recent progress and future direction
The Task
Mandarin broadcast news (BN) transcription Mainland Mandarin speech TV/radio programs in China, USA
CCTV 中央电视台 NTDTV 新唐人电视台 PHOENIX TV 凤凰卫视 VOA 美国之音 RFA 自由亚洲电台 CNR 中国广播网
Text Training Data
LM1: 1997 Mandarin BN Hub4 transcriptions Chinese TDT2,3,4 Multiple-translation Chinese (MTC) corpus, part 1, 2, 3
LM2: Gigaword XIN 2001-2004 (China) LM3: Gigaword ZBN 2001-2004 (Singapore) LM4: Gigaword CNA 2001-2004 (Taiwan) All together 420M words. 4 LMs interpolated
Chinese Word Segmentation
BBN 64k-word lexicon, derived from LDC Longest-first match with the 64k-lexicon Choose most frequent 49k words as new
lexicon Train n-gram Use unigram part to re-do word segmentation
based on the ML path
Chinese Word Segmentation
Longest-first 民进党 /和亲 /民党… The Green Party made peace with the Min Party
via marriage… Maximum-likelihood
民进党 / 和 /亲民党… The Green Party and the Qin-Min Party...
Perplexity
49k-word lexicon
Word perplexity
2-gram 495
4-gram 288
Acoustic Training Data
Corpus Size
1997 Hub4 BN 28.5 hrs
*TDT4-CCTV 25 hrs
*TDT4-VOA 43.5 hrs
Total 97 hours
*auto selection via a flexible alignment with closed caption
Acoustic Feature Representation
39-dim MFCC cepstra + + 3-dim pitch + + Auto speaker clustering VTLN per auto speaker Speaker-based CMN+CVN for training
Acoustic Models
2500 senones (clustered states) x 32 Gaussians ML training vs. MPE training with phone lattices Gender indepdent. nonCW vs. CW triphones Speaker-adaptive training (SAT):
N(x; a+b, AAt) = |A|-1 N(A-1(x-b); , )
Linear transformation A-1x + (-A-1b) applied to the feature domain.
2-Pass Search Architecture
Search 1
SAT
MLLR
Search 2
nonCW,nonSAT, ML model
Small bigram
hypothesis
CW,SAT,MPE model
Final word sequence
Big 4-gram
Adding Pitch: SA Results (CER)
Smoothing Dev04 Eval04
No pitch 14.5% 24.1%
IBM-style
(mean based)
14.0% 22.2%
SPLINE
(cubic smoothing)
12.7% 21.4%
2-pass Search Results (CER)
Acoustic model Dev04 Eval04
nonCW, nonSAT, ML
7.4% --
nonCW, nonSAT, MPE
6.9% --
nonCW, SAT, ML
6.8% --
CW, SAT, ML 6.4% --
CW,SAT,MPE 6.0% 16.0%
More Recent Progress
Add more acoustic (440 hrs) and text training data (840M words).
Increased and improved lexicon (60k words). fMPE training. Add ICSI feature as a second system. 5-gram LM. Between MFCC system and ICSI system,
Cross adaptation Rover
3.7% on dev04, 12.1% on eval04. Submitted to ICASSP 2007
Challenges
Channel compensation Conversational speech Overlapped speech Speech with music background Commercial Language ID (in addition to English) Is CER the best measurement for MT?