p1-swietojanski

REPRINT PERMITTED BY IEEE SP SOCIETY, FOR INCLUSION IN THE PROCEEDINGS AND PRESENTATION AT THE IEEE GLOBALSIP 2014 CONFERENCE. 1

Convolutional Neural Networksfor Distant Speech Recognition

Pawel Swietojanski, Student Member, IEEE, Arnab Ghoshal, Member, IEEE, and Steve Renals, Fellow, IEEE

AbstractWe investigate convolutional neural networks(CNNs) for large vocabulary distant speech recognition, trainedusing speech recorded from a single distant microphone (SDM)and multiple distant microphones (MDM). In the MDM case weexplore a beamformed signal input representation compared withthe direct use of multiple acoustic channels as a parallel input tothe CNN. We have explored different weight sharing approaches,and propose a channel-wise convolution with two-way pooling.Our experiments, using the AMI meeting corpus, found thatCNNs improve the word error rate (WER) by 6.5% relativecompared to conventional deep neural network (DNN) modelsand 15.7% over a discriminatively trained Gaussian mixturemodel (GMM) baseline. For cross-channel CNN training, theWER improves by 3.5% relative over the comparable DNNstructure. Compared with the best beamformed GMM system,cross-channel convolution reduces the WER by 9.7% relative,and matches the accuracy of a beamformed DNN.

Index Termsdistant speech recognition, deep neural net-works, convolutional neural networks, meetings, AMI corpus

I. INTRODUCTION

D ISTANT speech recognition (DSR) [1] is a challengingtask owing to reverberation and competing acousticsources. DSR systems may be configured to record audio datausing a single distant microphone (SDM), or multiple distantmicrophones (MDM). Current DSR systems for conversationalspeech are considerably less accurate than their close-talkingequivalents, and usually require complex multi-pass decodingschemes and sophisticated front-end processing techniques[2][4]. SDM systems usually result in significantly higherword error rates (WERs) compared to MDM systems.

Deep neural network (DNN) acoustic models [5] have ex-tended the state-of-the-art in acoustic modelling for automaticspeech recognition (ASR), using both hybrid configurations[6][11] in which the neural network is used to estimatehidden Markov model (HMM) output probabilities and poste-riorgram configurations [12][15] in which the neural networkprovides discriminative features for an HMM. It has alsobeen demonstrated that hybrid neural network systems cansignificantly increase the accuracy of conversational DSR [16].An advantage of the hybrid approach is the ability to usefrequency domain feature vectors, which provide a small butconsistent improvement over cepstral domain features [17].

P Swietojanski and S Renals are with the Centre for Speech TechnologyResearch, University of Edinburgh, email: {p.swietojanski,s.renals}@ed.ac.uk

A Ghoshal was with the University of Edinburgh when this work was done.He is now with Apple Inc., email: [email protected]

This research was supported by EPSRC Programme Grant grant, no.EP/I031022/1 (Natural Speech Technology), and the European Union underFP7 project grant agreement 287872 (inEvent).

Convolutional neural networks (CNNs) [18], which restrictthe network architecture using local connectivity and weightsharing, have been applied successfully to document recog-nition [19]. When the weight sharing is confined to the timedimension, the network is called a time-delay neural networkand has been applied to speech recognition [20][22]. CNNshave been used for speech detection [23], directly modellingthe raw speech signal [24], and for acoustic modelling inspeech recognition in which convolution and pooling areperformed in the frequency domain [25][27]. Compared toDNN-based acoustic models, CNNs have been found to reducethe WER on broadcast news transcription by an average of10% relative [26], [27].

Here we investigate weight sharing and pooling techniquesfor CNNs in the context of multi-channel DSR, in particularcross-channel pooling across hidden representations that corre-spond to multiple microphones. We evaluate these approachesthrough experiments on the AMI meeting corpus [28].

II. CNN ACOUSTIC MODELS

Context-dependent DNNHMM systems use DNNs to clas-sify the input acoustics into classes corresponding to the HMMtied states. After training, the output of the DNN provides anestimate of the posterior probability P (s | ot) of each HMMstate s given the acoustic observations ot at time t, whichmay be used to obtain the (scaled) log-likelihood of state sgiven observation ot: log p(ot | s) logP (s | ot) logP (s)[6], [8], [29], where P (s) is the prior probability of state scalculated from the training data.

A. Convolutional and pooling layers

The structure of feed-forward neural networks may beenriched through the use of convolutional layers [19] whichallows local feature receptors to be learned and reused acrossthe whole input space. A max-pooling operator [30] canbe applied to downsample the convolutional output bands,thus reducing variability in the hidden activations. Consider aneural network in which the acoustic feature vector V consistsof filter-bank outputs within an acoustic context window ofsize Z. V = [v1,v2, . . . ,vb, . . . ,vB ] RBZ is divided intoB frequency bands with the b-th band vb RZ comprisingall the Z relevant coefficients (statics, , 2, . . .) across allframes of the context window. The k-th hidden convolutionband hk = [h1,k, . . . , hj,k, . . . , hJ,k] RJ is then composedof a linear convolution of J weight vectors (filters) with Fconsecutive input bands uk = [v(k1)L+1, . . . ,v(k1)L+F ] RF Z , where L {1, . . . , F} is the filter shift. Fig 1 gives

GlobalSIP 2014: IEEE SPL/TASLP Papers Presentations978-1-4799-7088-9/14/$31.00 2014 IEEE 1


cross-band

maxpooling

...

...

multi-channel

input bands

shared

weights

convolutional

bands

...cross-channelmaxpooling

cross-band

maxpooling

...

...

Fig. 1. Frequency domain max-pooling multi-channel CNN layer (left), anda similar layer with cross-channel max-pooling (right).

an example of such a convolution with a filter size and shiftof F = 3 and L = 1 respectively. This may be extendedto S acoustic channels V1...VS (each corresponding to amicrophone), in which the hidden activation hj,k can becomputed by summing over the channels:

hj,k =

(bj,k +

Ss=1

wsj usk), (1)

where () is a sigmoid nonlinearity, denotes linear validconvolution1, wsj RF Z is a weight vector of the j-th filteracting on the local input usk of the s-th input channel, andbj,k is an additive bias for the j-th filter and k-th convolutionalband. Since the channels contain similar information (acousticfeatures shifted in time) we conjecture that the filter weightsmay be shared across different channels. Nevertheless, the for-mulation and implementation allow for different filter weightsin each channel. Similarly, it is possible for each convolutionalband to have a separate learnable bias parameter instead of thebiases only being shared across bands [25], [26].

The complete set of convolutional layer activations h =[h1, . . . ,hK ] RKJ is composed of K = (B F )/L + 1convolutional bands obtained by applying the (shared) set ofJ filters across the whole (multi-channel) input space V (asdepicted in Fig 1). In this work the weights are tied acrossthe input space (i.e. each uk is convolved with the samefilters); alternatively the weights may be partially shared, tyingonly those weights spanning neighbouring frequency bands[25]. Although limited weight sharing was reported to bringimprovements for phone classification [25] and small LVSRtasks [32], a recent study on larger tasks [27] suggests that fullweight sharing with a sufficient number of filters can workequally well, while being easier to implement.

A convolutional layer is usually followed by a pooling layerwhich downsamples the activations h. The max-pooling oper-ator [33] passes forward the maximum value within a groupof R activations. The m-th max-pooled band is composed ofJ related filters pm = [p1,m, . . . , pj,m, . . . , pJ,m] RJ :

pj,m =R

maxr=1

(hj,(m1)N+r

), (2)

1The convolution of two vectors of size X and Y may result either in thevector of size X + Y 1 for a full convolution with zero-padding of non-overlapping regions, or the vector of size X Y +1 for a valid convolutionwhere only the points which overlap completely are considered [31].

where N {1, . . . , R} is a pooling shift allowing for overlapbetween pooling regions when N < R (in Fig 1, R = N = 3).The pooling layer decreases the output dimensionality from Kconvolutional bands to M = (K R)/N + 1 pooled bandsand the resulting layer is p = [p1, ...,pM ] RM J .

B. Channel-wise convolution

Multi-channel convolution (1) builds feature maps similarlyto the LeNet-5 model [19] where each convolutional band iscomposed of filter activations spanning all input channels. Wealso constructed feature maps using max-pooling across chan-nels, in which the activations hsj,k are generated in channel-wise fashion and then max-pooled (4) to form a single cross-channel convolutional band ck = [c1,k, . . . , cj,k, . . . , cJ,k] RJ (Fig 1 (right)):

hsj,k = (bj,k + wj usk) (3)cj,k =

Smaxs=1

(hsj,k

). (4)

Note that here the filter weights wj need to be tied across thechannels such that the cross-channel max-pooling (4) operateson activations for the same feature receptor. The resultingcross-channel activations c = [c1, . . . , cK ] RKJ can befurther max pooled along frequency using (2). Channel-wiseconvolution may also be viewed as a special case of 2-dimensional convolution, where the effective pooling regionis determined in frequency but varies in time depending onthe actual time delays between the microphones.

C. Fully-connected layers

The complete acoustic model is composed of one or moreCNN layers, followed by a number of fully-connected layers,with a softmax output layer. With a single CNN layer, thecomputation performed by the network is as follows:

hl = (Wlhl1 + bl), for 2 l < L (5)aL = WLhL1 + bL,

P (s|ot) = exp{aL(s)}

s exp{aL(s)}, (6)

where hl is the input to the (l+1)-th layer, with h1 = p; Wl

is the matrix of connection weights and bl is the additive biasvector for the l-th layer; () is a sigmoid nonlinearity thatoperates element-wise on its input vector; aL is the activationat the output layer.

III. EXPERIMENTS

We have performed experiments using the AMI meetingcorpus [28] (http://corpus.amiproject.org/) using an identicaltraining and test configuration to [16]. The AMI corpus com-prises around 100 hours of meetings recorded in instrumentedmeeting rooms at three sites in the UK, the Netherlands, andSwitzerland. Each meeting usually has four participants andthe language is English, albeit with a large proportion of non-native speakers. Multiple microphones were used, includingindividual headset microphones (IHM), lapel microphones,and one or more microphone arrays. Every recording used a

GlobalSIP 2014: IEEE SPL/TASLP Papers Presentations2


primary 8-microphone uniform circular array (10 cm radius),as well as a secondary array whose geometry varied betweensites. In this work we use the primary array for our MDMexperiments, and the first microphone of the primary array forour SDM experiments. Our systems are trained and evaluatedusing the split recommended in the corpus release: an 80hour training set, and development and test sets each of9 hours. We use the segmentation provided with the AMIcorpus annotations (v1.6). For training purposes we considerall segments (including those with overlapped speech), andthe WERs of the speech recognition outputs are scored by theasclite tool [34] following the NIST RT recommendationsfor scoring simultaneous speech (http://nist.gov/speech/tests/rt/2009). WERs for non-overlapped segments only may alsobe produced by asclite, using the -overlap-limit 1option. Here, we report results using the development set only:both development and test sets are relatively large, and wepreviously found that the best parameters selected for thedevelopment set were also optimal for the evaluation set [16].

All CNN/DNN models were trained on 40-dimensional logMel filterbank (FBANK) features appended with the first andthe second time derivatives [17] which were presented inZ = 11 frames long symmetric context windows. Our distantmicrophone systems within this work remain unadapted toboth speakers and sessions. Ascribing speakers to segmentswithout diarisation is unrealistic while a small mismatchbetween training and evaluation acoustic environments makesfeature-space maximum likelihood linear regression only mod-erately effective (less than 1% absolute reduction in WER)for session adaptation. Our experiments were performed usingthe Kaldi speech recognition toolkit [35], and the pylearn2machine learning library [36].

Our experiments used a 50,000 word pronunciation dictio-nary [4]. An in-domain trigram language model (LM) was es-timated using the AMI training transcripts (801k words). Thiswas interpolated with two further trigram LMs one estimatedfrom the Switchboard training transcripts (3M words), and theother from the Fisher English transcripts (22M words) [37].The LMs are estimated using modified Kneser-Ney smoothing[38]. The LM interpolation weights were as follows: AMItranscripts (0.73); Switchboard (0.05); Fisher (0.22). The fi-nal interpolated LM had 1.6M trigrams and 1.5M bigrams,resulting in a perplexity of 78 on the development set.

IV. RESULTS

We have tested the CNNs with both SDM and MDM inputs.In each case we compare the CNN to two baseline systems:(1) a Gaussian mixture model (GMM) system, discriminativelytrained using boosted maximum mutual information (BMMI)[39], with mel-frequency cepstral coefficient (MFCC) featurespost-processed with linear discriminative analysis (LDA) anddecorrelated using a semi-tied covariance (STC) transform[40]; and (2) a deep neural network (DNN) with 6 hiddenlayers, with 2048 units in each layer [16] trained usingthe same FBANK features as used for the CNNs. We usedrestricted Boltzmann machine (RBM) pretraining [41] for thebaseline DNN systems, but not for the CNN systems. The

TABLE IWORD ERROR RATES (%) ON AMI DEVELOPMENT SET SDM.

System with overlap no overlapBMMI GMM-HMM (LDA+STC) 63.2 55.8DNN +RBM (FBANK) 53.1 42.4CNN (R = 3) 53.3 42.8CNN (R = 2) 51.3 40.4CNN (R = 1) 52.5 40.9

CNN results are reported for the networks composed witha single CNN layer followed by 5 fully-connected layers.The CNN hyperparameters are as follows: number of filtersJ = 128, filter size F = 9, and filter shift L = 1.

A. Single Distant Microphone

We applied two CNN approaches to the SDM case, inwhich acoustics from a single channel only is used. In the firstapproach the same bias terms were used for each band [26](section II-A), and the results of the single channel CNN canbe found in Table I. The first two rows are the SDM baselines(reported in [16])2. The following three lines are results forthe CNN using max-pool sizes (PS) of R = N = 1, 2, 3. Byusing CNNs we were able to obtain 3.4% relative reduction inWER with respect to the best DNN model and a 19% relativereduction in WER compared with a discriminatively trainedGMM-HMM. Note, the total number of parameters of theCNN models vary here as R = N while J is kept constantacross the experiments. However, the best performing modelhad neither the highest nor the lowest number of parameters,which suggests it is due to the optimal pooling setting.

B. Multiple Distant Microphones

For the MDM case we compared a delay-sum beamformerwith the direct use of multiple microphone channels as inputto the network. For beamforming experiments, we follownoise cancellation using a Wiener filter with a delay-sumbeamforming on 8 uniformly-spaced array channels usingBeamformIt [42]. The results are summarised in Table II.The first block of Table II presents the results for the casein which the models were trained on a beamformed signalfrom 8 microphones. The first two rows show the WER forthe baseline GMM and DNN acoustic models as reported in[16]. The following three rows contain the comparable CNNstructures with different pooling sizes (PS) R = N = 1, 2, 3.The best model (pool size R = 1, equivalent to no max-pooling) scored 46.3% WER which is 6.4% relative WERbetter than the best DNN network and a relative improvementin WER of 16% compared with a discriminatively trainedGMM-HMM system.

The second part of Table II shows WERs for the modelsdirectly utilising multi-channel features. The first row is abaseline DNN variant trained on 4 concatenated channels[16]. Then we present the CNN models with MDM inputconvolution performed as in equation (1) and pooling sizeof 2, which was optimal for the SDM experiments. This

2DNN baseline WERs are lower than [16] due to the intial values chosenfor the hyper-parameters.



TABLE IIWORD ERROR RATES (%) ON AMI DEVELOPMENT SET MDM.

System with overlap no overlapMDM with beamforming (8 microphones)BMMI GMM-HMM 54.8 46.1DNN +RBM 49.5 37.4CNN (R = 3) 46.5 34.2CNN (R = 2) 46.8 34.4CNN (R = 1) 46.3 34.3MDM without beamformerDNN +RBM 4ch concatenated 51.2 40.3CNN (R = 2) 2ch conventional 50.5 39.5CNN (R = 2) 4ch conventional 50.4 38.7CNN (R = 2) 2ch channel-wise 50.0 38.5CNN (R = 2) 4ch channel-wise 49.4 37.5

TABLE IIIWORD ERROR RATES (%) ON AMI DEVELOPMENT SET IHM

System WER(%)BMMI GMM-HMM (SAT) 29.4DNN +RBM 26.6CNN (R = 1) 25.6

scenario decreases WER by 1.6% relative when comparedto a DNN structure with concatenated channels. Applyingchannel-wise convolution with two-way pooling (outlined insection II-B) brings further gains of 3.5% WER relative.Furthermore, channel-wise pooling works better for more inputchannels: conventional convolution on 4 channels achieves50.4% WER, practically the same as the 2 channel network,while channel-wise convolution with 4 channels achieves49.5% WER, compared to 50.0% for the 2-channel case. Theseresults indicate that picking the best information (selectingthe feature receptors with maximum activations) within thechannels is crucial when doing model-based combination ofmultiple microphones.

C. Individual Headset Microphones

We observe similar relative WER improvements betweenDNN and CNN for close talking speech experiments (TableIII) as were observed for the DSR experiments (Tables Iand II). The CNN achieves 3.6% WER reduction relative tothe DNN model. Both DNN and CNN systems outperforma BMMI-GMM system trained in a speaker adaptive (SAT)fashion by 9.4% and 12.9% relative WER respectively. Wedid not see any improvements by increasing pooling size. [26]has previously suggested that pooling may be task dependent.

D. Different weight-sharing techniques

When using multiple distant microphones directly as inputto a CNN, we posit that the same filters should be used acrossthe different channels even when cross-channel pooling isnot used. Each channel contains the same information, albeitshifted in time, hence using the same feature detectors for eachchannel is a prudent constraint to learning. The first two rowsof Table IV show the results when a separate set of filtersare learned for each channel. Sharing the filter weights across

TABLE IVWORD ERROR RATES (%) ON AMI DEVELOPMENT SET.

DIFFERENT WEIGHT SHARING AND POOLING TECHNIQUES.

System with overlap no overlapMDM without beamformerCNN (R = 3) 2ch not tied wsj 51.2 -CNN (R = 2) 2ch not tied wsj 51.3 -SDMCNN (R = 3) bias bj 53.3 42.8CNN (R = 3) bias bj,k 52.5 40.9CNN (R = 2) bias bj,k 51.9 40.5

channels improves the WER by 0.7% absolute (comparingwith the 2 channel CNN, Table II).

The second block of Table IV shows the effect of traininga separate bias parameter for each of the K convolutionalbands for the SDM system of Table I. These results aregenerated for non-overlapping pools of size 3 and 2. If thepooling size is too large, we observe that the WER increases.This increase in WER is mitigated by using a band-specificbias. We hypothesise that, under noisy conditions, the max-pooling operator, which may be interpreted as a local hard-decision heuristic, selects non-optimal band activations, whilethe not-tied bias can actually boost the meaningful frequencyregions (on average). A band-specific bias does not lead tofurther improvements: e.g., when R = 2, the overlappedspeech CNN with tied biases had a WER of 51.3% comparedto 51.9% for the not-tied version.

V. DISCUSSION

We have investigated using CNNs for DSR with singleand multiple microphones. A CNN trained on a single distantmicrophone is found to produce a WER approaching these ofa DNN trained using beamforming across 8 microphones. Inexperiments with multiple microphones, we compared CNNstrained on the output of a delay-sum beamformer with thosetrained directly on the outputs of multiple microphones. Inthe latter configuration, channel-wise convolution followed bya cross-channel max-pooling was found to perform better thanmulti-channel convolution.

A beamformer uses time-delays between microphone pairswhose computation requires knowledge of the microphonearray geometry, while these convolutional approaches need nosuch knowledge. CNNs are able to compensate better for theconfounding factors in distant speech than DNNs. However,the compensation learned by CNNs is complementary tothat provided by a beamformer. In fact, when using CNNswith cross-channel pooling, similar WERs were obtained bychanging the order of the channels at test time from the orderin which they were presented at training time, suggesting thatthe model is able to pick the most informative channel.

Early work on CNNs for ASR focussed on learning shift-invariance in time [20], [43], while more recent work [25],[26] have indicated that shift-invariance in frequency is moreimportant for ASR. The results presented here suggest thatrecognition of distant multichannel speech is a scenario whereshift-invariance in time between channels is also important,thus benefitting from pooling in both time and frequency.



REFERENCES

[1] M Wolfel and J McDonough, Distant Speech Recognition, Wiley, 2009.[2] A Stolcke, Making the most from multiple microphones in meeting

recognition, in Proc IEEE ICASSP, 2011.[3] K Kumatani, J McDonough, and B Raj, Microphone array processing

for distant speech recognition: From close-talking microphones to far-field sensors, IEEE Signal Process. Mag., vol. 29, no. 6, pp. 127140,2012.

[4] T Hain, L Burget, J Dines, PN Garner, F Grezl, AE Hannani, M Hui-jbregts, M Karafiat, M Lincoln, and V Wan, Transcribing meetings withthe AMIDA systems, IEEE Trans. Audio, Speech, Language Process.,vol. 20, no. 2, pp. 486498, 2012.

[5] G Hinton, L Deng, D Yu, GE Dahl, A-R Mohamed, N Jaitly, A Senior,V Vanhoucke, P Nguyen, TN Sainath, and B Kingsbury, Deep neuralnetworks for acoustic modeling in speech recognition: The shared viewsof four research groups, IEEE Signal Process. Mag., vol. 29, no. 6, pp.8297, 2012.

[6] H Bourlard and N Morgan, Connectionist Speech Recognition: A HybridApproach, Kluwer Academic Publishers, 1994.

[7] S Renals, N Morgan, H Bourlard, M Cohen, and H Franco, Connec-tionist probability estimators in HMM speech recognition, IEEE Trans.Speech Audio Process., vol. 2, no. 1, pp. 161174, 1994.

[8] N Morgan and H Bourlard, Neural networks for statistical recognitionof continuous speech, Proceedings of the IEEE, vol. 83, no. 5, pp.742772, 1995.

[9] AJ Robinson, GD Cook, DPW Ellis, E Fosler-Lussier, SJ Renals, andDAG Williams, Connectionist speech recognition of broadcast news,Speech Communication, vol. 37, no. 12, pp. 2745, 2002.

[10] TN Sainath, B Kingsbury, B Ramabhadran, P Fousek, P Novak, andA Mohamed, Making deep belief networks effective for large vocab-ulary continuous speech recognition, in Proc IEEE ASRU, 2011.

[11] GE Dahl, D Yu, L Deng, and A Acero, Context-dependent pre-traineddeep neural networks for large-vocabulary speech recognition, IEEETransactions on Audio, Speech & Language Processing, vol. 20, no. 1,pp. 3042, 2012.

[12] H Hermansky, DPW Ellis, and S Sharma, Tandem connectionist featureextraction for conventional HMM systems, in Proc IEEE ICASSP,2000, pp. 16351638.

[13] Q Zhu, A Stolcke, BY Chen, and N Morgan, Using MLP features inSRIs conversational speech recognition system, in Proc. Eurospeech,2005.

[14] F Grezl, M Karafiat, S Kontar, and J Cernocky, Probabilistic and bottle-neck features for LVCSR of meetings, in Proc IEEE ICASSP, 2007,vol. 4, pp. IV757IV760.

[15] TN Sainath, B Kingsbury, and B Ramabhadran, Auto-encoder bot-tleneck features using deep belief networks, in Proc IEEE ICASSP,2012.

[16] P Swietojanski, A Ghoshal, and S Renals, Hybrid acoustic models fordistant and multichannel large vocabulary speech recognition, in Proc.IEEE ASRU, Dec. 2013.

[17] J Li, D Yu, J-T Huang, and Y Gong, Improving wideband speechrecognition using mixed-bandwidth training data in CD-DNN-HMM,in Proc IEEE SLT, 2012, pp. 131136.

[18] Y LeCun and Y Bengio, Convolutional networks for images, speechand time series, in The Handbook of Brain Theory and NeuralNetworks, pp. 255258. The MIT Press, 1995.

[19] Y LeCun, L Bottou, Y Bengio, and P Haffner, Gradient-based learningapplied to document recognition, Proc. IEEE, vol. 86, no. 11, pp. 22782324, 1998.

[20] A Waibel, T Hanazawa, G Hinton, K Shikano, and KJ Lang, Phonemerecognition using time-delay neural networks, IEEE Transactions onAudio, Speech and Language Processing, vol. 37, no. 3, pp. 328339,1989.

[21] KJ Lang, AH Waibel, and GE Hinton, A time-delay neural networkarchitecture for isolated word recognition, Neural Networks, vol. 3, no.1, pp. 2343, 1990.

[22] T Zeppenfeld, R Houghton, and A Waibel, Improving the MS-TDNNfor word spotting, in Proc IEEE ICASSP, 1993, vol. 2, pp. 475478.

[23] S Sukittanon, AC Surendran, JC Platt, and CJC Burges, Convolutionalnetworks for speech detection, in Proc. ICSLP, 2004.

[24] D Palaz, R Collobert, and M Magimai-Doss, Estimating phoneme classconditional probabilities from raw speech signal using convolutionalneural networks, in Proc Interspeech, 2013.

[25] O Abdel-Hamid, A-R Mohamed, J Hui, and G Penn, Applyingconvolutional neural networks concepts to hybrid NNHMM model forspeech recognition., in Proc IEEE ICASSP, 2012, pp. 42774280.

[26] TN Sainath, A Mohamed, B Kingsbury, and B Ramabhadran, Deepconvolutional neural networks for LVCSR, in Proc IEEE ICASSP,2013.

[27] TN Sainath, B Kingsbury, A Mohamed, GE Dahl, G Saon, H Soltau,T Beran, AY Aravkin, and B Ramabhadran, Improvements to deepconvolutional neural networks for LVCSR, in Proc IEEE ASRU, 2013.

[28] J Carletta, Unleashing the killer corpus: experiences in creatingthe multi-everything AMI Meeting Corpus, Language Resources &Evaluation Journal, vol. 41, no. 2, pp. 181190, 2007.

[29] MD Richard and RP Lippmann, Neural network classifiers estimateBayesian a posteriori probabilities, Neural Computation, vol. 3, no. 4,pp. 461483, 1991.

[30] MA Ranzato, FJ Huang, Y-L Boureau, and Y LeCun, Unsupervisedlearning of invariant feature hierarchies with applications to objectrecognition, in IEEE CVPR, 2007.

[31] NumPy Reference, March 2014, http://docs.scipy.org/doc/numpy/numpy-ref-1.8.1.pdf [Online; accessed 27-March-2014].

[32] O Abdel-Hamid, L Deng, and D Yu, Exploring convolutional neuralnetwork structures and optimisation techniques for speech recognition,in In Proc. Interspeech. 2013, ICSA.

[33] M Riesenhuber and T Poggio, Hierarchical models of object recogni-tion in cortex, Nature Neuroscience, vol. 2, pp. 10191025, 1999.

[34] JG Fiscus, J Ajot, N Radde, and C Laprun, Multiple dimensionLevenshtein edit distance calculations for evaluating ASR systemsduring simultaneous speech, in Proc. LREC, 2006.

[35] D Povey, A Ghoshal, G Boulianne, L Burget, O Glembek, N Goel,M Hannemann, P Motlcek, Y Qian, P Schwarz, J Silovsky, G Stemmer,and K Vesely, The Kaldi speech recognition toolkit, in Proc. IEEEASRU, December 2011.

[36] IJ Goodfellow, D Warde-Farley, P Lamblin, V Dumoulin, M Mirza,R Pascanu, J Bergstra, F Bastien, and Y Bengio, Pylearn2: a machinelearning research library, arXiv preprint arXiv:1308.4214, 2013.

[37] C Cieri, D Miller, and K Walker, From Switchboard to Fisher: Tele-phone collection protocols, their uses and yields, in Proc Eurospeech,2003.

[38] SF Chen and J Goodman, An empirical study of smoothing techniquesfor language modeling, Computer Speech & Language, vol. 13, no. 4,pp. 359393, 1999.

[39] D Povey, D Kanevsky, B Kingsbury, B Ramabhadran, G Saon, andK Visweswariah, Boosted MMI for model and feature-space discrimi-native training, in Proc IEEE ICASSP, 2008, pp. 40574060.

[40] MJF Gales, Semi-tied covariance matrices for hidden Markov models,IEEE Transactions on Speech and Audio Processing, vol. 7, no. 3, pp.272281, 1999.

[41] G Hinton, S Osindero, and Y Teh, A fast learning algorithm for deepbelief nets, Neural Computation, vol. 18, pp. 15271554, 2006.

[42] X Anguera, C Wooters, and J Hernando, Acoustic beamforming forspeaker diarization of meetings, IEEE Trans. Audio, Speech, LanguageProcess., vol. 15, no. 7, pp. 20112021, 2007.

[43] H Lee, P Pham, Y Largman, and A Ng, Unsupervised feature learningfor audio classification using convolutional deep belief networks, inAdvances in Neural Information Processing Systems 22, 2009, pp. 10961104.


p1-swietojanski

Documents