ASR Lecture Log 7

Lecture 8 - Neural Networks for Acoustic Modelling 1

After the introduction to neural networks in the previous lecture this lecture was about using neural networks for acoustic modelling. The main idea of the lecture was to show how a neural network could be trained to be a phone classifier, and such a trained neural network phone classifier could be used to either (1) replace the GMM output distributions in an HMM system (the hybrid HMM/NN approach), or (2) be used to generate discriminative features either from the neural network outputs (tandem or posteriorgram features) or from a narrow hidden layer in the neural network (bottleneck features).

First we recapped the previous lecture and looked at how a feed forward network could be trained to map acoustic input frames to phone labels using one or more hidden layers. The input layer can have acoustic context, using a window of frames either side of the current frame.

In a hybrid HMM/NN system the neural network is used to generate the state output likelihoods in place of a GMM. In this case the network outputs will correspond to HMM states - if we have 3 states per phone model and 39 phones, this results in 117 states, and thus the network would have 117 outputs

A network trained to classify HMM states will estimate the posterior probability P(state|acoustics). For an HMM we would like a likelihood of form P(acoustics|state). We obtained such (scaled) likelihoods by dividing the network outputs by the relatively frequency of each class in the training set, which gives us a scaled likelihood we can use for an HMM.

These hybrid HMM/NN systems have been in use since the early 1990s, but it is only in recent years that compute power has been sufficient to train large neural network models which are now the state of the art in acoustic modelling.

Neural networks can also be used to produce features which may be used in a GMM system (or with another neural network). Tandem features (or posteriorgrams) are produced from the output units (typically processed using principal components analysis), where as bottleneck features are obtained from narrow hidden layer.

Informatics Forum, 10 Crichton Street, Edinburgh, EH8 9AB, Scotland, UK
Tel: +44 131 651 5661, Fax: +44 131 651 1426, E-mail: school-office@inf.ed.ac.uk
Please contact our webadmin with any comments or corrections. Logging and Cookies
Unless explicitly stated otherwise, all material is copyright © The University of Edinburgh