Journals Proceedings

Abstract

This paper is about a lip reading technique for speech recognition by using motion estimation analysis. It presents a user authentication system based on password lip reading. Motion estimation is done for lip movement image sequences representing speech. In this methodology, the motion estimation is computed without extracting the speaker’s lip contours and location. This leads to obtaining robust visual features for lip movements representing utterances. The proposed methodology comprises of two phases, a training phase and a recognition phase. In both the phases an n x n video frame of the image sequence for an utterance (password) is divided into m x m blocks. This method calculates and fits eight curves for each frame. Each curve represents motion estimation of this frame in a specific direction. These eight curves are representing set of features of a specific frame and are extracted in an unsupervised manner. The feature set consists of the integral values of the motion estimation. The feature sets are used to characterize specific utterances with no additional acoustic feature set. A corpus of utterances and their motion estimation features are built in the training phase. The recognition phase is accomplished by extracting the feature set, from the new image sequence of lip movement of an utterance, and compares it to the corpus using the mean square error metric for recognition.