APPROACHES TO AUTOMATIC SPEECH RECOGNITION BY MACHINE

How Do Machines Understand Speech? A Look at Three Core Approaches

Have you ever wondered what’s happening under the hood when you ask Siri for the weather or tell Alexa to play your favorite song? Automatic Speech Recognition (ASR) seems like magic, but it’s the result of decades of research into how machines can process and understand human speech.

At its core, ASR aims to convert an acoustic waveform — the physical sound signal — into a written equivalent of the message. This process is incredibly challenging due to variations in pronunciation, speaking rates, accents, and background noise. To tackle this, researchers have developed several distinct methods. it points to three primary approaches to automatic speech recognition by machine: the acoustic-phonetic approach, the pattern recognition approach, and the artificial intelligence approach.

Let’s break down what each of these methods entails.

1. The Acoustic-Phonetic Approach

This classic approach is based on the idea that all spoken language is composed of a finite set of distinct sound units called phonemes. American English, for example, has about 42 phonemes. The acoustic-phonetic method tries to mimic how a human might perceive speech: by identifying these fundamental sound units in order and then piecing them together to form words.

The process, illustrated in the diagram below, generally involves a few key steps:

1. Speech Analysis: The system first performs a spectral analysis of the speech signal to understand its frequency content. Techniques like Linear Predictive Coding (LPC) or filter banks are often used for this step.
2. Feature Detection: It then identifies acoustic properties or features — like nasality, frication (the noisy sound in ‘sh’ or ‘f’), or formant locations (the resonant frequencies of the vocal tract) — that define each phoneme.
3. Segmentation and Labeling: The continuous speech signal is segmented into discrete regions corresponding to different sounds. Each segment is then labeled with the most likely phoneme(s), often resulting in what’s called a “phoneme lattice” — a chart of possible phoneme sequences over time.
4. Word Matching: Finally, the system searches a dictionary (lexicon) to find the word or sequence of words that best matches the phoneme lattice

While intuitive, this approach has proven difficult to implement reliably. The acoustic properties of phonemes are highly variable due to speaker differences and the influence of neighboring sounds (coarticulation). This makes the segmentation and labeling steps extremely error-prone.

2. The Pattern Recognition Approach

Instead of trying to explicitly identify phonetic features, the pattern recognition approach treats speech as a pattern to be classified. This method has become the dominant paradigm in modern speech recognition because it is robust, mathematically rigorous, and has consistently delivered high performance.

Get Helen K Joy’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

It involves two main phases:

1. Pattern Training: The system is “trained” on a large set of speech samples. For each word or sound unit in its vocabulary, it creates a representative reference pattern or statistical model (like a Hidden Markov Model or HMM) by analyzing multiple examples. This is where the machine “learns” the acoustic properties of each sound class.
2. Pattern Comparison (Recognition): When presented with an unknown spoken utterance (the “test pattern”), the system compares it against every stored reference pattern. It then uses a decision rule to find the reference pattern that is the best match, and that becomes the recognized word. To account for differences in speaking rates, this comparison often involves a flexible time-alignment process known as Dynamic Time Warping (DTW).

This approach is powerful because it doesn’t rely on incomplete or error-prone human knowledge of acoustic-phonetics. Instead, it lets the machine learn the essential, repeatable properties of speech sounds directly from data. Its main weakness is that its performance is highly dependent on the quality and quantity of the training data.

3. The Artificial Intelligence (AI) Approach

The AI approach isn’t a completely separate method but rather a hybrid that combines concepts from the two previous approaches to emulate human intelligence. It seeks to improve recognition accuracy by integrating multiple sources of knowledge.

Key features of the AI approach include:

• Knowledge Integration: It uses not only acoustic knowledge but also higher-level linguistic constraints like syntax (grammar rules), semantics (meaning), and pragmatics (context) to resolve ambiguities. For instance, syntax can help correct errors by favoring grammatically valid word sequences.

• Expert Systems: These systems can be used to perform more reliable segmentation and labeling by applying rules derived from various knowledge sources.
• Learning and Adaptation: A core concept in AI is the ability of a system to learn from new data and adapt its models over time.
• Neural Networks: Artificial Neural Networks (ANNs) are often used within the AI framework. These are powerful tools that can learn complex mappings between acoustic signals and phonetic labels or discriminate between similar-sounding words.

Ultimately, the AI approach represents a sophisticated attempt to build more intelligent and context-aware recognition systems by combining the strengths of both acoustic-phonetic theory and statistical pattern recognition.

Each of these approaches offers a different lens through which to view the complex problem of speech recognition. While the pattern recognition approach has dominated the field for years, ongoing research continues to blend these methods, pushing us ever closer to the goal of seamless human-machine communication.

While the pattern recognition approach has dominated the field for years, the future of speech recognition lies in increasingly sophisticated hybrids that blend the statistical power of pattern matching with the deep knowledge integration championed by the AI approach. As these technologies continue to evolve, we move ever closer to the ultimate goal: a world where speaking to our machines is as natural and effortless as speaking to each other.

ref: Rabiner, L. and Juang, B-H. (1993). Fundamentals of Speech Recognition. PTR Prentice-Hall, Inc.