DON’T Use Speech Recognition in Language Learning Apps



After researching ~100 language learning apps in South East Asia and the American app market, I found that only 27% allow users to practice speaking, and most of them use speech recognition as speaking feedback. 

Using speech recognition is ineffective in language learning for two reasons.



1. Good pronunciations are all alike; every mispronunciation is faulty in its own way.


One of the barriers to language learning is the mother tongue, especially for learners 12 years old and above. Deeply influenced by the sound system of their mother tongues, language learners may confuse sounds in their mother tongue with those in a new language. There are over 7000 languages globally, so a single language corresponds to a broad spectrum of mispronunciations from language learners. 


It is happy if a recognition system can recognize speech, but what matters the most is how to deal with mispronunciations, which fail the recognition system. It can not shed light on where and how to improve, but those are where learners should put more effort.


2. Speech recognition is inaccurate when recognizing words.


Words are building blocks of sentences. Trying to speak words first, then putting them together into sentences is classic speaking practice. However, speech recognition is inaccurate in recognizing words.


Traditionally, speech recognition relies on a language model and an acoustic model. However, when recognizing a single word, only acoustic model is involved because the language model can only provide unigram probability but no more. 


So it’s highly possible that when you read the word “OK” correctly, the app tells you that you’re wrong because the recognition result is “Okay”. Speech recognition can not tell apart homonyms.


In summary, if language learning apps make *an efficient learning experience* a priority, then they need to rethink what they need. Not a speech recognition system for speaking practice, at least.


At  SpeechSuper, we develop AI-based speech technologies to analyze speech from language learners, including pronunciation, fluency, completeness, and more. If you’re interested, please contact us on the website.


ABOUT US


Qiusi is a product manager in China’s EdTech industry focusing on language learning and AI. She enjoys writing stories. You can reach her at qiusi.dong@speechsuper.com


SpeechSuper provides cutting-edge AI speech assessment (a.k.a pronunciation assessment or pronunciation score) APIs for language learning products. Comprehensive feedback covers pronunciation score, fluency, completeness, rhythm, stress, liaison, etc. Languages supported include English, Mandarin Chinese, French, German, Korean, Japanese, Russian, Spanish, and more.


*Prior written consent is needed for any form of republication, modification, repost, or distribution of the contents.


Comments

Post a Comment

Popular posts from this blog

How Accurate is Pronunciation Assessment?

Image
Pronunciation assessment APIs usually offer pronunciation scores at phoneme level, syllable level, word level, and sentence level. Yet, how do you know if the scores are accurate? By comparing the predicted pronunciation scores of the testset with the golden standard (human label). The closer the two results, the more accurate the algorithm is.  To make the metric(s) representative and useful, we need to think carefully about 1) testset, and 2) performance metrics. Testset The testset is usually designed by an AI product manager. He/She should ensure that the testset can 1) reflect real user scenarios, 2) have good data variety, and 3) cover a wide range of use cases. For example, SpeechSuper's API testsets consist of masked audios from language learners sampled from our real user base. They usually cover a wide spectrum of phonetic combinations in a specific language. The testsets not only contain data recorded in a quiet environment but also with background noise. Metrics I guess...
Read more

SpeechSuper English Speech to Text API Supports Inverse Text Normalization

Image
SpeechSuper released a new speech-to-text (speech recognition) API feature: inverse text normalization. What is inverse text normalization? It converts the words to numerical or scientific expressions for better readability and understanding. For example, the recognized words 'seventeen dollars' can be converted to '$17' via inverse text normalization. We now support inverse text normalization in the following domains. 1. Cardinal number and currency   SpeechSuper's English speech-to-text API can support number and currency conversion. For example, if the recognized words are 'It costs three hundred and one dollars.', it will be converted to 'It costs $301.' 2. Date SpeechSuper's English speech-to-text API can support date conversion. For example, if the recognized words are 'I was born on November first nineteen ninety-seven.', it will be converted to 'I was born on November 1, 1997'. 3. Decimal number SpeechSuper's English s...
Read more

Enhanced Indian English Pronunciation Assessment Available on SpeechSuper API

Image
Introducing SpeechSuper's latest release: the   Indian English   pronunciation assessment API! Now, you can leverage this powerful tool to assess English word, phrase/sentence, and paragraph pronunciation specifically in the Indian English dialect. Click to experience the demo firsthand. With our API, Indian professionals proficient in English can achieve higher scores while embracing their unique accents. As long as an accent doesn't hinder understanding, it adds to the richness of communication. To make use of this cutting-edge feature, include the following line in your request parameters for English word, sentence, and paragraph pronunciation assessment API, and you're good to go: "accent_dialect": "indian" If you're eager to explore this feature and would like to experience a free trial, simply visit our  website , click the "Contact us" button, and fill out the form. Once submitted, I will promptly reach out to provide you with the fr...
Read more