Human voice analysis and virtual teacher for speech therapy

Abstract

Based on the literature review, researchers reported that at most, 24.6% of young children in the world were estimated to have speech delay or speech sound disorder (SSD). Once children with SSD are identified, speech-language pathologists (SLPs) select initial therapy programs for children with regular review and adjustments on therapy. The success of therapy highly relies on the effectiveness of long-term home training. In this project, we carry out human voice analysis and design and implement a virtual teacher for home training in speech therapy. For the first part of this project, we conduct sound analysis research to see if children’s Cantonese pronunciation is correct. Once the children’s voices are captured, human voices can be automatically transferred for waveform analysis, allowing a large number of tasks to be completed quickly. The created waveform is compared to the standard waveform. If the majority of the waveform is inconsistent, it suggests that the pronunciation of children is not standard. As a result, it points out children’s pronunciation problems and generates feedback quickly. Through the waveform diagram, our system can accurately process and analyze the sound, as well as eliminate the inaccuracy caused by varied timbres of children, making the analysis more accurate and effective. For the second part of this project, we implement a virtual teacher by using Blender and Audio2Face technology. Blender technology is often useful in areas such as live streaming and business, but it also has great potential in education. Therefore, it provides a more convenient way to conduct speech imitation and language learning for implementation of a virtual teacher. It can achieve low-cost popularization, timely correction of children’s pronunciation problems.