Speech & Audio
Fast text-to-speech system uses the least memory, rivals the fastest model on long speech, and earns the highest quality score in a listening test
In a 2024 paper at the IEEE International Conference on Acoustics Speech and Signal Processing, researchers at KTH Royal Institute of Technology describe a text-to-speech model called Matcha-TTS. They report that it had the smallest memory footprint among the systems they compared, rivaled the speed of the fastest model on long utterances, and received the highest mean opinion score, a listener rating of speech quality, in a listening test.
What does Matcha-TTS do?
The model is an encoder-decoder system for text-to-speech acoustic modeling, the part of speech synthesis that predicts acoustic features from written text. It is trained with optimal-transport conditional flow matching. The paper says that approach gives a decoder based on an ordinary differential equation that can produce high-quality sound in fewer synthesis steps than models trained with score matching. The paper says Matcha-TTS is probabilistic and non-autoregressive. The paper also says it learns to speak from scratch without external alignments, the labels that match pieces of text to pieces of audio.
How was the system tested?
Researchers compared Matcha-TTS with strong pre-trained baseline models. The paper reports on memory footprint, speed on long utterances, and listener ratings. Matcha-TTS had the smallest memory footprint, rivaled the fastest model on long utterances, and obtained the highest mean opinion score in a listening test. The abstract does not name the dataset or the baseline systems.
FAQ
What is text-to-speech acoustic modeling?
It is the stage of speech synthesis that turns written text into acoustic features, or sound characteristics, that can then become audible speech.
What does non-autoregressive mean?
It means the model does not have to produce one speech frame at a time in strict order. That design can make synthesis faster.
What is a mean opinion score?
A mean opinion score is a common listener rating of perceived speech quality, averaged across human listeners. The paper used it in a listening test.
Limits and what's next
The abstract describes one study by researchers at KTH Royal Institute of Technology. It does not name the setting, the dataset, the languages, the number of listeners, or the baseline models. Researchers compared Matcha-TTS against strong pre-trained baseline models on memory, speed on long utterances, and listener ratings. The abstract does not report an independent replication. Those details limit how far the result can be generalized. The abstract does not state what the researchers plan next.
Source: Shivam Mehta et al., “Matcha-TTS: A Fast TTS Architecture with Conditional Flow Matching,” IEEE International Conference on Acoustics Speech and Signal Processing, 2024.