Speech & Audio
A text-to-speech system showed no statistically significant difference from human recordings in a sentence-level test
In a 2024 paper in IEEE Transactions on Pattern Analysis and Machine Intelligence, researchers at Peking University, Microsoft Research Asia, Microsoft, and the University of Surrey describe a text-to-speech system called NaturalSpeech. Text-to-speech systems turn written words into spoken audio. On the LJSpeech dataset, NaturalSpeech scored -0.01 on a comparative mean opinion score, a listener comparison against human recordings, at the sentence level. A Wilcoxon signed-rank test gave a p-value much greater than 0.05, which the authors report as no statistically significant difference from human recordings.
What did the researchers do?
NaturalSpeech uses a variational autoencoder, a neural model that learns to produce a speech waveform from text. The authors added several components to strengthen the model's text side and simplify its speech side, including phoneme pre-training and a memory mechanism.
What does the result mean?
A comparative mean opinion score asks listeners to judge synthetic speech next to a human recording. NaturalSpeech's -0.01 means the two were rated very close in this test. The authors report that the statistical test found no statistically significant difference.
FAQ
What is a comparative mean opinion score?
It is a listener rating that compares synthetic speech with a human recording. A score near zero means the synthetic speech sounds close to the human sample.
What does a p-value much greater than 0.05 mean?
It is the result of a statistical test. Here the value is much greater than 0.05, and the authors interpret that as no statistically significant difference from human recordings.
Did an outside team verify the result?
The abstract describes one evaluation on the LJSpeech dataset and does not report an independent replication.
Limits and what's next
The reported result comes from one evaluation on the LJSpeech dataset, and the researchers measured it at the sentence level. The abstract does not say how many listeners took part in the test. It reports no independent replication. The abstract does not say how NaturalSpeech performs on longer passages or other languages.
Source: Xu Tan et al., “NaturalSpeech: End-to-End Text-to-Speech Synthesis With Human-Level Quality,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.