Speech & Audio
Real code-switching speech data improved word error rate by 12 to 15 percent under high-similarity conditions in Spanish, Portuguese and Russian speech recognition tests

In a 2025 conference paper, researchers trained a multilingual speech recognition system on real recordings of people switching between languages. They tested it on Spanish, Portuguese, and Russian. Under high-similarity conditions, the system improved word error rate, the share of words the machine transcribed wrong, by 12 to 15 percent. It outperformed baselines trained only on synthetic bilingual data.
The work appeared in the International Conference on Intelligent Communication Networks and Computational Techniques.
How did the researchers train the system?
The researchers started with a Wav2Vec 2.0-XLSR53 speech model, a system that learns cross-lingual speech representations. They added real code-switching training data to capture spontaneous language mixing. Before training, they preprocessed the audio with silence trimming, normalization, and segmentation. They then created sentence embeddings, numerical representations of sentence meaning, with the Language-agnostic BERT Sentence Embedding model. They used cosine similarity scores to pair sentences across languages and adjusted the Wav2Vec 2.0-XLSR53 model after preprocessing to align audio and transcripts.
What did the tests show?
The researchers tested the system on Spanish, Portuguese, and Russian. It improved word error rate by 12 to 15 percent under high-similarity conditions. The gains were strongest in midsentence code-switching, when a speaker changes languages in the middle of a sentence. The system beat baselines that had been trained only on synthetic bilingual data.
Why does word error rate matter?
Word error rate measures how many words a speech recognizer gets wrong. Lower is better. For cognitive voice interfaces, which respond to what a speaker says, a 12 to 15 percent improvement means fewer transcription mistakes in mixed-language speech.
Was the training data real speech?
Yes, the researchers trained with real code-switching data, not solely synthetic bilingual data.
Limits and what's next
The findings come from experimental calculations on Spanish, Portuguese, and Russian. The abstract reports the gains mainly in midsentence code-switching under high-similarity conditions. The comparison was against baselines trained only on synthetic bilingual data. The paper is one study, and the abstract reports no independent replication.
Source: Suresh Kurapati et al., “Improving Multilingual Speech Recognition for Cognitive Voice Interfaces Using Real Code-Switching Data,” International Conference on Intelligent Communication Networks and Computational Techniques (ICICNCT), 2025.