AI xPress

AI and computer science research.

Back to Foragentis

Speech & Audio

A speech language model handled speech tasks without speech instruction-tuning data

Speech & Audio November 5, 2026

Hands on the sliders of a mixing desk

In a 2025 paper in computer science and natural language processing, researchers at National Taiwan University and Nvidia describe a speech language model that handled speech tasks without a separate speech instruction-tuning stage. Speech language models combine large language models with pre-trained speech models. The team used an automatic process to create speech-text pairs, aiming to add speech understanding while preserving the text model's language abilities. In benchmark tests on Dynamic-SUPERB and AIR-Bench-Chat, the model handled speech-related tasks and followed complex instructions such as specific output formatting and chain-of-thought reasoning.

What did the researchers test?

They created speech-text pair data automatically instead of through extensive annotation. The aim was to teach the model to understand speech while keeping its original language skills. They then evaluated the model on the Dynamic-SUPERB and AIR-Bench-Chat benchmarks.

How did the model do on the benchmarks?

The authors report that the model performed well on Dynamic-SUPERB and AIR-Bench-Chat. It handled general speech-related tasks without speech instruction-tuning data. The model also followed complex instructions derived from large language models, including specific output formatting and chain-of-thought reasoning. The authors say the approach reduces reliance on large annotated datasets.

FAQ

What does "without speech instruction-tuning data" mean?

Many speech language models are trained with a large set of spoken instructions paired with correct text responses. This paper describes an automatic process for creating speech-text pairs instead, so the model did not need that instruction-tuning dataset.

Can the model follow complex instructions?

Yes, according to the authors. The abstract says the model followed complex instructions derived from large language models, such as specific output formatting and chain-of-thought reasoning.

Where did this work appear?

The paper was published in 2025 in the IEEE International Conference on Acoustics Speech and Signal Processing, a peer-reviewed conference.

Limits and what's next

The abstract describes one model tested on two benchmarks, Dynamic-SUPERB and AIR-Bench-Chat. It reports no independent replication by another team. The setting is a benchmark evaluation, not a test with people in everyday conditions. The abstract gives no exact benchmark scores or dataset sizes, so the size of the performance gains is not stated from this summary. The authors frame the approach as a step toward more efficient speech understanding systems, but the abstract does not name a specific follow-up study.

Source: Ke-Han Lu et al., “Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data,” IEEE International Conference on Acoustics Speech and Signal Processing, 2025.