Speech & Audio
A WavLM-based detector with a multi-fusion attentive classifier reported the strongest results on one audio-spoofing benchmark
In a 2024 paper, researchers at Zhejiang University reported that a detector pairing the WavLM speech model with a multi-fusion attentive classifier achieved the strongest results on the ASVspoof 2021 DF audio-spoofing benchmark. The authors reported competitive results on two other benchmarks, ASVspoof 2019 LA and ASVspoof 2021 LA. The paper was published in the proceedings of IEEE International Conference on Acoustics Speech and Signal Processing, a peer-reviewed conference.
What problem does the paper address?
The authors write that audio deepfakes, speech synthesized or converted to imitate a speaker, have become a serious threat to automatic speaker verification, the task of checking whether a voice belongs to the claimed speaker. They propose a countermeasure that uses the WavLM model to extract speech features useful for spoofing detection. Those features feed a Multi-Fusion Attentive classifier based on an Attentive Statistics Pooling layer.
What does the detector do?
WavLM is a self-supervised speech model, according to the abstract. The Multi-Fusion Attentive classifier captures complementary information from the audio at both time and layer levels. That means it brings together different views of the sound instead of relying on one summary. The authors report they use WavLM to extract features more conducive to spoofing detection, which they say is the first such use.
What did the tests show?
The authors tested the method on three ASVspoof benchmark sets. On ASVspoof 2021 DF, they report state-of-the-art results. On ASVspoof 2019 and ASVspoof 2021 LA, they report competitive results.
FAQ
What is ASVspoof?
ASVspoof is a family of benchmark evaluations for audio spoofing detection. The paper reports results on the 2019 and 2021 sets.
Does the method lead on every benchmark?
No. The authors report state-of-the-art results on the ASVspoof 2021 DF set. On the ASVspoof 2019 and 2021 LA sets, they describe the results as competitive. The abstract does not claim a lead on those sets.
What makes this detector different?
It uses WavLM to extract spoofing-relevant features and adds a multi-fusion attentive classifier that combines information across time and model layers. The authors report the method achieves state-of-the-art results on the ASVspoof 2021 DF set.
Limits and what's next
This is one conference paper. The evidence comes from three benchmark sets: ASVspoof 2021 DF, ASVspoof 2019 LA, and ASVspoof 2021 LA. The abstract reports no independent replication. It describes no testing outside these benchmarks. The next step would be evidence from other datasets or real-world recordings, which the abstract does not provide.
Source: Yinlin Guo et al., “Audio Deepfake Detection With Self-Supervised Wavlm And Multi-Fusion Attentive Classifier,” IEEE International Conference on Acoustics Speech and Signal Processing, 2024.