Enhancing Speech Separation: Noise vs. Quality

The Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) is a gold standard in supervised speech separation. However, its application isn't without challenges, especially when training references contain noise. The benchmark dataset WSJ0-2Mix exemplifies this problem, where noisy references limit SI-SDR's effectiveness.

The Noise Conundrum

Visualize this: you're trying to separate voices from a crowded room, but the references you're using are already muddled with chatter. That's the scenario with noisy references in datasets like WSJ0-2Mix. The core issue is that noise caps the potential SI-SDR, or worse, taints the separated outputs with noise.

To combat this, researchers have proposed enhancing references and augmenting audio mixtures with the WHAM! dataset. The goal? Train models that don't pick up noisy cues from their reference data.

Improvements with a Catch

Two models trained on these enhanced datasets were put to the test using the non-intrusive NISQA.v2 metric. Results showed a decrease in noise in the separated speech. But here's the kicker: processing references can introduce artifacts, which may limit the perceived quality improvements.

There's a negative correlation between SI-SDR and perceived noisiness across models on the WSJ0-2Mix and Libri2Mix test sets. Numbers in context: while noise is reduced, the trade-off is potential artifacts, which begs the question: Is the quality sacrifice worth the noise reduction?

A Balancing Act

One chart, one takeaway: the quest for cleaner output isn't just about eliminating noise. It's a balancing act between reducing noise and preserving audio quality. When models trained with these methods show improvements in one area, they may suffer in another.

The trend is clearer when you see it: speech separation technologies must evolve to handle noisy references better. Otherwise, we risk trading one problem for another.

The takeaway? The industry needs to refine how we handle noisy references to enhance the utility of SI-SDR without sacrificing the quality. The chart tells the story of an ongoing struggle to get the best from our models without compromise.