Few-Shot Transfer for Speech Enhancement Using SEGAN with Stability Guardrails

Abstract

High-quality speech communication is often compromised by background noise, reducing intelligibility and perceived quality. We investigate data-efficient few-shot transfer of the speech enhancement generative adversarial network (SEGAN) to a new noise domain. Starting from a generator pre-trained on VoiceBank–DEMAND, we adapt the model to MiniLibriMix using only 300 paired noisy–clean examples. To prevent overfitting and catastrophic forgetting, we introduce stable adversarial few-shot enhancement (SAFE), a three-fold stabilisation strategy with (1) exponential-moving-average (EMA) weight averaging, (2) L2-SP weight anchoring to the source-domain parameters, and (3) a teacher–student consistency loss. SAFE maintains VoiceBank performance (PESQ ≈ 1.84; STOI ≈ 90 %) and, after an optional perceptual fine-tuning stage (L1 + MR-STFT), yields substantial target-domain gains on MiniLibriMix (PESQ 1.11 → 1.26, STOI 71.5% → 81.5 %) with only a minor source-domain trade-off in STOI. Ablation experiments demonstrate that EMA provides the strongest stabilising effect, while L2-SP and consistency regularisation offer complementary benefits. These results suggest that stable few-shot adaptation may make lightweight time-domain speech enhancers practical for rapid deployment in novel acoustic environments.