基于自我引导的Transformer的多通道语音分离方法

doi:10.3969/j.issn.1006-2475.2025.06.005

摘要/Abstract

摘要： 摘要：在语音处理领域，多通道语音分离技术旨在从多通道混合语音中有效分离出不同说话人的语音信号。然而，现有方法在处理多通道特征点间长距离依赖关系时存在不足。针对此问题，本文提出一种新颖的基于自我引导的Transformer（SG-former）的多通道语音分离方法，旨在构建一个自适应细粒度的全局注意力机制。SG-former的核心机制在于通过显著性图对Token进行重分配。在此框架下，显著区域能够细粒度地提取关键信息，而次要区域则采取粗粒度提取方式以降低计算成本。显著性图的生成依赖于混合尺度的自注意机制，该机制能够准确捕捉多通道特征点间的长距离依赖关系。为了验证所提方法的有效性，在空间化的WSJ0-2MIX数据库上进行了实验。实验结果显示，SG-former方法相较于基线Beam-Guided TasNet方法，在信号失真比提升（Signal-to-Distortion Ratio Improvement， SDRi）上取得了显著的优势，达到了20.34 dB的提升。这一结果充分表明了SG-former在处理多通道语音分离问题中，特别是在建立长距离依赖关系方面的优越性，且在性能上优于现有技术，为多通道语音分离领域的研究提供了新的思路和方法。

关键词: 关键词：语音分离, 自我引导注意力, 混合尺度注意力, 多通道处理, 波束形成器

Abstract: Abstract: In the field of speech processing， multi-channel speech separation technology aims to effectively separate the speech signals of different speakers from multi-channel mixed speech. However， existing methods have shortcomings in handling long-distance dependency relationships between multi-channel feature points. In response to this issue， this study proposes a novel multi-channel speech separation method based on self-guided Transformer （SG-former）， which aims to construct an adaptive fine-grained global attention mechanism. The core mechanism of SG-former is to reassign tokens through saliency maps. Under this framework， significant regions can extract key information at a fine-grained level， while secondary regions adopt a coarse-grained extraction approach to reduce computational costs. The generation of saliency maps relies on the hybrid-scale self-attention mechanism， which can accurately capture the long-distance dependency relationships between multi-channel feature points. To verify the effectiveness of the proposed method， experiments were conducted on a spatialized WSJ0-2MIX database. The experimental results show that the SG-former method has a significant advantage in Signal to Distortion Ratio Improvement （SDRi） compared to the baseline Beam Guided TasNet method， achieving a 20.34 dB improvement. This result fully demonstrates the superiority of SG-former in dealing with multi-channel speech separation problems， especially in establishing long-distance dependency relationships. The experimental results show that this method outperforms existing technologies in terms of performance， providing new ideas and methods for research in the field of multi-channel speech separation.

Key words: Key words: speech separation, self-guided attention, hybrid-scale attention, multi-channel processing, beamformer

中图分类号:

TN912.3

谭应伟, 丁雪枫. 基于自我引导的Transformer的多通道语音分离方法[J]. 计算机与现代化, 2025, 0(06): 28-33.

TAN Yingwei, DING Xuefeng. Multi-Channel Speech Separation Method with Self-guided Transformer[J]. Computer and Modernization, 2025, 0(06): 28-33.

参考文献

［1］ WANG Z Q， WANG P D， WANG D L. Complex spectral mapping for single-and multi-channel speech enhancement and robust ASR［J］. IEEE/ACM Transactions on Audio， Speech， and Language Processing， 2020，28:1778-1787.
［2］ QUAN C S， LI X F. Multichannel speech separation with narrow-band conformer［EB/OL］. ［2022-04-09］. https://arxiv.org/pdf/2204.04464.pdf.
［3］ WANG S， KONG X Y， PENG X L， et al. Dasformer: Deep alternating spectrogram transformer for multi/single-channel speech separa-tion［C］// Proceedings of the 2023 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2023:1-5.
［4］ TAHERIAN H， WANG D L. Multi-resolution location-based training for multi-channel continuous speech separation［C］// Proceedings of the 2023 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2023:1-5.
［5］ CHEN L W， YU M， SU D， et al. Multi-band pit and model integration for improved multi-channel speech separation［C］// Proceedings of the 2019 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2019:705-709.
［6］ YOSHIOKA T， ITO N， DELCROIX M， et al. The NTT CHiME-3 system: Advances in speech enhancement and recognition for mobile multi-microphone devices［C］// Proceedings of the 2015 IEEE Workshop on Automatic Speech Recognition and Understanding （ASRU）. IEEE， 2015:436-443.
［7］ CHEN Z， YOSHIOKA T， XIAO X， et al. Efficient integration of fixed beamformers and speech separation networks for multi-channel far-field speech separation［C］// Proceedings of the 2018 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2018:5384-5388.
［8］ DRUDE L， HAEB-UMBACH R. Tight integration of spatial and spectral features for BSS with deep clustering embeddings［C］// Proceedings of the 2017 Interspeech. ISCA， 2017:2650-2654.
［9］ WANG Z Q， WANG P， WANG D L. Multi-microphone complex spectral mapping for utterance-wise and continuous speech separation［J］. IEEE/ACM Transactions on Audio， Speech， and Language Processing， 2021，29:2001-2014.
［10］ GU R Z， WU J， ZHANG S X， et al. End-to-end multi-channel speech separation［EB/OL］. ［2019-05-15］. https://arxiv.org/pdf/1905.06286.pdf.
［11］ GU R Z， ZHANG S X， CHEN L W， et al. Enhancing end-to-end multi-channel speech separation via spatial feature learning［C］// Proceedings of the 2020 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2020:7319-7323.
［12］ LUO Y， MESGARANI N. Tasnet: Time-domain audio separation network for real-time， single-channel speech separation［C］// Proceedings of the 2018 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2018:696-700.
［13］ LUO Y， MESGARANI N. Conv-tasnet: Surpassing ideal time-frequency magnitude masking for speech separation［J］. IEEE/ACM Transactions on Audio， Speech， and Language Processing， 2019，27（8）:1256-1266.
［14］ GANNOT S， VINCENT E， MARKOVICH-GOLAN S， et al. A consolidated perspective on multimicrophone speech enhancement and source separation［J］. IEEE/ACM Transactions on Audio， Speech， and Language Processing， 2017，25（4）:692-730.
［15］ HEYMANN J， DRUDE L， HAEB-UMBACH R. Neural network based spectral mask estimation for acoustic beamforming［C］// Proceedings of the 2016 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2016:196-200.
［16］ ERDOGAN H， HERSHEY J R， WATANABE S， et al. Improved mvdr beamforming using single-channel mask prediction net-works［C］// Proceedings of the 2016 Interspeech. ISCA， 2016:1981-1985.
［17］ OCHIAI T， DELCROIX M， IKESHITA R， et al. Beam-TasNet:Time-domain audio separation network meets frequency-domain beam-former［C］// Proceedings of the 2020 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2020:6384-6388.
［18］ CHEN H T， YI Y， FENG D， et al. Beam-guided TasNet: An iterative speech separation framework with multi-channel output［EB/OL］. ［2021-02-05］. https://arxiv.org/pdf/2102.02998.pdf.
［19］ SUBAKAN C， RAVANELLI M， CORNELL S， et al. Attention is all you need in speech separation［C］// Proceedings of the 2021 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2021:21-25.
［20］ VINCENT E， GRIBONVAL R， FÉVOTTE C. Performance measurement in blind audio source separation［J］. IEEE Transactions on Audio， Speech， and Language Processing， 2006，14（4）:1462-1469.
［21］ LE ROUX J， WISDOM S， ERDOGAN H， et al. SDR–half-baked or well done?［C］// Proceedings of the 2019 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE， 2019:626-630.
［22］ TAAL C H， HENDRIKS R C， HEUSDENS R， et al. An algorithm for intelligibility prediction of time-frequency weighted noisy speech［J］. IEEE Transactions on Audio， Speech， and Language Processing， 2011，19（7）:2125-2136.
［23］ SOUDEN M， BENESTY J， AFFES S. On optimal frequency-domain multichannel linear filtering for noise reduction ［J］. IEEE Transactions on Audio， Speech， and Language Processing， 2009，18（2）:260-276.
［24］ HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2016:770-778.
［25］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］. Advances in Neural Information Processing Systems， ACM， 2017：6000-6010.
［26］ REN S C， YANG X Y， LIU S H， et al. SG-Former: Self-guided transformer with evolving token reallocation［C］// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. IEEE， 2023:6003-6014.
［27］ WICHERN G， ANTOGNINI J， FLYNN M， et al. Wham!: Extending speech separation to noisy environments［EB/OL］. ［2019-07-02］. https://arxiv.org/pdf/1907.01160.pdf.
［28］ PARIENTE M， CORNELL S， COSENTINO J， et al. Asteroid: The PyTorch-based audio source separation toolkit for researchers［EB/OL］. ［2020-05-08］. https://arxiv.org/pdf/2005.04132.pdf.
［29］ LOIZOU P C. Speech enhancement: Theory and practice［M］. Boca Raton：CRC Press， 2007.

[1]	何若男1, 范翔2, 陈益1, 姜羽菲1, 曹辉1. 比例优势逻辑回归优化嗓音障碍指数算法[J]. 计算机与现代化, 2024, 0(08): 1-4.
[2]	黎雨星，梁正友，孙宇. 结合差分演化和逻辑回归的构音障碍自动识别方法[J]. 计算机与现代化, 2019, 0(08): 1-.
[3]	缑新科1，2，3,徐高鹏1，2，3. 基于Gabor滤波的语音识别鲁棒性研究[J]. 计算机与现代化, 2018, 0(05): 20-.
[4]	卞毓伟，郭玲. 基于权值分配的音符识别研究[J]. 计算机与现代化, 2017, 0(3): 122-.
[5]	陈觉之;张贵荣;周宇欢. 一种基于贪婪策略的说话人语音特征优选方法[J]. 计算机与现代化, 2013, 1(5): 201-205.
[6]	陈觉之;张贵荣;周宇欢. 一种基于SVDD模型的说话人确认方法研究[J]. 计算机与现代化, 2013, 1(4): 5-9.
[7]	叶蕾;方鹏. 文本无关的说话人识别系统抗噪方法研究[J]. 计算机与现代化, 2010, 1(10): 16-19.