Researchers have developed a novel deep learning model that leverages human perception to significantly improve audio quality in real-world scenarios. By incorporating subjective ratings of sound quality made by individuals, the model outperforms standard approaches in minimizing the presence of noisy audio, leading to enhanced speech quality as measured by objective metrics.
Traditional methods of limiting background noise in audio signals rely on AI algorithms to extract noise from the desired signal. However, these objective methods do not always align with listeners' assessments of speech intelligibility and clarity. The new model developed by the researchers utilizes human perception to train the model to remove unwanted sounds effectively.
The study, published in the journal IEEE/ACM Transactions on Audio, Speech, and Language Processing, focused on enhancing monaural speech—speech from a single audio channel—by training the model on datasets containing recordings of people speaking in various background noise conditions. Listeners rated the speech quality of each recording on a scale of 1 to 100, providing subjective evaluations for model training.
The model employs a joint-learning method that combines a specialized speech enhancement language module with a prediction model capable of anticipating the mean opinion score assigned by human listeners to noisy signals. Results demonstrated that the new approach outperformed other models in terms of perceptual quality, intelligibility, and human ratings.
Despite the effectiveness of incorporating human perception into the model, challenges related to the subjective nature of sound quality evaluation remain. Factors such as individual hearing capabilities and experiences influence perception, highlighting the need for ongoing refinement of the model to ensure user-friendly audio enhancement.
Enhancing the quality of noisy speech is essential for various applications, including hearing aids, speech recognition programs, and hands-free communication systems. Future advancements may involve real-time audio augmentation technologies that adjust sound environments to improve the listening experience.
The researchers emphasize the importance of continued human involvement in the machine learning AI process to address the complexities of audio enhancement and meet the evolving expectations of users. By incorporating human subjective evaluations, the model can adapt to handle more complex audio systems and deliver enhanced audio experiences to consumers.
