Speech Emotion Recognition(SER)is a critical component of affective computing with broad applications in human–computer interaction,mental health monitoring,and intelligent multimedia systems.However,SER remains chal...Speech Emotion Recognition(SER)is a critical component of affective computing with broad applications in human–computer interaction,mental health monitoring,and intelligent multimedia systems.However,SER remains challenging due to the emotional ambiguity,lack of labeled data,class imbalance,and speaker variability.This study presents an effective SER framework that integrates contrastive representation learning,optimized spectrogram-based data augmentation,and selective synthetic data generation by using TimeGAN to enhance emotion classification performance.Contrastive learning enables the model to better discriminate acoustically similar emotions while Optuna automatically tunes augmentation strategies such as noise injection,time shifting,and time-frequency masking.Unlike existing approaches that apply synthetic generation uniformly across all classes,the proposed method targets only confusing or under-represented emotion classes to preserve the inter-class separability.A CNN-BiLSTM architecture is used to extract spectral and temporal information of the speech.The framework is evaluated with benchmark SER datasets—EMO-DB and RAVDESS—under speaker independent protocols.Experimental results demonstrate improved accuracy,robustness,and generalization under limited and imbalanced data conditions,supported by confusion matrices,UMAP,and t-SNE visualizations.展开更多
Human activity recognition(HAR)is a method to predict human activities from sensor signals using machine learning(ML)techniques.HAR systems have several applications in various domains,including medicine,surveillance,...Human activity recognition(HAR)is a method to predict human activities from sensor signals using machine learning(ML)techniques.HAR systems have several applications in various domains,including medicine,surveillance,behavioral monitoring,and posture analysis.Extraction of suitable information from sensor data is an important part of the HAR process to recognize activities accurately.Several research studies on HAR have utilizedMel frequency cepstral coefficients(MFCCs)because of their effectiveness in capturing the periodic pattern of sensor signals.However,existing MFCC-based approaches often fail to capture sufficient temporal variability,which limits their ability to distinguish between complex or imbalanced activity classes robustly.To address this gap,this study proposes a feature fusion strategy that merges time-based and MFCC features(MFCCT)to enhance activity representation.The merged features were fed to a convolutional neural network(CNN)integrated with long shortterm memory(LSTM)—DeepConvLSTM to construct the HAR model.The MFCCT features with DeepConvLSTM achieved better performance as compared to MFCCs and time-based features on PAMAP2,UCI-HAR,and WISDM by obtaining an accuracy of 97%,98%,and 97%,respectively.In addition,DeepConvLSTM outperformed the deep learning(DL)algorithms that have recently been employed in HAR.These results confirm that the proposed hybrid features are not only practical but also generalizable,making them applicable across diverse HAR datasets for accurate activity classification.展开更多
Deepfake audio created with sophisticated speech synthesis and voice cloning technologies is a threat to the credibility of digital communication.Its realism has raised serious concerns in different applications such ...Deepfake audio created with sophisticated speech synthesis and voice cloning technologies is a threat to the credibility of digital communication.Its realism has raised serious concerns in different applications such as digital forensics,cybersecurity,media authentication and voice-based security systems.However,deepfake audio detection still remains difficult.Synthetic speech tends to have subtle artifacts that can mimic the natural vocal pattern very closely.Variations in speakers,recording conditions and background noise make the task more complex.In addition,dataset imbalance and low diversity in training samples could lead to low robustness in the model.To overcome these limitations,the present study aims to propose a framework of transfer learning-based methods based on a combination of fine-tuned pre-trained models,as well as systematic data augmentation.Augmentation methods are introduced to increase the variability and mimic real acoustic conditions.This approach supports the learning of more stable and generalizable representations for both genuine and manipulated speech.The framework employs three DL models:ResNet50 to capture global spectro-temporal structures,VGGish to extract mid-level semantic audio embeddings and YAMNet to identify fine-grained temporal irregularities associated with synthetic speech artifacts.Features from these models are fused through concatenation to construct a unified hybrid feature space.A feature selection stage then reduces redundancy before classification using a lightweight model.Experimental results demonstrate the superiority of the proposed hybrid approach and achieved an accuracy of 99.7%.This performance significantly outperformed individual baseline models and achieved strong generalization across diverse acoustic conditions.展开更多
Diagnosing gastrointestinal tract diseases is a critical task requiring accurate and efficient methodologies.While deep learning models have significantly advanced medical image analysis,challenges such as imbalanced ...Diagnosing gastrointestinal tract diseases is a critical task requiring accurate and efficient methodologies.While deep learning models have significantly advanced medical image analysis,challenges such as imbalanced datasets and redundant features persist.This study proposes a novel framework that customizes two deep learning models,NasNetMobile and ResNet50,by incorporating bottleneck architectures,named as NasNeck and ResNeck,to enhance feature extraction.The feature vectors are fused into a combined vector,which is further optimized using an improved Whale Optimization Algorithm to minimize redundancy and improve discriminative power.The optimized feature vector is then classified using artificial neural network classifiers,effectively addressing the limitations of traditional methods.Data augmentation techniques are employed to tackle class imbalance,improving model learning and generalization.The proposed framework was evaluated on two publicly available datasets:Hyper-Kvasir and Kvasir v2.The Hyper-Kvasir dataset,comprising 23 gastrointestinal disease classes,yielded an impressive 96.0%accuracy.On the Kvasir v2 dataset,which contains 8 distinct classes,the framework achieved a remarkable 98.9%accuracy,further demonstrating its robustness and superior classification performance across different gastrointestinal datasets.The results demonstrate the effectiveness of customizing deep models with bottleneck architectures,feature fusion,and optimization techniques in enhancing classification accuracy while reducing computational complexity.展开更多
Automatic Speaker Identification(ASI)involves the process of distinguishing an audio stream associated with numerous speakers’utterances.Some common aspects,such as the framework difference,overlapping of different s...Automatic Speaker Identification(ASI)involves the process of distinguishing an audio stream associated with numerous speakers’utterances.Some common aspects,such as the framework difference,overlapping of different sound events,and the presence of various sound sources during recording,make the ASI task much more complicated and complex.This research proposes a deep learning model to improve the accuracy of the ASI system and reduce the model training time under limited computation resources.In this research,the performance of the transformer model is investigated.Seven audio features,chromagram,Mel-spectrogram,tonnetz,Mel-Frequency Cepstral Coefficients(MFCCs),delta MFCCs,delta-delta MFCCs and spectral contrast,are extracted from the ELSDSR,CSTRVCTK,and Ar-DAD,datasets.The evaluation of various experiments demonstrates that the best performance was achieved by the proposed transformer model using seven audio features on all datasets.For ELSDSR,CSTRVCTK,and Ar-DAD,the highest attained accuracies are 0.99,0.97,and 0.99,respectively.The experimental results reveal that the proposed technique can achieve the best performance for ASI problems.展开更多
Environmental sound classification(ESC)involves the process of distinguishing an audio stream associated with numerous environmental sounds.Some common aspects such as the framework difference,overlapping of different...Environmental sound classification(ESC)involves the process of distinguishing an audio stream associated with numerous environmental sounds.Some common aspects such as the framework difference,overlapping of different sound events,and the presence of various sound sources during recording make the ESC task much more complicated and complex.This research is to propose a deep learning model to improve the recognition rate of environmental sounds and reduce the model training time under limited computation resources.In this research,the performance of transformer and convolutional neural networks(CNN)are investigated.Seven audio features,chromagram,Mel-spectrogram,tonnetz,Mel-Frequency Cepstral Coefficients(MFCCs),delta MFCCs,delta-delta MFCCs and spectral contrast,are extracted fromtheUrbanSound8K,ESC-50,and ESC-10,databases.Moreover,this research also employed three data enhancement methods,namely,white noise,pitch tuning,and time stretch to reduce the risk of overfitting issue due to the limited audio clips.The evaluation of various experiments demonstrates that the best performance was achieved by the proposed transformer model using seven audio features on enhanced database.For UrbanSound8K,ESC-50,and ESC-10,the highest attained accuracies are 0.98,0.94,and 0.97 respectively.The experimental results reveal that the proposed technique can achieve the best performance for ESC problems.展开更多
The Coronavirus Disease 2019(COVID-19)pandemic poses the worldwide challenges surpassing the boundaries of country,religion,race,and economy.The current benchmark method for the detection of COVID-19 is the reverse tr...The Coronavirus Disease 2019(COVID-19)pandemic poses the worldwide challenges surpassing the boundaries of country,religion,race,and economy.The current benchmark method for the detection of COVID-19 is the reverse transcription polymerase chain reaction(RT-PCR)testing.Nevertheless,this testing method is accurate enough for the diagnosis of COVID-19.However,it is time-consuming,expensive,expert-dependent,and violates social distancing.In this paper,this research proposed an effective multimodality-based and feature fusion-based(MMFF)COVID-19 detection technique through deep neural networks.In multi-modality,we have utilized the cough samples,breathe samples and sound samples of healthy as well as COVID-19 patients from publicly available COSWARA dataset.Extensive set of experimental analyses were performed to evaluate the performance of our proposed approach.Several useful features were extracted from the aforementioned modalities that were then fed as an input to long short-term memory recurrent neural network algorithms for the classification purpose.Extensive set of experimental analyses were performed to evaluate the performance of our proposed approach.The experimental results showed that our proposed approach outperformed compared to four baseline approaches published recently.We believe that our proposed technique will assists potential users to diagnose the COVID-19 without the intervention of any expert in minimum amount of time.展开更多
基金supported by Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia through the Researchers Supporting Project PNURSP2026R760.
摘要Speech Emotion Recognition(SER)is a critical component of affective computing with broad applications in human–computer interaction,mental health monitoring,and intelligent multimedia systems.However,SER remains challenging due to the emotional ambiguity,lack of labeled data,class imbalance,and speaker variability.This study presents an effective SER framework that integrates contrastive representation learning,optimized spectrogram-based data augmentation,and selective synthetic data generation by using TimeGAN to enhance emotion classification performance.Contrastive learning enables the model to better discriminate acoustically similar emotions while Optuna automatically tunes augmentation strategies such as noise injection,time shifting,and time-frequency masking.Unlike existing approaches that apply synthetic generation uniformly across all classes,the proposed method targets only confusing or under-represented emotion classes to preserve the inter-class separability.A CNN-BiLSTM architecture is used to extract spectral and temporal information of the speech.The framework is evaluated with benchmark SER datasets—EMO-DB and RAVDESS—under speaker independent protocols.Experimental results demonstrate improved accuracy,robustness,and generalization under limited and imbalanced data conditions,supported by confusion matrices,UMAP,and t-SNE visualizations.
基金supported by Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia through the Researchers Supporting Project PNURSP2025R333.
摘要Human activity recognition(HAR)is a method to predict human activities from sensor signals using machine learning(ML)techniques.HAR systems have several applications in various domains,including medicine,surveillance,behavioral monitoring,and posture analysis.Extraction of suitable information from sensor data is an important part of the HAR process to recognize activities accurately.Several research studies on HAR have utilizedMel frequency cepstral coefficients(MFCCs)because of their effectiveness in capturing the periodic pattern of sensor signals.However,existing MFCC-based approaches often fail to capture sufficient temporal variability,which limits their ability to distinguish between complex or imbalanced activity classes robustly.To address this gap,this study proposes a feature fusion strategy that merges time-based and MFCC features(MFCCT)to enhance activity representation.The merged features were fed to a convolutional neural network(CNN)integrated with long shortterm memory(LSTM)—DeepConvLSTM to construct the HAR model.The MFCCT features with DeepConvLSTM achieved better performance as compared to MFCCs and time-based features on PAMAP2,UCI-HAR,and WISDM by obtaining an accuracy of 97%,98%,and 97%,respectively.In addition,DeepConvLSTM outperformed the deep learning(DL)algorithms that have recently been employed in HAR.These results confirm that the proposed hybrid features are not only practical but also generalizable,making them applicable across diverse HAR datasets for accurate activity classification.
基金supported by Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia through the Researchers Supporting Project PNURSP2026R333.
摘要Deepfake audio created with sophisticated speech synthesis and voice cloning technologies is a threat to the credibility of digital communication.Its realism has raised serious concerns in different applications such as digital forensics,cybersecurity,media authentication and voice-based security systems.However,deepfake audio detection still remains difficult.Synthetic speech tends to have subtle artifacts that can mimic the natural vocal pattern very closely.Variations in speakers,recording conditions and background noise make the task more complex.In addition,dataset imbalance and low diversity in training samples could lead to low robustness in the model.To overcome these limitations,the present study aims to propose a framework of transfer learning-based methods based on a combination of fine-tuned pre-trained models,as well as systematic data augmentation.Augmentation methods are introduced to increase the variability and mimic real acoustic conditions.This approach supports the learning of more stable and generalizable representations for both genuine and manipulated speech.The framework employs three DL models:ResNet50 to capture global spectro-temporal structures,VGGish to extract mid-level semantic audio embeddings and YAMNet to identify fine-grained temporal irregularities associated with synthetic speech artifacts.Features from these models are fused through concatenation to construct a unified hybrid feature space.A feature selection stage then reduces redundancy before classification using a lightweight model.Experimental results demonstrate the superiority of the proposed hybrid approach and achieved an accuracy of 99.7%.This performance significantly outperformed individual baseline models and achieved strong generalization across diverse acoustic conditions.
基金supported by Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia through the Researchers Supporting Project PNURSP2025R333.
摘要Diagnosing gastrointestinal tract diseases is a critical task requiring accurate and efficient methodologies.While deep learning models have significantly advanced medical image analysis,challenges such as imbalanced datasets and redundant features persist.This study proposes a novel framework that customizes two deep learning models,NasNetMobile and ResNet50,by incorporating bottleneck architectures,named as NasNeck and ResNeck,to enhance feature extraction.The feature vectors are fused into a combined vector,which is further optimized using an improved Whale Optimization Algorithm to minimize redundancy and improve discriminative power.The optimized feature vector is then classified using artificial neural network classifiers,effectively addressing the limitations of traditional methods.Data augmentation techniques are employed to tackle class imbalance,improving model learning and generalization.The proposed framework was evaluated on two publicly available datasets:Hyper-Kvasir and Kvasir v2.The Hyper-Kvasir dataset,comprising 23 gastrointestinal disease classes,yielded an impressive 96.0%accuracy.On the Kvasir v2 dataset,which contains 8 distinct classes,the framework achieved a remarkable 98.9%accuracy,further demonstrating its robustness and superior classification performance across different gastrointestinal datasets.The results demonstrate the effectiveness of customizing deep models with bottleneck architectures,feature fusion,and optimization techniques in enhancing classification accuracy while reducing computational complexity.
基金The authors are grateful to the Taif University Researchers Supporting Project Number(TURSP-2020/36)Taif University,Taif,Saudi Arabia.
摘要Automatic Speaker Identification(ASI)involves the process of distinguishing an audio stream associated with numerous speakers’utterances.Some common aspects,such as the framework difference,overlapping of different sound events,and the presence of various sound sources during recording,make the ASI task much more complicated and complex.This research proposes a deep learning model to improve the accuracy of the ASI system and reduce the model training time under limited computation resources.In this research,the performance of the transformer model is investigated.Seven audio features,chromagram,Mel-spectrogram,tonnetz,Mel-Frequency Cepstral Coefficients(MFCCs),delta MFCCs,delta-delta MFCCs and spectral contrast,are extracted from the ELSDSR,CSTRVCTK,and Ar-DAD,datasets.The evaluation of various experiments demonstrates that the best performance was achieved by the proposed transformer model using seven audio features on all datasets.For ELSDSR,CSTRVCTK,and Ar-DAD,the highest attained accuracies are 0.99,0.97,and 0.99,respectively.The experimental results reveal that the proposed technique can achieve the best performance for ASI problems.
基金the Taif University Researchers Supporting Project number(TURSP-2020/36),Taif University,Taif,Saudi Arabia.
摘要Environmental sound classification(ESC)involves the process of distinguishing an audio stream associated with numerous environmental sounds.Some common aspects such as the framework difference,overlapping of different sound events,and the presence of various sound sources during recording make the ESC task much more complicated and complex.This research is to propose a deep learning model to improve the recognition rate of environmental sounds and reduce the model training time under limited computation resources.In this research,the performance of transformer and convolutional neural networks(CNN)are investigated.Seven audio features,chromagram,Mel-spectrogram,tonnetz,Mel-Frequency Cepstral Coefficients(MFCCs),delta MFCCs,delta-delta MFCCs and spectral contrast,are extracted fromtheUrbanSound8K,ESC-50,and ESC-10,databases.Moreover,this research also employed three data enhancement methods,namely,white noise,pitch tuning,and time stretch to reduce the risk of overfitting issue due to the limited audio clips.The evaluation of various experiments demonstrates that the best performance was achieved by the proposed transformer model using seven audio features on enhanced database.For UrbanSound8K,ESC-50,and ESC-10,the highest attained accuracies are 0.98,0.94,and 0.97 respectively.The experimental results reveal that the proposed technique can achieve the best performance for ESC problems.
摘要The Coronavirus Disease 2019(COVID-19)pandemic poses the worldwide challenges surpassing the boundaries of country,religion,race,and economy.The current benchmark method for the detection of COVID-19 is the reverse transcription polymerase chain reaction(RT-PCR)testing.Nevertheless,this testing method is accurate enough for the diagnosis of COVID-19.However,it is time-consuming,expensive,expert-dependent,and violates social distancing.In this paper,this research proposed an effective multimodality-based and feature fusion-based(MMFF)COVID-19 detection technique through deep neural networks.In multi-modality,we have utilized the cough samples,breathe samples and sound samples of healthy as well as COVID-19 patients from publicly available COSWARA dataset.Extensive set of experimental analyses were performed to evaluate the performance of our proposed approach.Several useful features were extracted from the aforementioned modalities that were then fed as an input to long short-term memory recurrent neural network algorithms for the classification purpose.Extensive set of experimental analyses were performed to evaluate the performance of our proposed approach.The experimental results showed that our proposed approach outperformed compared to four baseline approaches published recently.We believe that our proposed technique will assists potential users to diagnose the COVID-19 without the intervention of any expert in minimum amount of time.