To address the challenges of low accuracy and insufficient real-time performance in dynamic object detection for UAV surveillance,this paper proposes a novel tracking framework that integrates a lightweight improved Y...To address the challenges of low accuracy and insufficient real-time performance in dynamic object detection for UAV surveillance,this paper proposes a novel tracking framework that integrates a lightweight improved YOLOv5s model with adaptive motion compensation.A UAV-view dynamic feature enhancement strategy is innovatively introduced,and a lightweight detection network combining attention mechanisms and multi-scale fusion is constructed.The robustness of tracking under motion blur scenarios is also optimized.Experimental results demonstrate that the proposed method achieves a mAP@0.5 of 68.2%on the VisDrone dataset and reaches an inference speed of 32 FPS on the NVIDIA Jetson TX2 platform.This significantly improves the balance between accuracy and efficiency in complex scenes,offering reliable technical support for real-time applications such as emergency response.展开更多
Artificial Intelligence(AI)and Computer Vision(CV)advancements have led to many useful methodologies in recent years,particularly to help visually-challenged people.Object detection includes a variety of challenges,fo...Artificial Intelligence(AI)and Computer Vision(CV)advancements have led to many useful methodologies in recent years,particularly to help visually-challenged people.Object detection includes a variety of challenges,for example,handlingmultiple class images,images that get augmented when captured by a camera and so on.The test images include all these variants as well.These detection models alert them about their surroundings when they want to walk independently.This study compares four CNN-based pre-trainedmodels:ResidualNetwork(ResNet-50),Inception v3,DenseConvolutional Network(DenseNet-121),and SqueezeNet,predominantly used in image recognition applications.Based on the analysis performed on these test images,the study infers that Inception V3 outperformed other pre-trained models in terms of accuracy and speed.To further improve the performance of the Inception v3 model,the thermal exchange optimization(TEO)algorithm is applied to tune the hyperparameters(number of epochs,batch size,and learning rate)showing the novelty of the work.Better accuracy was achieved owing to the inclusion of an auxiliary classifier as a regularizer,hyperparameter optimizer,and factorization approach.Additionally,Inception V3 can handle images of different sizes.This makes Inception V3 the optimum model for assisting visually challenged people in real-world communication when integrated with Internet of Things(IoT)-based devices.展开更多
Knowledge distillation(KD)is an emerging model compression technique for learning compact object detector models.Previous KD often focused solely on distilling from the logits layer or the feature intermediate layers,...Knowledge distillation(KD)is an emerging model compression technique for learning compact object detector models.Previous KD often focused solely on distilling from the logits layer or the feature intermediate layers,which may limit the comprehensive learning of the student network.Additionally,the imbalance between the foreground and background also affects the performance of the model.To address these issues,this paper employs feature-based distillation to enhance the detection performance of the bounding box localization part,and logit-based distillation to improve the detection performance of the category prediction part.Specifically,for the intermediate layer feature distillation,we introduce feature resampling to reduce the risk of the student model merely imitating the teacher model.At the same time,we incorporate a Spatial Attention Mechanism(SAM)to highlight the foreground features learned by the student model.In terms of output layer feature distillation,we divide the traditional distillation targets into target-class objects and non-target-class objects,aiming to improve overall distillation performance.Furthermore,we introduce a one-to-many matching distillation strategy based on Feature Alignment Module(FAM),which further enhances the studentmodel’s feature representation ability,making its feature distribution closer to that of the teacher model,and thus demonstrating superior localization and classification capabilities in object detection tasks.Experimental results demonstrate that our proposedmethodology outperforms conventional distillation techniques in terms of object detecting performance.展开更多
alient object detection aims at identifying the visually interesting object regions that are consistent with human perception. Multispectral remote sensing images provide rich radiometric information in revealing the ...alient object detection aims at identifying the visually interesting object regions that are consistent with human perception. Multispectral remote sensing images provide rich radiometric information in revealing the physical properties of the observed objects, which leads to great potential to perform salient object detection for remote sensing images. Conventional salient object detection methods often employ handcrafted features to predict saliency by evaluating the pixel-wise or superpixel-wise contrast. With the recent use of deep learning framework, in particular, fully convolutional neural networks, there has been profound progress in visual saliency detection. However, this success has not been extended to multispectral remote sensing images, and existing multispectral salient object detection methods are still mainly based on handcrafted features, essentially due to the difficulties in image acquisition and labeling. In this paper, we propose a novel deep residual network based on a top-down model, which is trained in an end-to-end manner to tackle the above issues in multispectral salient object detection. Our model effectively exploits the saliency cues at different levels of the deep residual network. To overcome the limited availability of remote sensing images in training of our deep residual network, we also introduce a new spectral image reconstruction model that can generate multispectral images from RGB images. Our extensive experimental results using both multispectral and RGB salient object detection datasets demonstrate a significant performance improvement of more than 10% improvement compared with the state-of-the-art methods.展开更多
Presently,video surveillance is commonly employed to ensure security in public places such as traffic signals,malls,railway stations,etc.A major chal-lenge in video surveillance is the identification of anomalies that...Presently,video surveillance is commonly employed to ensure security in public places such as traffic signals,malls,railway stations,etc.A major chal-lenge in video surveillance is the identification of anomalies that exist in it such as crimes,thefts,and so on.Besides,the anomaly detection in pedestrian walkways has gained significant attention among the computer vision communities to enhance pedestrian safety.The recent advances of Deep Learning(DL)models have received considerable attention in different processes such as object detec-tion,image classification,etc.In this aspect,this article designs a new Panoptic Feature Pyramid Network based Anomaly Detection and Tracking(PFPN-ADT)model for pedestrian walkways.The proposed model majorly aims to the recognition and classification of different anomalies present in the pedestrian walkway like vehicles,skaters,etc.The proposed model involves panoptic seg-mentation model,called Panoptic Feature Pyramid Network(PFPN)is employed for the object recognition process.For object classification,Compact Bat Algo-rithm(CBA)with Stacked Auto Encoder(SAE)is applied for the classification of recognized objects.For ensuring the enhanced results better anomaly detection performance of the PFPN-ADT technique,a comparison study is made using Uni-versity of California San Diego(UCSD)Anomaly data and other benchmark data-sets(such as Cityscapes,ADE20K,COCO),and the outcomes are compared with the Mask Recurrent Convolutional Neural Network(RCNN)and Faster Convolu-tional Neural Network(CNN)models.The simulation outcome demonstrated the enhanced performance of the PFPN-ADT technique over the other methods.展开更多
Human hand detection in uncontrolled environments is a challenging visual recognition task due to numerous variations of hand poses and background image clutter.To achieve highly accurate results as well as provide re...Human hand detection in uncontrolled environments is a challenging visual recognition task due to numerous variations of hand poses and background image clutter.To achieve highly accurate results as well as provide real-time execution,we proposed a deep transfer learning approach over the state-of-the-art deep learning object detector.Our method,denoted as YOLOHANDS,is built on top of the You Only Look Once(YOLO)deep learning architecture,which is modified to adapt to the single class hand detection task.The model transfer is performed by modifying the higher convolutional layers including the last fully connected layer,while initializing lower non-modified layers with the generic pre-trained weights.To address robustness issues,we introduced a comprehensive augmentation procedure over the training image dataset,specifically adapted for the hand detection problem.Experimental evaluation of the proposed method,which is performed on a challenging public dataset,has demonstrated highly accurate results,comparable to the state-of-the-art methods.展开更多
The nutritional value of perishable food items,such as fruits and vegetables,depends on their freshness levels.The existing approaches solve a binary class problem by classifying a known fruit\vegetable class into fre...The nutritional value of perishable food items,such as fruits and vegetables,depends on their freshness levels.The existing approaches solve a binary class problem by classifying a known fruit\vegetable class into fresh or rotten only.We propose an automated fruits and vegetables categorization approach that first recognizes the class of object in an image and then categorizes that fruit or vegetable into one of the three categories:purefresh,medium-fresh,and rotten.We gathered a dataset comprising of 60K images of 11 fruits and vegetables,each is further divided into three categories of freshness,using hand-held cameras.The recognition and categorization of fruits and vegetables are performed through two deep learning models:Visual Geometry Group(VGG-16)and You Only Look Once(YOLO),and their results are compared.VGG-16 classifies fruits and vegetables and categorizes their freshness,while YOLO also localizes them within the image.Furthermore,we have developed an android based application that takes the image of the fruit or vegetable as input and returns its class label and its freshness degree.A comprehensive experimental evaluation of proposed approach demonstrates that the proposed approach can achieve a high accuracy and F1score on gathered FruitVeg Freshness dataset.The dataset is publicly available for further evaluation by the research community.展开更多
在海洋监测领域,对遥感影像中的船舶进行实时、精准的目标检测是确保海上交通安全和海洋环境监管的重要任务。本文提出了一种基于双YOLOv7(Dual You Only Look Once v7,Dual-YOLOv7)模型的船舶检测方法,旨在提高遥感影像中船舶检测的效...在海洋监测领域,对遥感影像中的船舶进行实时、精准的目标检测是确保海上交通安全和海洋环境监管的重要任务。本文提出了一种基于双YOLOv7(Dual You Only Look Once v7,Dual-YOLOv7)模型的船舶检测方法,旨在提高遥感影像中船舶检测的效率和准确性。引入交叉注意力融合(Criss-Cross Attention Fusion,CC Attention-Fusion)模块,并对随机融合(Shuffle-Fusion)模块进行改进,以有效融合红外图像与可见光图像的特征,从而增强模型对不同光照和天气条件下的适应能力。同时,引入深度可分离卷积模型,构造深度可分离空间金字塔池化快速(Depthwise Separable Spatial Pyramid Pooling Fast,D-SimSPPF)模块,在保证检测精度的同时,显著提升了检测速度。此外,构造了CIoU-C(Complete Intersection over Union-Complement)损失函数,使得目标框尺寸的选定更加贴近真实船舶的尺寸,进一步提高了检测的准确性。为了验证模型的实际应用能力,本文将Dual-YOLOv7模型部署在AGX Orin嵌入式平台上,并在多模态船舶MMShip数据集上进行了训练和测试。实验结果表明:与现有的掩码-区域卷积神经网络(Mask Region-Convolutional Neural Networks,Mask R-CNN)、YOLOv8分割(YOLOv8-seg)模型、YOLOv5分割(YOLOv5-seg)模型、YOLOv7分割(YOLOv7-seg)模型和实时检测变换器(Real-time Detection Transformer,RT-DETR)相比,Dual-YOLOv7模型的平均精度(mean Average Precision 50%,mAP50%)指标达到了91.6%,展现出最高的检测精度。同时,该模型的总计算量仅为1176亿次浮点运算,即117.6 GFLOPs(Giga Floating-Point Operations Per Second),算力消耗最低,表明其在实际应用中具有显著的效率优势。结果表明:Dual-YOLOv7模型在船舶检测任务中具有良好的应用前景,能够为海洋监测和管理提供有力的技术支持。展开更多
摘要To address the challenges of low accuracy and insufficient real-time performance in dynamic object detection for UAV surveillance,this paper proposes a novel tracking framework that integrates a lightweight improved YOLOv5s model with adaptive motion compensation.A UAV-view dynamic feature enhancement strategy is innovatively introduced,and a lightweight detection network combining attention mechanisms and multi-scale fusion is constructed.The robustness of tracking under motion blur scenarios is also optimized.Experimental results demonstrate that the proposed method achieves a mAP@0.5 of 68.2%on the VisDrone dataset and reaches an inference speed of 32 FPS on the NVIDIA Jetson TX2 platform.This significantly improves the balance between accuracy and efficiency in complex scenes,offering reliable technical support for real-time applications such as emergency response.
基金Princess Nourah bint Abdulrahman University Researchers Supporting Project number(PNURSP2023R191)Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.The authors would like to thank the Deanship of Scientific Research at Umm Al-Qura University for supporting this work by Grant Code:(22UQU4310373DSR61)This study is supported via funding from Prince Sattam bin Abdulaziz University project number(PSAU/2023/R/1444).
摘要Artificial Intelligence(AI)and Computer Vision(CV)advancements have led to many useful methodologies in recent years,particularly to help visually-challenged people.Object detection includes a variety of challenges,for example,handlingmultiple class images,images that get augmented when captured by a camera and so on.The test images include all these variants as well.These detection models alert them about their surroundings when they want to walk independently.This study compares four CNN-based pre-trainedmodels:ResidualNetwork(ResNet-50),Inception v3,DenseConvolutional Network(DenseNet-121),and SqueezeNet,predominantly used in image recognition applications.Based on the analysis performed on these test images,the study infers that Inception V3 outperformed other pre-trained models in terms of accuracy and speed.To further improve the performance of the Inception v3 model,the thermal exchange optimization(TEO)algorithm is applied to tune the hyperparameters(number of epochs,batch size,and learning rate)showing the novelty of the work.Better accuracy was achieved owing to the inclusion of an auxiliary classifier as a regularizer,hyperparameter optimizer,and factorization approach.Additionally,Inception V3 can handle images of different sizes.This makes Inception V3 the optimum model for assisting visually challenged people in real-world communication when integrated with Internet of Things(IoT)-based devices.
基金funded by National Natural Science Foundation of China(61603245).
摘要Knowledge distillation(KD)is an emerging model compression technique for learning compact object detector models.Previous KD often focused solely on distilling from the logits layer or the feature intermediate layers,which may limit the comprehensive learning of the student network.Additionally,the imbalance between the foreground and background also affects the performance of the model.To address these issues,this paper employs feature-based distillation to enhance the detection performance of the bounding box localization part,and logit-based distillation to improve the detection performance of the category prediction part.Specifically,for the intermediate layer feature distillation,we introduce feature resampling to reduce the risk of the student model merely imitating the teacher model.At the same time,we incorporate a Spatial Attention Mechanism(SAM)to highlight the foreground features learned by the student model.In terms of output layer feature distillation,we divide the traditional distillation targets into target-class objects and non-target-class objects,aiming to improve overall distillation performance.Furthermore,we introduce a one-to-many matching distillation strategy based on Feature Alignment Module(FAM),which further enhances the studentmodel’s feature representation ability,making its feature distribution closer to that of the teacher model,and thus demonstrating superior localization and classification capabilities in object detection tasks.Experimental results demonstrate that our proposedmethodology outperforms conventional distillation techniques in terms of object detecting performance.
基金National 1000 Young Talents Plan of ChinaNational Natural Science Foundation of China(61420106007,61671387,61871325)DECRA of Australica Resenrch Council (DE140100180).
摘要alient object detection aims at identifying the visually interesting object regions that are consistent with human perception. Multispectral remote sensing images provide rich radiometric information in revealing the physical properties of the observed objects, which leads to great potential to perform salient object detection for remote sensing images. Conventional salient object detection methods often employ handcrafted features to predict saliency by evaluating the pixel-wise or superpixel-wise contrast. With the recent use of deep learning framework, in particular, fully convolutional neural networks, there has been profound progress in visual saliency detection. However, this success has not been extended to multispectral remote sensing images, and existing multispectral salient object detection methods are still mainly based on handcrafted features, essentially due to the difficulties in image acquisition and labeling. In this paper, we propose a novel deep residual network based on a top-down model, which is trained in an end-to-end manner to tackle the above issues in multispectral salient object detection. Our model effectively exploits the saliency cues at different levels of the deep residual network. To overcome the limited availability of remote sensing images in training of our deep residual network, we also introduce a new spectral image reconstruction model that can generate multispectral images from RGB images. Our extensive experimental results using both multispectral and RGB salient object detection datasets demonstrate a significant performance improvement of more than 10% improvement compared with the state-of-the-art methods.
摘要Presently,video surveillance is commonly employed to ensure security in public places such as traffic signals,malls,railway stations,etc.A major chal-lenge in video surveillance is the identification of anomalies that exist in it such as crimes,thefts,and so on.Besides,the anomaly detection in pedestrian walkways has gained significant attention among the computer vision communities to enhance pedestrian safety.The recent advances of Deep Learning(DL)models have received considerable attention in different processes such as object detec-tion,image classification,etc.In this aspect,this article designs a new Panoptic Feature Pyramid Network based Anomaly Detection and Tracking(PFPN-ADT)model for pedestrian walkways.The proposed model majorly aims to the recognition and classification of different anomalies present in the pedestrian walkway like vehicles,skaters,etc.The proposed model involves panoptic seg-mentation model,called Panoptic Feature Pyramid Network(PFPN)is employed for the object recognition process.For object classification,Compact Bat Algo-rithm(CBA)with Stacked Auto Encoder(SAE)is applied for the classification of recognized objects.For ensuring the enhanced results better anomaly detection performance of the PFPN-ADT technique,a comparison study is made using Uni-versity of California San Diego(UCSD)Anomaly data and other benchmark data-sets(such as Cityscapes,ADE20K,COCO),and the outcomes are compared with the Mask Recurrent Convolutional Neural Network(RCNN)and Faster Convolu-tional Neural Network(CNN)models.The simulation outcome demonstrated the enhanced performance of the PFPN-ADT technique over the other methods.
基金financed by the Ministry of Education,Science and Technological Development of the Republic of Serbia.
摘要Human hand detection in uncontrolled environments is a challenging visual recognition task due to numerous variations of hand poses and background image clutter.To achieve highly accurate results as well as provide real-time execution,we proposed a deep transfer learning approach over the state-of-the-art deep learning object detector.Our method,denoted as YOLOHANDS,is built on top of the You Only Look Once(YOLO)deep learning architecture,which is modified to adapt to the single class hand detection task.The model transfer is performed by modifying the higher convolutional layers including the last fully connected layer,while initializing lower non-modified layers with the generic pre-trained weights.To address robustness issues,we introduced a comprehensive augmentation procedure over the training image dataset,specifically adapted for the hand detection problem.Experimental evaluation of the proposed method,which is performed on a challenging public dataset,has demonstrated highly accurate results,comparable to the state-of-the-art methods.
摘要The nutritional value of perishable food items,such as fruits and vegetables,depends on their freshness levels.The existing approaches solve a binary class problem by classifying a known fruit\vegetable class into fresh or rotten only.We propose an automated fruits and vegetables categorization approach that first recognizes the class of object in an image and then categorizes that fruit or vegetable into one of the three categories:purefresh,medium-fresh,and rotten.We gathered a dataset comprising of 60K images of 11 fruits and vegetables,each is further divided into three categories of freshness,using hand-held cameras.The recognition and categorization of fruits and vegetables are performed through two deep learning models:Visual Geometry Group(VGG-16)and You Only Look Once(YOLO),and their results are compared.VGG-16 classifies fruits and vegetables and categorizes their freshness,while YOLO also localizes them within the image.Furthermore,we have developed an android based application that takes the image of the fruit or vegetable as input and returns its class label and its freshness degree.A comprehensive experimental evaluation of proposed approach demonstrates that the proposed approach can achieve a high accuracy and F1score on gathered FruitVeg Freshness dataset.The dataset is publicly available for further evaluation by the research community.