期刊文献+
任意字段
题名或关键词
题名
关键词
摘要
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息
共找到35篇文章
< 1 2 >
每页显示 20 50 100
已选0条
引用分析
参考文献
引证文献
 统计分析
检索结果
已选文献
显示方式：
A Comprehensive Review of Complex Logical Reasoning in Large Vision-Language Models 认领 引用
1
作者 Weiqiang Jin Yang Liu +9 位作者 Yang Gao Shixiang Tang Yanghao Zhou Jinhu Qi Wentao Zhang Junli Wang Jing Gao Yue Ma Ziwei Zhang Biao Zhao 《Computer Modeling in Engineering & Sciences》 SCIE EI 2026年第7期1-51,共51页
relational reasoning,and cross-modal evidence integration.Reasoning abilities such as deductive,inductive,abductive,multi-hop,and causal inference are fundamental to robust decision making,trustworthy interaction,and ... relational reasoning,and cross-modal evidence integration.Reasoning abilities such as deductive,inductive,abductive,multi-hop,and causal inference are fundamental to robust decision making,trustworthy interaction,and real-world deployment,yet they have not been systematically examined in the LVLM literature.Existing surveys mainly discuss mathematical reasoning,general multimodal intelligence,or benchmark progress,but they do not provide a unified account of complex logical reasoning in LVLMs,including its definition,reasoning types,modeling paradigms,evaluation protocols,and unresolved limitations.To address this gap,this survey develops a unified analytical framework for complex logical reasoning in LVLMs.This survey provides a structured review of this emerging area.We first formalize complex logical reasoning in multimodal settings and organize the literature into five recurrent reasoning families:deductive,inductive,abductive,multi-hop,and causal reasoning.We then review reasoning-oriented LVLM architectures,including unified,modular,and tool-augmented paradigms,and summarize major reasoning mechanisms such as chain-of-thought,program-based reasoning,self-correction,and interpretability-oriented analysis.We further examine representative benchmarks and evaluation protocols,with particular attention to the mismatch between final-answer accuracy and genuine reasoning validity.Based on empirical evidence from representative LVLMs and datasets,we identify common capability trends,recurring failure modes,and key open challenges.Our analysis shows that current LVLMs still struggle with reasoning faithfulness,long-horizon inference,cross-modal grounding,hallucination control,and process-aware evaluation.Finally,we outline future directions in reasoning-oriented data construction,model design,training strategies,evaluation methodology,and deployment.Overall,this survey offers a unified conceptual framework and technical roadmap for advancing LVLMs from strong perceptual systems toward reliable multimodal reasoning agents. 展开更多
关键词 **:Large vision-language model complex logical reasoning multimodal reasoning chain-of-thought evaluation benchmark reasoning faithfulness
暂未订购 下载PDF
MMD-Net:A Vision-Language Perception Model for Maritime Object Detection and Application 认领 引用
2
作者 SHI Bo WANG Zitao +2 位作者 CAO Tianyu ZHAO Hong GE Qiqi 《Journal of Ocean University of China》 SCIE CAS CSCD 2026年第2期505-518,共14页
Deep neural network(DNN)-based detectors assist humans in object detection,particularly in maritime transportation,contributing to the autonomy of marine vehicles and systems.In this paper,a multimodal detector networ... Deep neural network(DNN)-based detectors assist humans in object detection,particularly in maritime transportation,contributing to the autonomy of marine vehicles and systems.In this paper,a multimodal detector network(MMD-Net)incorporating natural language as an additional supervisory signal is proposed to improve the performance and generalization of maritime detectors.Motivated by the limitations of traditional neural networks in maritime object detection,the distribution focal scaling is introduced for object classification,and an algorithm that dynamically adjusts the learning rate to accelerate convergence is proposed,aiming to mitigate the significant class imbalance present in maritime datasets.Furthermore,the performance of the region proposal network is improved through our layer-wise training strategy to better capture the diverse sizes and appearances of sea objects.Additionally,different from traditional maritime object detectors,which are confined to predicting a fixed set of predefined object categories,our detector can infer novel categories with rich prior knowledge gained from region-text pre-training.Validation experiments demonstrated that our proposed method achieves a mean average precision(mAP)of 75.8%on the Singapore Maritime Dataset,surpassing other state-of-the-art DNNs.Impressively,even without extra training on novel maritime classes and scenes,the detection accuracy of our model reached 21.1%. 展开更多
关键词 maritime object detection vision-language model zero-shot inference class-imbalanced learning MMD-Net
暂未订购 下载PDF
From Algorithm to Expert:RLHF-Guided Vision-Language Model for 3D-EEM Fluorescence Spectroscopy Matching 认领 引用
3
作者 Chenglong Lu Jiehui Li +5 位作者 Tonglin Chen Changhua Zhou Yixin Fan Xinlin Ren Ziyi Ju Wei Wang 《Computers, Materials & Continua》 SCIE EI 2026年第5期1883-1900,共18页
Existing methods for tracing water pollution sources typically integrate three-dimensional excitationemission matrix(3D-EEM)fluorescence spectroscopy with similarity-based matching algorithms.However,these approaches ... Existing methods for tracing water pollution sources typically integrate three-dimensional excitationemission matrix(3D-EEM)fluorescence spectroscopy with similarity-based matching algorithms.However,these approaches exhibit high error rates in borderline cases and necessitate expert manual review,which limits scalability and introduces inconsistencies between algorithmic outputs and expert judgment.To address these limitations,we propose a large vision-language model(VLM)designed as an“expert agent”to automatically refine similarity scores,ensuring alignment with expert decisions and overcoming key application bottlenecks.The model consists of two core components:(1)rule-based similarity calculation module generate initial spectral similarity scores,and(2)pre-trained large vision-language model fine-tuned via supervised learning and reinforcement learning with human feedback(RLHF)to emulate expert assessments.To facilitate training and evaluation,we introduce two expert-annotated datasets,Spec1k and SpecReason,which capture both quantitative corrections and qualitative reasoning patterns,allowing the model to emulate expert decision-making processes.Experimental results demonstrate that our method achieves 81.45%source attribution accuracy,38.24%higher than rule-based and machine learning baselines.Real-world deployment further validates its effectiveness. 展开更多
关键词 Vision-language model reinforcement learning with human feedback pollution source tracing 3D fluorescence spectroscopy
暂未订购 下载PDF
Adversarial Example Transfer Method for Vision-Language Pre-Training Models Based on Negative Sample Feature Perturbation 认领 引用
4
作者 Zhichao Pei Ou Ye +1 位作者 Panyu Yang Kaiwen He 《Computers, Materials & Continua》 SCIE EI 2026年第8期1201-1221,共21页
To address the issue of insufficient transferability of existing adversarial example generation methods for vision-language pre-training(VLP)models,this paper proposes an adversarial example transfer method for VLP mo... To address the issue of insufficient transferability of existing adversarial example generation methods for vision-language pre-training(VLP)models,this paper proposes an adversarial example transfer method for VLP models based on negative sample feature perturbation.First,a novel cross-modal collaborative perturbation strategy is constructed.By introducing negative samples into the cross-modal perturbation mechanism,the strategy explores more perturbation directions,breaks the original modal alignment constraints and avoids the local focus of adversarial perturbations.Then,to reduce the computational cost,a dynamic threshold attack strategy is built to measure the modal similarity of the generated adversarial examples.Finally,with the help of a multi-modal fusion encoder,a cross-modal fusion semantic attack(CFSA)module is designed.This module extracts the middle-layer features of image-text pairs and improves the transfer attack effect of adversarial examples.The proposed attack method is experimentally evaluated on the Flickr30K and MSCOCO datasets.The results show that for the adversarial examples generated on the Flickr30K dataset,the attack success rate(ASR)of the proposed method reaches up to 95.3%on multiple black-box models;for those generated on the MSCOCO dataset,the maximum attack success rate on multiple black-box models reaches 70.17%.Compared with the current methods,the adversarial examples generated by the proposed method achieve better attack performance. 展开更多
关键词 Vision-language pre-training model multimodal adversarial attack transferability cross-modality perturbation negative samples
暂未订购 下载PDF
A Review on Vision-Language-Based Approaches: Challenges and Applications 认领 引用 被引量：1
5
作者 Huu-Tuong Ho Luong Vuong Nguyen +4 位作者 Minh-Tien Pham Quang-Huy Pham Quang-Duong Tran Duong Nguyen Minh Huy Tri-Hai Nguyen 《Computers, Materials & Continua》 SCIE EI 2025年第2期1733-1756,共24页
In multimodal learning, Vision-Language Models (VLMs) have become a critical research focus, enabling the integration of textual and visual data. These models have shown significant promise across various natural lang... In multimodal learning, Vision-Language Models (VLMs) have become a critical research focus, enabling the integration of textual and visual data. These models have shown significant promise across various natural language processing tasks, such as visual question answering and computer vision applications, including image captioning and image-text retrieval, highlighting their adaptability for complex, multimodal datasets. In this work, we review the landscape of Bootstrapping Language-Image Pre-training (BLIP) and other VLM techniques. A comparative analysis is conducted to assess VLMs’ strengths, limitations, and applicability across tasks while examining challenges such as scalability, data quality, and fine-tuning complexities. The work concludes by outlining potential future directions in VLM research, focusing on enhancing model interpretability, addressing ethical implications, and advancing multimodal integration in real-world applications. 展开更多
关键词 Bootstrapping language-image pre-training(BLIP) multimodal learning vision-language model(VLM) vision-language pre-training(VLP)
暂未订购 下载PDF
VOTI:Jailbreaking Vision-Language Models via Visual Obfuscation and Task Induction 认领 引用
6
作者 ZHU Yifan CHU Zhixuan REN Kui 《ZTE Communications》 2025年第3期15-26,共12页
In recent years,large vision-language models(VLMs)have achieved significant breakthroughs in cross-modal understanding and generation.However,the safety issues arising from their multimodal interactions become promine... In recent years,large vision-language models(VLMs)have achieved significant breakthroughs in cross-modal understanding and generation.However,the safety issues arising from their multimodal interactions become prominent.VLMs are vulnerable to jailbreak attacks,where attackers craft carefully designed prompts to bypass safety mechanisms,leading them to generate harmful content.To address this,we investigate the alignment between visual inputs and task execution,uncovering locality defects and attention biases in VLMs.Based on these findings,we propose VOTI,a novel jailbreak framework leveraging visual obfuscation and task induction.VOTI subtly embeds malicious keywords within neutral image layouts to evade detection,and breaks down harmful queries into a sequence of subtasks.This approach disperses malicious intent across modalities,exploiting VLMs’over-reliance on local visual cues and their fragility in multi-step reasoning to bypass global safety mechanisms.Implemented as an automated framework,VOTI integrates large language models as red-team assistants to generate and iteratively optimize jailbreak strategies.Extensive experiments across seven mainstream VLMs demonstrate VOTI’s effectiveness,achieving a 73.46%attack success rate on GPT-4o-mini.These results reveal critical vulnerabilities in VLMs,highlighting the urgent need for improving robust defenses and multimodal alignment. 展开更多
关键词 large vision-language models jailbreak attacks red teaming security of large models safety alignment
暂未订购 下载PDF
The Synergy of Seeing and Saying: Revolutionary Advances in Multi-modality Medical Vision-Language Large Models 认领 引用 被引量：1
7
作者 Xiang LI Yu SUN +3 位作者 Jia LIN Like LI Ting FENG Shen YIN 《Artificial Intelligence Science and Engineering》 2025年第2期79-97,共19页
The application of visual-language large models in the field of medical health has gradually become a research focus.The models combine the capability for image understanding and natural language processing,and can si... The application of visual-language large models in the field of medical health has gradually become a research focus.The models combine the capability for image understanding and natural language processing,and can simultaneously process multi-modality data such as medical images and medical reports.These models can not only recognize images,but also understand the semantic relationship between images and texts,effectively realize the integration of medical information,and provide strong support for clinical decision-making and disease diagnosis.The visual-language large model has good performance for specific medical tasks,and also shows strong potential and high intelligence in the general task models.This paper provides a comprehensive review of the visual-language large model in the field of medical health.Specifically,this paper first introduces the basic theoretical basis and technical principles.Then,this paper introduces the specific application scenarios in the field of medical health,including modality fusion,semi-supervised learning,weakly supervised learning,unsupervised learning,cross-domain model and general models.Finally,the challenges including insufficient data,interpretability,and practical deployment are discussed.According to the existing challenges,four potential future development directions are given. 展开更多
关键词 large language models vision-language models medical health multimodality models
暂未订购 下载PDF
VLCA: vision-language aligning model with cross-modal attention for bilingual remote sensing image captioning 认领 引用 被引量：3
8
作者 WEI Tingting YUAN Weilin +2 位作者 LUO Junren ZHANG Wanpeng LU Lina 《Journal of Systems Engineering and Electronics》 SCIE CSCD 2023年第1期9-18,共10页
In the field of satellite imagery, remote sensing image captioning(RSIC) is a hot topic with the challenge of overfitting and difficulty of image and text alignment. To address these issues, this paper proposes a visi... In the field of satellite imagery, remote sensing image captioning(RSIC) is a hot topic with the challenge of overfitting and difficulty of image and text alignment. To address these issues, this paper proposes a vision-language aligning paradigm for RSIC to jointly represent vision and language. First, a new RSIC dataset DIOR-Captions is built for augmenting object detection in optical remote(DIOR) sensing images dataset with manually annotated Chinese and English contents. Second, a Vision-Language aligning model with Cross-modal Attention(VLCA) is presented to generate accurate and abundant bilingual descriptions for remote sensing images. Third, a crossmodal learning network is introduced to address the problem of visual-lingual alignment. Notably, VLCA is also applied to end-toend Chinese captions generation by using the pre-training language model of Chinese. The experiments are carried out with various baselines to validate VLCA on the proposed dataset. The results demonstrate that the proposed algorithm is more descriptive and informative than existing algorithms in producing captions. 展开更多
关键词 remote sensing image captioning(RSIC) vision-language representation remote sensing image caption dataset attention mechanism
暂未订购 下载PDF
IQAGPT:computed tomography image quality assessment with vision-language and ChatGPT models 认领 引用
9
作者 Zhihao Chen Bin Hu +4 位作者 Chuang Niu Tao Chen Yuxin Li Hongming Shan Ge Wang 《Visual Computing for Industry,Biomedicine,and Art》 EI 2024年第1期165-181,共17页
Large language models(LLMs),such as ChatGPT,have demonstrated impressive capabilities in various tasks and attracted increasing interest as a natural language interface across many domains.Recently,large vision-langua... Large language models(LLMs),such as ChatGPT,have demonstrated impressive capabilities in various tasks and attracted increasing interest as a natural language interface across many domains.Recently,large vision-language models(VLMs)that learn rich vision–language correlation from image–text pairs,like BLIP-2 and GPT-4,have been intensively investigated.However,despite these developments,the application of LLMs and VLMs in image quality assessment(IQA),particularly in medical imaging,remains unexplored.This is valuable for objective performance evaluation and potential supplement or even replacement of radiologists’opinions.To this end,this study intro-duces IQAGPT,an innovative computed tomography(CT)IQA system that integrates image-quality captioning VLM with ChatGPT to generate quality scores and textual reports.First,a CT-IQA dataset comprising 1,000 CT slices with diverse quality levels is professionally annotated and compiled for training and evaluation.To better leverage the capabilities of LLMs,the annotated quality scores are converted into semantically rich text descriptions using a prompt template.Second,the image-quality captioning VLM is fine-tuned on the CT-IQA dataset to generate qual-ity descriptions.The captioning model fuses image and text features through cross-modal attention.Third,based on the quality descriptions,users verbally request ChatGPT to rate image-quality scores or produce radiological qual-ity reports.Results demonstrate the feasibility of assessing image quality using LLMs.The proposed IQAGPT outper-formed GPT-4 and CLIP-IQA,as well as multitask classification and regression models that solely rely on images. 展开更多
关键词 Deep learning Medical imaging Image captioning Multimodality Large language model Vision-language model GPT-4 Subjective evaluation
暂未订购 下载PDF
A systematic review of vision and vision-language foundation models in ophthalmology 认领 引用
10
作者 Kai Jin Tao Yu +7 位作者 Gui-shuang Ying Zongyuan Ge Kelvin Zhenghao Li Yukun Zhou Danli Shi Meng Wang Polat Goktas Andrzej Grzybowski 《Advances in Ophthalmology Practice and Research》 CAS 2026年第1期8-19,共12页
Background:Vision and vision-language foundation models,a subset of advanced artificial intelligence(AI)frameworks,have shown transformative potential in various medical fields.In ophthalmology,these models,particular... Background:Vision and vision-language foundation models,a subset of advanced artificial intelligence(AI)frameworks,have shown transformative potential in various medical fields.In ophthalmology,these models,particularly large language models and vision-based models,have demonstrated great potential to improve diagnostic accuracy,enhance treatment planning,and streamline clinical workflows.However,their deployment in ophthalmology has faced several challenges,particularly regarding generalizability and integration into clinical practice.This systematic review aims to summarize the current evidence on the use of vision and visionlanguage foundation models in ophthalmology,identifying key applications,outcomes,and challenges.Main text:A comprehensive search on PubMed,Web of Science,Scopus,and Google Scholar was conducted to identify studies published between January 2020 and July 2025.Studies were included if they developed or applied foundation models,such as vision-based models and large language models,to clinically relevant ophthalmic applications.A total of 10 studies met the inclusion criteria,covering areas such as retinal diseases,glaucoma,and ocular surface tumor.The primary outcome measures are model performance metrics,integration into clinical workflows,and the clinical utility of the models.Additionally,the review explored the limitations of foundation models,such as the reliance on large datasets,computational resources,and interpretability challenges.The majority of studies demonstrated that foundation models could achieve high diagnostic accuracy,with several reports indicating excellent performance comparable to or exceeding those of experienced clinicians.Foundation models achieved high accuracy rates up to 95%for diagnosing retinal diseases,and similar performances for detecting glaucoma progression.Despite promising results,concerns about algorithmic bias,overfitting,and the need for diverse training data were common.High computational demands,EHR compatibility,and the need for clinician validation also posed challenges.Additionally,model interpretability issues hindered clinician trust and adoption.Conclusions:Vision and vision-language foundation models in ophthalmology show significant potential for advancing diagnostic accuracy and treatment strategies,particularly in retinal diseases,glaucoma,and ocular oncology.However,challenges such as data quality,transparency,and ethical considerations must be addressed.Future research should focus on refining model performance,improving interpretability and generalizability,and exploring strategies for integrating these models into routine clinical practice to maximize their impact in clinical ophthalmology. 展开更多
关键词 Ophthalmology Vision foundation models Vision-language models Artificial intelligence Clinical integration
Explainable flood damage assessment using multi-atrous self-attention and vision-language integration 认领 引用
11
作者 Ilhan Aydin Emre Güçlü +1 位作者 Taha Kubilay Sener Erhan Akin 《Artificial Intelligence in Geosciences》 EI CSCD 2026年第1期130-145,共16页
Flood disasters triggered by excessive rainfall cause severe damage to infrastructure and pose significant risks to human life.Within the context of disaster management,accurately identifying affected structures and p... Flood disasters triggered by excessive rainfall cause severe damage to infrastructure and pose significant risks to human life.Within the context of disaster management,accurately identifying affected structures and providing interpretable analytical results are of critical importance.This study proposes a new disaster analysis framework that integrates the Multi-Atrous Self-Attention(MASA)mechanism,which is designed to capture multi-scale spatial features effectively,with vision-language models for explainable flood assessment.The proposed approach consists of two main components.The first component performs segmentation to detect and quantify flood-affected structures,while the second component employs a fine-tuned vision language model to generate natural language descriptions of the disaster scene.The MASA module processes image-mask pairs from the FloodNet dataset to segment disaster related structures,whereas the LoRA(Low Rank Adaptation)enhanced BLIP-2(Bootstrapped Language Image Pre-training)model learns image-text pairs from the LADI-v2 dataset to produce textual disaster descriptions.Through this dual stage structure,the system provides both quantitative and linguistic outputs,enabling interpretable flood impact assessment.Experimental results demonstrate that the proposed MASA-based segmentation model achieves a mean Intersection over Union(mIoU)of 73.78%on FloodNet,outperforming state-of-the-art segmentation models.Furthermore,the LoRA-fine-tuned BLIP-2 model achieves a BLEU score of 80.77%on the LADI-v2 dataset,indicating fluent,contextually relevant,and semantically coherent textual outputs.The proposed system contributes to disaster analysis by enhancing explainability and interpretability in flood damage assessment. 展开更多
关键词 Flood detection Disaster analysis Multi-atrous self-attention(MASA) Vision-language models Explainable artificial intelligence LoRA BLIP-2
Vision-Language Model-Driven Human-Vehicle Interaction for Autonomous Driving:Status,Challenge,and Innovation 认领 引用
12
作者 Rongfeng Zhao Aimin Du +2 位作者 Mobing Cai Zhongpan Zhu Bin He 《Big Data Mining and Analytics》 EI CSCD 2026年第2期425-447,共23页
This paper investigates the potential of Vision-Language Models(VLMs)to enhance Human–Vehicle Interaction(HVI)in Autonomous Driving(AD)scenarios,particularly in interactions between vehicles and other traffic partici... This paper investigates the potential of Vision-Language Models(VLMs)to enhance Human–Vehicle Interaction(HVI)in Autonomous Driving(AD)scenarios,particularly in interactions between vehicles and other traffic participants,with a focus on rationality and safety in external HVI.Leveraging recent advancements in large language models,VLMs demonstrate remarkable capabilities in understanding real-world contexts and generating significant interest in HVI applications.This paper provides an overview of AD,HVI,and VLMs,along with the historical context of large language model applications in HVI.The HVI discussed herein involves dynamic game processes encompassing perception and decision-making between vehicles and traffic participants,such as pedestrians.Furthermore,we examine the perceptual challenges associated with applying VLMs to HVI and compile relevant datasets.This research fills a gap in the existing literature by systematically analyzing the current status,challenges,and future opportunities of VLM applications in HVI.To advance VLM integration in AD,various implementation strategies are discussed.The findings highlight the potential of VLMs to transform HVI in AD,improving both passenger experience and driving safety.Overall,this study contributes to a comprehensive understanding of VLM applications in HVI and provides insights to guide future research and development. 展开更多
关键词 Human-Vehicle Interaction(HVI) Large Language Model(LLM) Vision-Language large Model(VLM) Autonomous Driving(AD) perception technology
KEPT:Knowledge-enhanced prediction of trajectories from consecutive driving frames with vision-language models 认领 引用
13
作者 Yujin Wang Tianyi Wang +7 位作者 Quanfeng Liu Wenxian Fan Junfeng Jiao Christian Claudel Yunbing Yan Bingzhao Gao Jianqiang Wang Hong Chen 《Communications in Transportation Research》 SCIE EI CSCD 2026年第1期167-187,共21页
Accurate short-horizon trajectory prediction is crucial for safe and reliable autonomous driving.However,existing vision language models(VLMs)often fail to accurately understand driving scenes and generate trustworthy... Accurate short-horizon trajectory prediction is crucial for safe and reliable autonomous driving.However,existing vision language models(VLMs)often fail to accurately understand driving scenes and generate trustworthy trajectories.To address this challenge,this study introduces KEPT,a knowledge-enhanced VLM framework that predicts ego trajectories directly from consecutive front-view driving frames.KEPT integrates a temporal frequency–spatial fusion(TFSF)video encoder,which is trained via self-supervised learning with hard-negative mining,with a k-means&HNSW retrieval-augmented generation(RAG)pipeline.Retrieved prior knowledge is added into chain-of-thought(CoT)prompts with explicit planning constraints,while a triple-stage finetuning paradigm aligns the VLM backbone to enhance spatial perception and trajectory prediction capabilities.Evaluated on nuScenes dataset,KEPT achieves the best open-loop performance compared with baseline methods.Ablation studies on finetuning stages,Top-K value of RAG,different retrieval strategies,vision encoders,and VLM backbones are conducted to demonstrate the effectiveness of KEPT.These results indicate that KEPT offers a promising,data-efficient way toward trustworthy trajectory prediction in autonomous driving. 展开更多
关键词 autonomous driving trajectory prediction vision-language model retrieval-augmented generation chain-ofthought prompt
Vision-language model-based human-robot collaboration for smart manufacturing:A state-of-the-art survey 认领 引用 被引量：1
14
作者 Junming FAN Yue YIN +3 位作者 Tian WANG Wenhang DONG Pai ZHENG Lihui WANG 《Frontiers of Engineering Management》 CSCD 2025年第1期177-200,共24页
human-robot collaboration(HRC)is set to transform the manufacturing paradigm by leveraging the strengths of human flexibility and robot precision.The recent breakthrough of Large Language Models(LLMs)and Vision-Langua... human-robot collaboration(HRC)is set to transform the manufacturing paradigm by leveraging the strengths of human flexibility and robot precision.The recent breakthrough of Large Language Models(LLMs)and Vision-Language Models(VLMs)has motivated the preliminary explorations and adoptions of these models in the smart manufacturing field.However,despite the considerable amount of effort,existing research mainly focused on individual components without a comprehensive perspective to address the full potential of VLMs,especially for HRC in smart manufacturing scenarios.To fill the gap,this work offers a systematic review of the latest advance-ments and applications of VLMs in HRC for smart manu-facturing,which covers the fundamental architectures and pretraining methodologies of LLMs and VLMs,their applications in robotic task planning,navigation,and manipulation,and role in enhancing human-robot skill transfer through multimodal data integration.Lastly,the paper discusses current limitations and future research directions in VLM-based HRC,highlighting the trend in fully realizing the potential of these technologies for smart manufacturing. 展开更多
关键词 vision-language models large language models human-robot collaboration smart manufacturing
暂未订购 下载PDF
A survey on pre-training and transfer learning for multimodal Vision-Language Models 认领 引用
15
作者 Zhongren Liang 《Advances in Engineering Innovation》 2025年第7期135-139,共5页
In recent years,Vision-Language Models(VLMs)have emerged as a significant breakthrough in multimodal learning,demonstrating remarkable progress in tasks such as image-text alignment,image generation,and semantic reaso... In recent years,Vision-Language Models(VLMs)have emerged as a significant breakthrough in multimodal learning,demonstrating remarkable progress in tasks such as image-text alignment,image generation,and semantic reasoning.This paper systematically reviews current VLM pretraining methodologies,including contrastive learning and generative paradigms,while providing an in-depth analysis of efficient transfer learning strategies such as prompt tuning,LoRA,and adapter modules.Through representative models like CLIP,BLIP,and GIT,we examine their practical applications in visual grounding,imagetext retrieval,visual question answering,affective computing,and embodied AI.Furthermore,we identify persistent challenges in fine-grained semantic modeling,cross-modal reasoning,and cross-lingual transfer.Finally,we envision future trends in unified architectures,multimodal reinforcement learning,and domain adaptation,aiming to provide systematic reference and technical insights for subsequent research. 展开更多
关键词 Vision-Language Models multimodal learning pre-training transfer learning contrastive learning
Masked Vision-language Transformer in Fashion 认领 引用 被引量：5
16
作者 Ge-Peng Ji Mingchen Zhuge +3 位作者 Dehong Gao Deng-Ping Fan Christos Sakaridis Luc Van Gool 《Machine Intelligence Research》 EI CSCD 2023年第3期421-434,共14页
We present a masked vision-language transformer(MVLT)for fashion-specific multi-modal representation.Technically,we simply utilize the vision transformer architecture for replacing the bidirectional encoder representa... We present a masked vision-language transformer(MVLT)for fashion-specific multi-modal representation.Technically,we simply utilize the vision transformer architecture for replacing the bidirectional encoder representations from Transformers(BERT)in the pre-training model,making MVLT the first end-to-end framework for the fashion domain.Besides,we designed masked image reconstruction(MIR)for a fine-grained understanding of fashion.MVLT is an extensible and convenient architecture that admits raw multimodal inputs without extra pre-processing models(e.g.,ResNet),implicitly modeling the vision-language alignments.More importantly,MVLT can easily generalize to various matching and generative tasks.Experimental results show obvious improvements in retrieval(rank@5:17%)and recognition(accuracy:3%)tasks over the Fashion-Gen 2018 winner,Kaleido-BERT.The code is available at http://gffzz188fe103f8f1460aspno9buvbxcbc6pwx.ffgz.tsg.suse.edu.cn/GewelsJI/MVLT. 展开更多
关键词 Vision-language masked image reconstruction transformer fashion e-commercial
Effectiveness assessment of recent large vision-language models 认领 引用 被引量：3
17
作者 Yao Jiang Xinyu Yan +5 位作者 Ge-Peng Ji Keren Fu Meijun Sun Huan Xiong Deng-Ping Fan Fahad Shahbaz Khan 《Visual Intelligence》 2024年第1期197-213,共17页
The advent of large vision-language models(LVLMs)represents a remarkable advance in the quest for artificial general intelligence.However,the models’effectiveness in both specialized and general tasks warrants furthe... The advent of large vision-language models(LVLMs)represents a remarkable advance in the quest for artificial general intelligence.However,the models’effectiveness in both specialized and general tasks warrants further investigation.This paper endeavors to evaluate the competency of popular LVLMs in specialized and general tasks,respectively,aiming to offer a comprehensive understanding of these novel models.To gauge their effectiveness in specialized tasks,we employ six challenging tasks in three different application scenarios:natural,healthcare,and industrial.These six tasks include salient/camouflagedransparent object detection,as well as polyp detection,skin lesion detection,and industrial anomaly detection.We examine the performance of three recent open-source LVLMs,including MiniGPT-v2,LLaVA-1.5,and Shikra,on both visual recognition and localization in these tasks.Moreover,we conduct empirical investigations utilizing the aforementioned LVLMs together with GPT-4V,assessing their multi-modal understanding capabilities in general tasks including object counting,absurd question answering,affordance reasoning,attribute recognition,and spatial relation reasoning.Our investigations reveal that these LVLMs demonstrate limited proficiency not only in specialized tasks but also in general tasks.We delve deep into this inadequacy and uncover several potential factors,including limited cognition in specialized tasks,object hallucination,text-to-image interference,and decreased robustness in complex problems.We hope that this study can provide useful insights for the future development of LVLMs,helping researchers improve LVLMs for both general and specialized applications. 展开更多
关键词 Large vision-language models(LVLMs) Recognition Localization Multi-modal understanding
LaRP-CLIP:Layer-Aware Refinement with Prototype Guidance for Zero-Shot Anomaly Detection 认领 引用
18
作者 Xing Fang Yuanfang Chen +2 位作者 Qiang Lin Kun Yang Gyu Myoung Lee 《Computers, Materials & Continua》 SCIE EI 2026年第9期1422-1441,共20页
The deployment of supervised anomaly detection is typically limited by the high cost of annotation,privacy constraints,and the scarcity of anomalous samples.These constraints have motivated the use of vision-language ... The deployment of supervised anomaly detection is typically limited by the high cost of annotation,privacy constraints,and the scarcity of anomalous samples.These constraints have motivated the use of vision-language pre-trained models for zero-shot anomaly detection.However,existing CLIP-based methods still face three limitations:a shared set of prompts is applied across feature layers,anomaly maps are fused by fixed strategies,and image-level anomaly scores are determined solely by global image-text similarity.These limitations reduce the accuracy of pixel-level localization and weaken the reliability of image-level anomaly prediction.To overcome these limitations,LaRP-CLIP is proposed.It introduces layer-aware prompt decoupling to better match feature layers with different semantic characteristics,adaptive fusion with error-prior-guided local refinement to produce cleaner and more precise anomaly maps,and a prototype branch to improve image-level scoring.Experiments on four industrial datasets and seven medical datasets show that LaRP-CLIP achieves strong performance in both image-level detection and pixel-level localization. 展开更多
关键词 Zero-shot anomaly detection vision-language models layer-aware prompts local refinement prototype branch
暂未订购 下载PDF
LiRA-CLIP:Training-Free Posterior-Predictive Uncertainty for Few-Shot CLIP Classification 认领 引用
19
作者 Mustafa Qaid Khamisi Zuping Zhang +2 位作者 Mohammed Al-Habib Muhammad Asim Sajid Shah 《Computers, Materials & Continua》 SCIE EI 2026年第7期294-320,共27页
Large Vision-Language models(VLMs)such as Contrastive Language-Image Pretraining(CLIP)have transformed open world image recognition.Nevertheless,few-shot classification,particularly in the extremely low-shot regime,re... Large Vision-Language models(VLMs)such as Contrastive Language-Image Pretraining(CLIP)have transformed open world image recognition.Nevertheless,few-shot classification,particularly in the extremely low-shot regime,requires not only high accuracy but also reliably calibrated uncertainty for decisions with high confidence.Existing training-free CLIP adapters are primarily designed to increase accuracy and efficiency;integrate the zero-shot text logits with the few-shot feature caches,but not definitely model predictive uncertainty and therefore often exhibit considerable miscalibration and weak selective performance.Bayesian adapters move in the direction of probabilistic modeling by placing priors over adapter parameters and employing task-specific variational training;however,this requires gradient-based optimization for every new task,increases computational costs,and becomes fragile when only one or two labeled examples per class are available.Starting from this observation,we introduce a training-free posterior-predictive Likelihood Ratio Adapter(LiRA-CLIP)for few-shot CLIP classification,which directly addresses probabilistic reliability under strict low-shot and deployment constraints.LiRA-CLIP extends the frozen CLIP head by a text-conditioned generative model in feature space that produces heavy-tailed posterior-predictive likelihood ratios,fused with the CLIP logits via a small,reliability-driven calibration layer.This layer is optimized in order to minimize the negative log-likelihood under an explicit accuracy side constraint,which leads to calibrated probabilities and dependable selective decisions without any gradient-based task-specific training.Extensive experiments show that LiRA-CLIP matches or slightly surpasses strong CLIP adapters in top-1 accuracy,while reducing calibration error by roughly 40%-50%and significantly increasing 95%and 99%reliable coverage in the low-shot regime,and thus establishes a new state of the art with respect to probabilistic reliability for training-free few-shot CLIP models. 展开更多
关键词 Vision-language models few-shot learning CLIP training-free uncertainty calibration selective classification posterior predictive modeling
暂未订购 下载PDF
A Novel Unified Framework for Automated Generation and Multimodal Validation of UML Diagrams 认领 引用
20
作者 Van-Viet Nguyen Huu-Khanh Nguyen +4 位作者 Kim-Son Nguyen Thi Minh-Hue Luong Duc-Quang Vu Trung-Nghia Phung The-Vinh Nguyen 《Computer Modeling in Engineering & Sciences》 SCIE EI 2026年第1期1023-1050,共28页
It remains difficult to automate the creation and validation of Unified Modeling Language(UML)dia-grams due to unstructured requirements,limited automated pipelines,and the lack of reliable evaluation methods.This stu... It remains difficult to automate the creation and validation of Unified Modeling Language(UML)dia-grams due to unstructured requirements,limited automated pipelines,and the lack of reliable evaluation methods.This study introduces a cohesive architecture that amalgamates requirement development,UML synthesis,and multimodal validation.First,LLaMA-3.2-1B-Instruct was utilized to generate user-focused requirements.Then,DeepSeek-R1-Distill-Qwen-32B applies its reasoning skills to transform these requirements into PlantUML code.Using this dual-LLM pipeline,we constructed a synthetic dataset of 11,997 UML diagrams spanning six major diagram families.Rendering analysis showed that 89.5%of the generated diagrams compile correctly,while invalid cases were detected automatically.To assess quality,we employed a multimodal scoring method that combines Qwen2.5-VL-3B,LLaMA-3.2-11B-Vision-Instruct and Aya-Vision-8B,with weights based on MMMU performance.A study with 94 experts revealed strong alignment between automatic and manual evaluations,yielding a Pearson correlation of r=0.82 and a Fleiss’Kappa of 0.78.This indicates a high degree of concordance between automated metrics and human judgment.Overall,the results demonstrated that our scoring system is effective and that the proposed generation pipeline produces UML diagrams that are both syntactically correct and semantically coherent.More broadly,the system provides a scalable and reproducible foundation for future work in AI-driven software modeling and multimodal verification. 展开更多
关键词 Automated dataset generation vision-language models multimodal validation software engineering automation UMLCode
暂未订购 下载PDF
已选0条
引用分析
参考文献
引证文献
 统计分析
检索结果
已选文献
上一页 1 2 下一页 到第 页
在线咨询 使用帮助 返回顶部 意见反馈