期刊文献+
任意字段
题名或关键词
题名
关键词
摘要
作者
第一作者
机构
刊名
分类号
参考文献
作者简介
基金资助
栏目信息
共找到15篇文章
< 1 >
每页显示 20 50 100
已选0条
引用分析
参考文献
引证文献
 统计分析
检索结果
已选文献
显示方式：
UniTrans:Unified Parameter-Efficient Transfer Learning and Multimodal Alignment for Large Multimodal Foundation Model 认领 引用
1
作者 Jiakang Sun Ke Chen +3 位作者 Xinyang He Xu Liu Ke Li Cheng Peng 《Computers, Materials & Continua》 SCIE EI 2025年第4期219-238,共20页
With the advancements in parameter-efficient transfer learning techniques,it has become feasible to leverage large pre-trained language models for downstream tasks under low-cost and low-resource conditions.However,ap... With the advancements in parameter-efficient transfer learning techniques,it has become feasible to leverage large pre-trained language models for downstream tasks under low-cost and low-resource conditions.However,applying this technique to multimodal knowledge transfer introduces a significant challenge:ensuring alignment across modalities while minimizing the number of additional parameters required for downstream task adaptation.This paper introduces UniTrans,a framework aimed at facilitating efficient knowledge transfer across multiple modalities.UniTrans leverages Vector-based Cross-modal Random Matrix Adaptation to enable fine-tuning with minimal parameter overhead.To further enhance modality alignment,we introduce two key components:the Multimodal Consistency Alignment Module and the Query-Augmentation Side Network,specifically optimized for scenarios with extremely limited trainable parameters.Extensive evaluations on various cross-modal downstream tasks demonstrate that our approach surpasses state-of-the-art methods while using just 5%of their trainable parameters.Additionally,it achieves superior performance compared to fully fine-tuned models on certain benchmarks. 展开更多
关键词 Parameter-efficient transfer learning multimodal alignment image captioning image-text retrieval visual question answering
暂未订购 下载PDF
Abnormal Action Detection Based on Parameter-Efficient Transfer Learning in Laboratory Scenarios 认领 引用
2
作者 Changyu Liu Hao Huang +2 位作者 Guogang Huang Chunyin Wu Yingqi Liang 《Computers, Materials & Continua》 SCIE EI 2024年第9期4219-4242,共24页
Laboratory safety is a critical area of broad societal concern,particularly in the detection of abnormal actions.To enhance the efficiency and accuracy of detecting such actions,this paper introduces a novel method ca... Laboratory safety is a critical area of broad societal concern,particularly in the detection of abnormal actions.To enhance the efficiency and accuracy of detecting such actions,this paper introduces a novel method called TubeRAPT(Tubelet Transformer based onAdapter and Prefix TrainingModule).Thismethod primarily comprises three key components:the TubeR network,an adaptive clustering attention mechanism,and a prefix training module.These components work in synergy to address the challenge of knowledge preservation in models pretrained on large datasets while maintaining training efficiency.The TubeR network serves as the backbone for spatio-temporal feature extraction,while the adaptive clustering attention mechanism refines the focus on relevant information.The prefix training module facilitates efficient fine-tuning and knowledge transfer.Experimental results demonstrate the effectiveness of TubeRAPT,achieving a 68.44%mean Average Precision(mAP)on the CLA(Crazy LabActivity)small-scale dataset,marking a significant improvement of 1.53%over the previous TubeR method.This research not only showcases the potential applications of TubeRAPT in the field of abnormal action detection but also offers innovative ideas and technical support for the future development of laboratory safety monitoring technologies.The proposed method has implications for improving safety management systems in various laboratory environments,potentially reducing accidents and enhancing overall workplace safety. 展开更多
关键词 Parameter-efficient transfer learning laboratory scenarios TubeRAPT abnormal action detection
暂未订购 下载PDF
CAMAGRI-GPT:A parameter-efficient agricultural knowledge system using domain-adapted large language models with retrieval augmentation 认领 引用
3
作者 Qianchuan Li Xin Dai +6 位作者 Feng Yu Junsheng Dai Shengwei Wang Rupeng Luan Xiaowei Wang Xiaojing Qin Shiwei Xu 《International Journal of Agricultural and Biological Engineering》 SCIE 2026年第2期245-261,共17页
Despite their transformative potential,large language models(LLMs)remain underutilized in agriculture due to domain-specific data scarcity and computational constraints.This study presents CAMAGRI-GPT,a parameter-effi... Despite their transformative potential,large language models(LLMs)remain underutilized in agriculture due to domain-specific data scarcity and computational constraints.This study presents CAMAGRI-GPT,a parameter-efficient agricultural consultation system that addresses these critical challenges through innovative domain adaptation.A corpus of 2.3 million annotated entries was constructed from raw documents(κ=0.82 agreement,18 categories)and employed LoRA(r=8)and P-tuning v2 to reduce trainable parameters to 0.2%while maintaining 95.8%performance.The RAG framework with HNSW indexing achieves(87±12)ms retrieval latency,enabling real-time consultation.CAMAGRI-GPT demonstrated over 90.0%accuracy across three representative agricultural tasks(crop management,pest and disease diagnosis,and agricultural Q&A),consistently outperforming GPT-3 and BERT-Agri baselines,p<0.001.Median response latency remained below 2 s across all query categories,meeting field deployment requirements.These results demonstrate that domain-adapted LLMs can effectively deliver expert-level agricultural knowledge to resource-constrained farming communities,offering scalable and sustainable solutions to complement declining traditional extension services. 展开更多
关键词 agricultural large language model CAMAGRI-GPT domain adaptation retrieval-augmented generation knowledge dissemination parameter-efficient fine-tuning
MVLFLM:A parameter-efficient large language model framework for cross-domain multi-voltage load forecasting in smart grids 认领 引用
4
作者 Guolong Liu Yan Bai +7 位作者 Huan Zhao Keen Wen Xinlei Wang Jinjin Gu Yanli Liu Gaoqi Liang Junhua Zhao Zhao Yang Dong 《Energy and AI》 EI CSCD 2025年第4期956-975,共20页
Modern smart grids face significant challenges in short-term load forecasting due to increasing complexity across transmission,distribution,and consumer levels.While recent studies have explored large language models ... Modern smart grids face significant challenges in short-term load forecasting due to increasing complexity across transmission,distribution,and consumer levels.While recent studies have explored large language models for load forecasting,existing methods are limited by computational overhead,voltage-level specificity,and inadequate cross-domain generalization.This paper introduces Multi-Voltage Load Forecasting Large Model(MVLFLM),a unified Transformer-based framework that addresses multi-voltage STLF through efficient parameter-fine-tuning of a Llama 2-7B foundation model.Unlike previous LLM-based forecasting methods that focus on single voltage levels or require extensive retraining,MVLFLM employs selective layer freezing to preserve pre-trained knowledge while adapting only essential parameters for load pattern recognition.prehensive Com-evaluation across four real-world datasets spanning high(transmission),medium(distribution),and low(consumer)voltage levels demonstrates MVLFLM’s superior performance,achieving higher performance than benchmarks.Most significantly,MVLFLM exhibits exceptional zero-shot generalization with only 9.07%average performance degradation when applied to unseen grid entities,substantially outperforming existing methods.These results establish MVLFLM as the unified,computationally efficient solution for multi-voltage load forecasting that maintains forecasting accuracy while enabling seamless deployment across heterogeneous smart grid infrastructures. 展开更多
关键词 Large language model Short-term load forecasting Multi-voltage forecasting Parameter-efficient fine-tuning Zero-shot learning Smart grids
Advancing Large Language Models for Low-Resource Languages:A Systematic Review of Pretraining,Adaptation,and Ethical Challenges 认领 引用
5
作者 Ismail Hossain Mridul Banik +2 位作者 Fahmid Al Farid Jia Uddin Hezerul bin Abdul Karim 《Computer Modeling in Engineering & Sciences》 SCIE EI 2026年第7期122-159,共38页
In recent years,the rapid advancement of Large Language Models(LLMs)has significantly transformed natural language processing(NLP),enabling impressive performance across a wide range of tasks.However,these development... In recent years,the rapid advancement of Large Language Models(LLMs)has significantly transformed natural language processing(NLP),enabling impressive performance across a wide range of tasks.However,these developments have largely benefited high-resource languages,leaving many low-resource and underrepresented languages at risk of further digital marginalization.Addressing this imbalance is crucial to building more inclusive and culturally sustainable AI systems,which is motivating growing research interest in adapting LLMs for linguistically diverse and resource-scarce communities.This systematic review examines recent progress(2020–2025)in the pretraining and adaptation of LLMs for Low-Resource Languages(LRLs).Analysed 812 records obtained in the large databases and using PRISMA criteria,140 core studies were identified.The innovations in data augmentation and parameter-efficient fine-tuning approaches can be outlined in this selection process.It combines major innovations on data-driven augmentation,parameter-efficient fine-tuning and morphologically rich and underrepresented language script-sensitive tokenization.The results highlight the growing effectiveness of culturally aware standards such as IrokoBench and BLEnD and show that approaches to lightweight adaptation eliminate high computational costs while maintaining language accuracy.The review focuses on the ethics in AI practice,the development of corpora through communities,and interdisciplinary research collaboration among computational linguists,social scientists,and digital humanists.The task of generating a diversified dataset,typology-conscious modelling strategies,and open-source multilingual benchmarks should be prioritized in future research as one possible solution to the existing digital language gap worldwide. 展开更多
关键词 LLMs low-resource languages(LRLs) parameter-efficient fine-tuning(PEFT) tokenization cross-lingual transfer multilingual benchmarks
暂未订购 下载PDF
ET-BERT with Adapter Fusion:Operating-Regime Analysis of Modular Continual Adaptation for Encrypted Traffic Classification 认领 引用
6
作者 Minsu Kim Daeho Choi +4 位作者 Younghyo Cho Yeog Kim Jun Lee Changhoon Lee Kiwook Sohn 《Computer Modeling in Engineering & Sciences》 SCIE EI 2026年第7期1492-1507,共16页
Future mobile Internet and convergence applications increasingly rely on encrypted protocols,making security monitoring difficult because payload inspection is unavailable while traffic classes and threats evolve cont... Future mobile Internet and convergence applications increasingly rely on encrypted protocols,making security monitoring difficult because payload inspection is unavailable while traffic classes and threats evolve continuously.Encrypted traffic classification models must therefore adapt to newly emerging traffic classes without repeatedly overwriting or fully retraining large Transformer backbones.This study presents and extends an ET-BERT Adapter Fusion framework for AI/ML-driven encrypted-traffic security monitoring in future mobile Internet and convergence applications.The framework keeps the ET-BERT backbone frozen,trains a Base Adapter on USTC-TFC2016 classes 0-9,trains an Incremental Adapter for class 10,and composes them through Adapter Fusion.Across three seeds,Adapter Fusion in the core USTC-TFC201610+1 setting reached 0.9947 Macro-F1,close to Full Fine-Tuning(0.9978)and Consolidated Adapter training on classes 0-10(0.9967),while preserving a modular update structure.A controlled Fusion Boundary study over 2-11 fused adapters showed a limited decrease in Macro-F1 from 0.9947 to 0.9681 rather than an abrupt collapse.Consolidated Adapter capacity experiments showed strong USTC-TFC2016 performance up to 20 classes and a clearer scale-dependent decline on CSTNET-TLS 1.3 from 0.9238 Macro-F1 at 10 classes to 0.7944 at 120 classes.Latency measurements further show that Adapter Fusion has higher inference overhead than Full Fine-Tuning and single-adapter alternatives.The conclusion is therefore not that Adapter Fusion is universally faster or more accurate,but that it offers a modular continual-adaptation regime for AI/ML-driven security in future mobile Internet and convergence applications,with benefits and limitations that depend on adapter count,data availability,and deployment constraints. 展开更多
关键词 Encrypted traffic classification adapter fusion parameter-efficient transfer learning continual learning ET-BERT
暂未订购 下载PDF
DyLoRA-TAD:Dynamic Low-Rank Adapter for End-to-End Temporal Action Detection 认领 引用
7
作者 Jixin Wu Mingtao Zhou +3 位作者 Di Wu Wenqi Ren Jiatian Mei Shu Zhang 《Computers, Materials & Continua》 SCIE EI 2026年第3期2146-2162,共17页
End-to-end Temporal Action Detection(TAD)has achieved remarkable progress in recent years,driven by innovations in model architectures and the emergence of Video Foundation Models(VFMs).However,existing TAD methods th... End-to-end Temporal Action Detection(TAD)has achieved remarkable progress in recent years,driven by innovations in model architectures and the emergence of Video Foundation Models(VFMs).However,existing TAD methods that perform full fine-tuning of pretrained video models often incur substantial computational costs,which become particularly pronounced when processing long video sequences.Moreover,the need for precise temporal boundary annotations makes data labeling extremely expensive.In low-resource settings where annotated samples are scarce,direct fine-tuning tends to cause overfitting.To address these challenges,we introduce Dynamic LowRank Adapter(DyLoRA),a lightweight fine-tuning framework tailored specifically for the TAD task.Built upon the Low-Rank Adaptation(LoRA)architecture,DyLoRA adapts only the key layers of the pretrained model via low-rank decomposition,reducing the number of trainable parameters to less than 5%of full fine-tuning methods.This significantly lowers memory consumption and mitigates overfitting in low-resource settings.Notably,DyLoRA enhances the temporal modeling capability of pretrained models by optimizing temporal dimension weights,thereby alleviating the representation misalignment of temporal features.Experimental results demonstrate that DyLoRA-TAD achieves impressive performance,with 73.9%mAP on THUMOS14,39.52%on ActivityNet-1.3,and 28.2%on Charades,substantially surpassing the best traditional feature-based methods. 展开更多
关键词 Temporal action detection end-to-end training dynamic low-rank adapter parameter-efficient finetuning video understanding
暂未订购 下载PDF
Can Causal Circuits Enable Efficient Spatial Reasoning for Geoinformatics in Large Language Models? 认领 引用
8
作者 Sahil Tripathi Manaswi Kulahara +1 位作者 Abdul Khader Jilani Saudagar Hatoon S.AlSagri 《Computer Modeling in Engineering & Sciences》 SCIE EI 2026年第7期1072-1095,共24页
Spatial reasoning,defined as the ability to infer and compose relations among entities—is fundamental to geoinformatics applications such as spatial querying and map understanding.However,existing works such as Bidir... Spatial reasoning,defined as the ability to infer and compose relations among entities—is fundamental to geoinformatics applications such as spatial querying and map understanding.However,existing works such as Bidirectional Encoder Representations from Transformers(BERT)-based spatial Question Answering(QA)models and neuro-symbolic models rely on dataset-specific patterns,leading to shortcut learning,where reliance on superficial lexical cues rather than true relational understanding.Recent Large Language Models(LLMs)-based works,including fine-tuning and Chain-of-Thought(CoT)prompting,partially alleviate shortcut learning but remain limited by non-causal reasoning,where predictions depend on spurious correlations rather than stable relational structure.To address these limitations,we propose Causal Inference and Reasoning via Compact sUbnetwork IdenTification(CIRCUIT-X),motivated by the hypothesis that spatial reasoning in LLMs is governed by compact causal parameter subsets(a.k.a causal circuits).CIRCUIT-X operates in two stages:(i)Causal Importance Estimation(Stage Ⅰ)via structured interventions to mitigate shortcut learning,and(ii)Minimal Circuit Discovery(Stage Ⅱ)via structured pruning to mitigate non-causal reasoning.Empirically,CIRCUIT-X achieves up to 91%accuracy on SPAtial Reasoning on Textual Question Answering(SPARTQA)and 87%on StepGame,outperforming State-of-the-Art(SOTA)methods while improving intervention robustness by up to+11%and causal consistency by up to+16%.Therefore,it retains up to 96%of full-model performance using only 3%–6%of active parameters,while demonstrating strong robustness under cross-domain transfer with improvements of up to+10%in accuracy and substantially higher intervention stability under distribution shifts. 展开更多
关键词 Causal circuits geoinformatics large language models parameter-efficient learning spatial reasoning structured pruning
暂未订购 下载PDF
H-LoRA:Rethinking Rank Selection for Controllable Knowledge Retention in Edge AI 认领 引用
9
作者 Darren Chai Xin Lun Lim Tong Ming 《Computers, Materials & Continua》 SCIE EI 2026年第7期2010-2031,共22页
The deployment of specialized language models in resource-constrained edge environments(≤1 B parameters,≤2 GB memory,≤100 ms latency)faces a critical challenge:Supervised Fine-Tuning(SFT)achieves domain expertise b... The deployment of specialized language models in resource-constrained edge environments(≤1 B parameters,≤2 GB memory,≤100 ms latency)faces a critical challenge:Supervised Fine-Tuning(SFT)achieves domain expertise but suffers from irreversible catastrophic forgetting,while traditional Low-Rank Adaptation(LoRA)with conservative ranks(r≤64)often underperforms due to insufficient adaptation capacity.This work introduces H-LoRA(High-Rank LoRA)for edge-deployable models and establishes a fundamental distinction between destructive forgetting and controllable knowledge retention.Through comprehensive experiments on compact models(0.12 B Minimind and Qwen-0.5 B)across three domains(Human Resources,Medical,Mathematics)using 29,647 samples,we demonstrate that while both SFT and H-LoRA exhibit general capability degradation,they differ fundamentally:SFT completely destroys the original knowledge structure(1%topic retention),while H-LoRA maintains knowledge integrity with 90%topic retention—an 89 percentage point improvement—enabling post-deployment capability recovery.H-LoRA employs simplified scaling and strategic high-rank adaptation at approximately two-thirds of the model’s hidden dimension(r=512 for d=768),achieving SFT-level domain performance(99.81%precision)with 5×greater parameter efficiency(20.35%trainable parameters)and robust cross-domain generalization(93.5±6.8%average precision).In addition,H-LoRA reduces over-the-air(OTA)update size from 1.4 GB to 96 MB(≈93%),enabling practical and frequent deployment of specialized models in bandwidth-limited edge environments.Beyond demonstrating effectiveness,this work establishes the first comprehensive framework for characterizing specialization-retention trade-offs in parameter-efficient fine-tuning,providing practical guidance for method selection in real-world deployments. 展开更多
关键词 LoRA edge AI knowledge retention domain adaptation parameter-efficient fine-tuning catastrophic forgetting
暂未订购 下载PDF
AdaptForever:Elastic and Mutual Learning for Continuous NLP Task Mastery 认领 引用
10
作者 Ke Chen Cheng Peng +4 位作者 Xinyang He Jiakang Sun Xu Liu Xiaolin Qin Yong Zhong 《Computers, Materials & Continua》 SCIE EI 2025年第3期4003-4019,共17页
In natural language processing(NLP),managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models,leading to practical inefficiencies.To address thi... In natural language processing(NLP),managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models,leading to practical inefficiencies.To address this challenge,we introduce AdaptForever,a novel approach that enables continuous mastery of NLP tasks through the integration of elastic and mutual learning strategies with a stochastic expert mechanism.Our method freezes the pre-trained model weights while incorporating adapters enhanced with mutual learning capabilities,facilitating effective knowledge transfer from previous tasks to new ones.By combining Elastic Weight Consolidation(EWC)for knowledge preservation with specialized regularization terms,AdaptForever successfully maintains performance on earlier tasks while acquiring new capabilities.Experimental results demonstrate that AdaptForever achieves superior performance across a continuous sequence of NLP tasks compared to existing parameter-efficient methods,while effectively preventing catastrophic forgetting and enabling positive knowledge transfer between tasks. 展开更多
关键词 Adapter-tuning large language model pre-trained language model parameter-efficient fine tuning continue learning mutual learning mixture of expert
暂未订购 下载PDF
Optimizing Fine-Tuning in Quantized Language Models:An In-Depth Analysis of Key Variables 认领 引用 被引量：1
11
作者 Ao Shen Zhiquan Lai +1 位作者 Dongsheng Li Xiaoyu Hu 《Computers, Materials & Continua》 SCIE EI 2025年第1期307-325,共19页
Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in speci... Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments. 展开更多
关键词 Large-scale Language Model Parameter-Efficient Fine-Tuning parameter quantization key variable trainable parameters experimental analysis
暂未订购 下载PDF
Semi-Supervised Prefix Tuning of Large Language Models for Industrial Fault Diagnosis with Big Data 认领 引用
12
作者 Gecheng Chen Jiahao Yuan +3 位作者 Jiayu Yao Zheng Luo Jianqiang Li Chengwen Luo 《Big Data Mining and Analytics》 EI CSCD 2025年第6期1353-1368,共16页
Industrial fault diagnosis is crucial for ensuring the safety and efficiency of modern production systems.Industrial big data,particularly large-scale tabular data capturing multivariate time-series processes,offer va... Industrial fault diagnosis is crucial for ensuring the safety and efficiency of modern production systems.Industrial big data,particularly large-scale tabular data capturing multivariate time-series processes,offer valuable operational insights.Existing methods face significant challenges due to extreme label scarcity and massive unlabeled data volumes.Large Language Models(LLMs)hold great potential to address these issues due to their strong heterogeneous and few-shot learning capabilities.However,the application of LLMs to fault diagnosis with industrial big data,especially for tabular data,remains unexplored.In view of this,we propose a novel semi-supervised prefix tuning of LLMs for fault diagnosis with industrial big data.We first generate auxiliary prediction tasks based on the unlabeled data as the semi-supervised training materials for LLMs.Then we design a prefix-based soft embedding layer to fine-tune the LLMs,so that the model is able to learn the task-specific information in a parameter-efficient way.To make the model applicable to industrial big data,we also implement the Sparse Gaussian Processes(SGP)to filter the most informative samples to relieve the computational cost.Finally,we design a hybrid prompt template to effectively combine the hard and soft prompts and formulate the final prediction prompt for the industrial diagnosis tasks.The experiments have proven the superiority of the proposed method. 展开更多
关键词 Industrial big data Large Language Models(LLMs) parameter-efficient fine-tuning Sparse Gaussian Processes(SGP) prompt engineering
DC-LoRA:Domain correlation low-rank adaptation for domain incremental learning 认领 引用
13
作者 Lin Li Shiye Wang +2 位作者 Changsheng Li Ye Yuan Guoren Wang 《High-Confidence Computing》 EI CSCD 2025年第4期1-7,共7页
Continual learning,characterized by the sequential acquisition of multiple tasks,has emerged as a prominent challenge in deep learning.During the process of continual learning,deep neural networks experience a phenome... Continual learning,characterized by the sequential acquisition of multiple tasks,has emerged as a prominent challenge in deep learning.During the process of continual learning,deep neural networks experience a phenomenon known as catastrophic forgetting,wherein networks lose the acquired knowledge related to previous tasks when training on new tasks.Recently,parameter-efficient fine-tuning(PEFT)methods have gained prominence in tackling the challenge of catastrophic forgetting.However,within the realm of domain incremental learning,a type characteristic of continual learning,there exists an additional overlooked inductive bias,which warrants attention beyond existing approaches.In this paper,we propose a novel PEFT method called Domain Correlation Low-Rank Adaptation for domain incremental learning.Our approach put forward a domain correlated loss,which encourages the weights of the LoRA module for adjacent tasks to become more similar,thereby leveraging the correlation between different task domains.Furthermore,we consolidate the classifiers of different task domains to improve prediction performance by capitalizing on the knowledge acquired from diverse tasks.To validate the effectiveness of our method,we conduct comparative experiments and ablation studies on publicly available domain incremental learning benchmark dataset.The experimental results demonstrate that our method outperforms state-of-the-art approaches. 展开更多
关键词 Continual learning Domain incremental learning Parameter-efficient fine-tuning Domain correlation
Collaborative Knowledge Infusion for Low-Resource Stance Detection 认领 引用 被引量：1
14
作者 Ming Yan Tianyi Zhou Joey W.Tsang Ivor 《Big Data Mining and Analytics》 EI CSCD 2024年第3期682-698,共17页
Stance detection is the view towards a specific target by a given context(e.g.tweets,commercial reviews).Target-related knowledge is often needed to assist stance detection models in understanding the target well and ... Stance detection is the view towards a specific target by a given context(e.g.tweets,commercial reviews).Target-related knowledge is often needed to assist stance detection models in understanding the target well and making detection correctly.However,prevailing works for knowledge-infused stance detection predominantly incorporate target knowledge from a singular source that lacks knowledge verification in limited domain knowledge.The low-resource training data further increase the challenge for the data-driven large models in this task.To address those challenges,we propose a collaborative knowledge infusion approach for low-resource stance detection tasks,employing a combination of aligned knowledge enhancement and efficient parameter learning techniques.Specifically,our stance detection approach leverages target background knowledge collaboratively from different knowledge sources with the help of knowledge alignment.Additionally,we also introduce the parameter-efficient collaborative adaptor with a staged optimization algorithm,which collaboratively addresses the challenges associated with low-resource stance detection tasks from both network structure and learning perspectives.To assess the effectiveness of our method,we conduct extensive experiments on three public stance detection datasets,including low-resource and cross-target settings.The results demonstrate significant performance improvements compared to the existing stance detection approaches. 展开更多
关键词 parameter-efficient learning low-resource stance detection knowledge infusion
VAGen:waterbody segmentation with prompting for visual in‑context learning 认领 引用
15
作者 Jiapei Zhao Nobuyoshi Yabuki Tomohiro Fukuda 《AI in Civil Engineering》 2024年第1期1-20,共20页
Effective water management and flood prevention are critical challenges encountered by both urban and rural areas,necessitating precise and prompt monitoring of waterbodies.As a fundamental step in the monitoring proc... Effective water management and flood prevention are critical challenges encountered by both urban and rural areas,necessitating precise and prompt monitoring of waterbodies.As a fundamental step in the monitoring process,waterbody segmentation involves precisely delineating waterbody boundaries from imagery.Previous research using satellite images often lacks the resolution and contextual detail needed for local-scale analysis.In response to these challenges,this study seeks to address them by leveraging common natural images that are more easily accessible and provide higher resolution and more contextual information compared to satellite images.However,the segmentation of waterbodies from ordinary images faces several obstacles,including variations in lighting,occlusions from objects like trees and buildings,and reflections on the water surface,all of which can mislead algorithms.Additionally,the diverse shapes and textures of waterbodies,alongside complex backgrounds,further complicate this task.While large-scale vision models have typically been leveraged for their generalizability across various downstream tasks that are pre-trained on large datasets,their application to waterbody segmentation from ground-level images remains underexplored.Hence,this research proposed the Visual Aquatic Generalist(VAGen)as a countermeasure.Being a lightweight model for waterbody segmentation inspired by visual In-Context Learning(ICL)and Visual Prompting(VP),VAGen refines large visual models by innovatively adding learnable perturbations to enhance the quality of prompts in ICL.As demonstrated by the experimental results,VAGen demonstrated a significant increase in the mean Intersection over Union(mIoU)metric,showing a 22.38%enhancement when compared to the baseline model that lacked the integration of learnable prompts.Moreover,VAGen surpassed the current stateof-the-art(SOTA)task-specific models designed for waterbody segmentation by 6.20%.The performance evaluation and analysis of VAGen indicated its capacity to substantially reduce the number of trainable parameters and computational overhead,and proved its feasibility to be deployed on cost-limited devices including unmanned aerial vehicles(UAVs)and mobile computing platforms.This study thereby makes a valuable contribution to the field of computer vision,offering practical solutions for engineering applications related to urban flood monitoring,agricultural water resource management,and environmental conservation efforts. 展开更多
关键词 Visual in-context learning Visual prompting Vision foundation model Parameter-efficient fine-tuning Waterbody segmentation Deep learning
已选0条
引用分析
参考文献
引证文献
 统计分析
检索结果
已选文献
上一页 1 下一页 到第 页
在线咨询 使用帮助 返回顶部 意见反馈