Gait recognition is a key biometric for long-distance identification,yet its performance is severely degraded by real-world challenges such as varying clothing,carrying conditions,and changing viewpoints.While combini...Gait recognition is a key biometric for long-distance identification,yet its performance is severely degraded by real-world challenges such as varying clothing,carrying conditions,and changing viewpoints.While combining silhouette and skeleton data is a promising direction,effectively fusing these heterogeneous modalities and adaptively weighting their contributions in response to diverse conditions remains a central problem.This paper introduces GaitMAFF,a novelMulti-modal Adaptive Feature Fusion Network,to address this challenge.Our approach first transforms discrete skeleton joints into a dense SkeletonMap representation to align with silhouettes,then employs an attention-based module to dynamically learn the fusion weights between the two modalities.These fused features are processed by a powerful spatio-temporal backbone withWeighted Global-Local Feature FusionModules(WFFM)to learn a discriminative representation.Extensive experiments on the challenging CCPG and Gait3D datasets show that GaitMAFF achieves state-of-the-art performance,with an average Rank-1 accuracy of 84.6%on CCPG and 58.7%on Gait3D.These results demonstrate that our adaptive fusion strategy effectively integrates complementary multimodal information,significantly enhancing gait recognition robustness and accuracy in complex scenes and providing a practical solution for real-world applications.展开更多
The traditional method of performance degradation prediction and maintenance of rolling bearings only considers a single sensor signal,which makes it difficult to automatically partition degradation stages and prone t...The traditional method of performance degradation prediction and maintenance of rolling bearings only considers a single sensor signal,which makes it difficult to automatically partition degradation stages and prone to over-detection.A new method of performance degradation evaluation and maintenance of rolling bearings based on data-level fusion,adaptive health state partitioning,and state maintenance is proposed.Firstly,considering the degradation and impact in the process of bearing deterioration,the multi-sensor signals are dynamically weighted to achieve data-level fusion.Secondly,a bearing health index was established based on fast spectral correlation,Wasserstein distance,and linear rectification techniques.On this basis,by combining the Bayesian information criterion and the elbow rule,the precise division of rolling bearing health state is realized through hidden Markov model regression.Then,random forest was used to classify and predict the data to verify the validity of the proposed data fusion method and health indicator.Finally,condition-based maintenance strategy based on the fourth moment,stress-strength interference model,and Gamma process is proposed to avoid excessive detection and reduce maintenance costs.Through accelerated degradation experiments and field validation tests on the rolling bearing test data set of Xi’an Jiaotong University and FEMTO(PRONOSTIA),the accuracy and superiority of the proposed method in the prediction and maintenance of bearing health state are verified.展开更多
Outbreaks of the larch caterpillar(Dendrolimus superans)cause severe ecological and economic damage to boreal forests,underscoring the urgent need for effective monitoring and early warning systems.However,the utility...Outbreaks of the larch caterpillar(Dendrolimus superans)cause severe ecological and economic damage to boreal forests,underscoring the urgent need for effective monitoring and early warning systems.However,the utility of space-borne multispectral imagery(MSI)for this purpose is often constrained by either coarse spatial resolution or insufficient spectral bands,limiting the accurate classification of pest occurrence levels.To address this challenge,we developed an NDVI-constrained Dynamic Ridge Polynomial Neural Network(NDRPNN)to fuse Sentinel-2 MSI data with Gaofen-2(GF-2)panchromatic imagery,thereby enhancing spatial detail while preserving spectral integrity.Timeseries spectral,textural,and polarimetric features derived from Sentinel-1/2 imagery were subsequently integrated,and correlation analysis was applied to identify the most sensitive indicators.Four classification models—Random Forest,Light Gradient Boosting Machine,Stacking Ensemble,and Soft Voting Ensemble(SVE)—were evaluated for detecting infestation levels,with Shapley(SHAP)analysis employed to interpret feature contributions.The NDRPNN exhibited robust fusion performance in forested landscapes.Ensemble methods outperformed single classifiers,with the SVE model achieving the highest accuracy(overall accuracy=87.6%,Kappa=0.83).SHAP analysis identified the mean and maximum Normalized Difference Vegetation Index(NDVI),minimum Anthocyanin Reflectance Index(ARI),minimum Normalized Burn Ratio(NBR),and seasonal amplitude of Enhanced Vegetation Index(EVI)as key contributing features,highlighting the critical role of time-series vegetation indices and textural metrics in early pest detection.This study demonstrates that the integration of high-quality Sentinel-2 and GF-2 imagery with ensemble learning enables rapid and precise assessment of pest occurrence,offering a robust foundation for the early warning and scientific management of forest pests in mountain regions.展开更多
The spatial offset of bridge has a significant impact on the safety,comfort,and durability of high-speed railway(HSR)operations,so it is crucial to rapidly and effectively detect the spatial offset of operational HSR ...The spatial offset of bridge has a significant impact on the safety,comfort,and durability of high-speed railway(HSR)operations,so it is crucial to rapidly and effectively detect the spatial offset of operational HSR bridges.Drive-by monitoring of bridge uneven settlement demonstrates significant potential due to its practicality,cost-effectiveness,and efficiency.However,existing drive-by methods for detecting bridge offset have limitations such as reliance on a single data source,low detection accuracy,and the inability to identify lateral deformations of bridges.This paper proposes a novel drive-by inspection method for spatial offset of HSR bridge based on multi-source data fusion of comprehensive inspection train.Firstly,dung beetle optimizer-variational mode decomposition was employed to achieve adaptive decomposition of non-stationary dynamic signals,and explore the hidden temporal relationships in the data.Subsequently,a long short-term memory neural network was developed to achieve feature fusion of multi-source signal and accurate prediction of spatial settlement of HSR bridge.A dataset of track irregularities and CRH380A high-speed train responses was generated using a 3D train-track-bridge interaction model,and the accuracy and effectiveness of the proposed hybrid deep learning model were numerically validated.Finally,the reliability of the proposed drive-by inspection method was further validated by analyzing the actual measurement data obtained from comprehensive inspection train.The research findings indicate that the proposed approach enables rapid and accurate detection of spatial offset in HSR bridge,ensuring the long-term operational safety of HSR bridges.展开更多
The fasteners employed in the railway tracks are susceptible to defects arising from their intricate composition.Foreign objects are frequently observed on the track bed in an open environment.These two types of defec...The fasteners employed in the railway tracks are susceptible to defects arising from their intricate composition.Foreign objects are frequently observed on the track bed in an open environment.These two types of defects pose potential threats to high-speed trains,thus necessitating timely and accurate track inspection.The majority of extant automatic inspection methods are predicated on the utilization of single visible light data,and the efficacy of the algorithmic processes is influenced by complex environments.Furthermore,due to the single information dimension,the detection accuracy of defects in similar,occluded,and small object categories is low.To address the aforementioned issues,this paper proposes a track defect detectionmethod based on dynamicmulti-modal fusion and challenging object enhanced perception.First,in light of the variances in the representation dimensions ofmultimodal information,this paper proposes a dynamic weighted multi-modal feature fusion module.The fused multi-modal features are assigned weights,and thenmultiplied with the extracted single-modal features atmultiple levels,achieving adaptive adjustment of the response degree of fusion features.Second,a novel stepwise multi-scale convolution feature aggregation module is proposed for challenging objects.The proposed method employs depth separable convolution and cross-scale aggregation operations of different receptive fields to enhance feature extraction and reuse,thereby reducing the degree of progressive loss of effective information.The experimental results demonstrate the efficacy of the proposed method in comparison to eight established methods,encompassing both single-modal and multi-modal methods,as evidenced by the extensive findings within the constructed RGBD dataset.展开更多
Global sea surface temperature(SST)is a crucial climate element owing to its significant heat capacity and extensive global ocean coverage.This study evaluates an SST fusion dataset derived from Fengyun-3 satellite ob...Global sea surface temperature(SST)is a crucial climate element owing to its significant heat capacity and extensive global ocean coverage.This study evaluates an SST fusion dataset derived from Fengyun-3 satellite observations(FY-3_fusion_SST;2011-2019),with emphasis on its accuracy and potential applications in climate monitoring and op-erational services.Validation against the operational SST and sea ice analysis(OSTIA_SST)dataset revealed that the accuracy of daily FY-3_fusion_SST data was 0.03±0.55℃,while that of monthly FY-3_fusion_SST data was 0.02±0.20℃.Spatially,FY-3_fusion_SST corresponded well with OSTIA_SST in mid-latitude regions and the tropical central-eastern Pacific region,whereas significant differences were observed in the tropical regions of the Atlantic and Indian Oceans and the western Pacific warm pool.In addition,FY-3_fusion_SST and OSTIA_SST data varied in different seasons.The Niño 3.4 index calculated using the FY-3_fusion_SST data effectively characterizes the onset,termination,and intensity of El Niño-Southern Oscillation cycles,although it exhibits a systematic warm bias relative to the index calculated from the Optimal Interpolation SST data.In the tropical regions of the Atlantic and Indian Oceans,the SST indices derived from the FY-3_fusion_SST data exhibited large biases that were comparable to the amplitudes of the SST indices for certain months.These larger SST biases may be due to the influence of strong convective clouds and weak precipitation in tropical regions.In addition,these larger biases may be related to the definitions of the SST indices.When the areas defined by the indices are small,the indices are more prone to large deviations,limiting the application of the FY-3_fusion_SST dataset in climate monitoring and prediction operations.展开更多
The era of big data has profoundly transformed mechanics research,with data-driven approaches playing a vital role in modeling and optimization.This study focuses on tunnel boring machine(TBM),where the thrust-torque ...The era of big data has profoundly transformed mechanics research,with data-driven approaches playing a vital role in modeling and optimization.This study focuses on tunnel boring machine(TBM),where the thrust-torque ratio is a key determinant of their tunneling energy efficiency.However,due to the complexity of experiments and the testing requirements,obtaining sufficient high-quality data under varying geological conditions remains a major challenge in optimizing the tunneling energy efficiency of TBM.To address this,multi-cutter rotary cutting machine experiments and numerical simulations were conducted on 22 different rock types.Comprehensive datasets of normal and rolling forces were systematically collected.Using specific energy(SE)as the rock-breaking efficiency metric,we integrated physical and numerical data through a CatBoost-based fusion framework.The predictive model was initially trained on simulation data to capture the relationships among penetration,uniaxial compressive strength,tensile strength,and SE,and was subsequently fine-tuned with experimental data to develop the final fused model.Compared to models trained solely on experimental or simulated data,the fused model reduced RMSE by 37.1%and 58.6%,respectively,and improved R2by 19.0%and 44.6%,thereby enhancing both prediction accuracy and generalization capability.Furthermore,Bayesian optimization was employed to minimize SE and identify the optimal penetration.The results indicate that as rock strength increases,the optimal penetration decreases,while the corresponding minimal SE increases.These findings provide theoretical and engineering insights for improving TBM energy efficiency and parameter optimization,while establishing a robust data fusion framework for mechanical data analysis.展开更多
Continuous monitoring of high spatiotemporal resolution evapotranspiration(ET)is crucial for accurately assessing water resource management and irrigation efficiency at both global and regional scales.However,constrai...Continuous monitoring of high spatiotemporal resolution evapotranspiration(ET)is crucial for accurately assessing water resource management and irrigation efficiency at both global and regional scales.However,constraints such as satellite image transit time and cloud contamination can inhibit a single satellite to provide fine spatial resolution and continuous daily sequences of remote sensing data.In this study,we employed a coupled multi-scale fusion and interpolation method evapotranspiration model(CFIEM)framework integrated the remote sensing ET models,data fusion models,and the HANTS-GEE interpolation method to generate highresolution surface feature parameters and hydrological variables.Landsat remote sensing parameters and MODIS-Landsat fusion data were adopted to simulate daily regional ET,and the simulated results were subsequently validated.The results revealed the strong consistency between the simulated CFIEM-ET and Landsat-ET values and measured data,with the R2 values of 0.83 and 0.73.Among the simulations across the sand dunes,grasslands,rice,and maize ecosystems,maize exhibited the most accurate simulation(R2?0.84),while the sand dunes demonstrated certain deviation(R2?0.72).The geographical detector analysis identified the net radiation(Rn)as the primary driver of multi-year ET variation,followed by the land surface temperature(LST)and leaf area index(LAI).This study sheds light on the interannual ET variation and its driving mechanisms in arid regions,laying a theoretical foundation and offering technical support for regional water resource management and desertification control.展开更多
Background and Aims:Metabolic dysfunction-associated fatty liver disease(MAFLD)represents a predominant cause of chronic liver disease,underscoring the demand for accessible,non-invasive diagnostic tools.Tongue diagno...Background and Aims:Metabolic dysfunction-associated fatty liver disease(MAFLD)represents a predominant cause of chronic liver disease,underscoring the demand for accessible,non-invasive diagnostic tools.Tongue diagnosis in Traditional Chinese Medicine provides a distinctive perspective on systemic health,though it remains largely subjective.This study aimed to develop an interpretable multimodal deep learning model for MAFLD screening by integrating quantitative tongue image features with routine clinical data.Methods:From 904 screened candidates,477 subjects(157 healthy,320 MAFLD)were included and randomly allocated to training,validation,and test sets in an 8:1:1 ratio.All participants underwent standardized tongue imaging(International Commission on Illumination L*a*b color features)and comprehensive clinical evaluation.We constructed a dual-stream deep learning model,combining a ConvNeXt-Tiny network for tongue images and a multilayer perceptron for clinical variables.Feature fusion was achieved via a Dynamic Affine Feature Transformation module,and the model was trained using weighted cross-entropy loss.Results:MAFLD patients showed significant metabolic abnormalities compared to healthy controls.A progressive decrease in tongue yellowness(b* value)was observed with advancing fibrosis.On an independent test set(n=48),the multimodal model achieved 97.92%accuracy,Quadratic Weighted Kappa of 0.9538,and 96.88%sensitivity,and 100%specificity,outperforming single-modality and serological models.Interpretability analyses confirmed the model’s focus on clinically relevant tongue regions and key metabolic drivers.Conclusions:We developed an accurate and interpretable multimodal model that synergizes tongue image features with metabolic indicators for MAFLD screening.This approach presents a promising,low-cost tool potentially well-suited for resource-limited settings.展开更多
Accurate characterization of regional seismic activity is crucial for assessing earthquake hazards.Seismic activity depends on multiple factors,including tectonic loading,fault geometry and distribution,crustal and ma...Accurate characterization of regional seismic activity is crucial for assessing earthquake hazards.Seismic activity depends on multiple factors,including tectonic loading,fault geometry and distribution,crustal and mantle structures,local topography,physical properties,and global environmental changes.Human influences,such as reservoir impoundment,enhanced geothermal systems,and shale gas extraction,further complicate these relationships.Establishing an integrated theoretical and methodological framework through data to analyze these natural and anthropogenic factors represents a frontier challenge in contemporary seismology and geodynamics.This study introduces three novel visualization methods for earthquake catalogs,efficiently capturing the complex relationships between magnitude,frequency,seismic origin time,and epicentral location.Utilizing these methods with comprehensive heterogeneous geophysical datasets from the Sichuan-Yunnan region,including over 420,000 earthquake records,160 three-dimensional active fault datasets,high-resolution topography,Moho depth,community velocity models,and crustal deformation data,the seismic characteristics of the region over the past 50 years were systematically analyzed.Results indicate that:(1)High seismic activity and hazard areas in the Sichuan-Yunnan region are primarily concentrated near deep major faults,block boundaries,and brittle transition zones with distinct low-and high-velocity anomalies,showing clear spatial banding,temporal clustering,and cyclicity;(2)Fault segments such as Longmenshan,Lijiang-Xiaojinhe,and Nujiang-Irrawaddy likely facilitate internal material exchange within the Qinghai-Xizang Plateau.Significant crustal thickening in their northwestern sections corresponds with lower seismic activity;(3)Crustal strain varies notably along the Xianshuihe-Anninghe-Zemuhe-Xiaojiang and Longmenshan fault zones,which delineate the boundary between regions of high-and low-velocity ratio anomalies.These zones host the majority of the regional earthquakes,requiring intensified monitoring due to frequent events despite moderate mainshock magnitudes.Overall,the proposed methodology provides a new reference for deepening our understanding of regional seismicity and developing an improved earthquake visualization technique.展开更多
With the rapid development of Industrial 4.0 and Industrial Internet of Things,the data collection with multisource has significantly improved.How to effectively fuse these data for various engineering applications is...With the rapid development of Industrial 4.0 and Industrial Internet of Things,the data collection with multisource has significantly improved.How to effectively fuse these data for various engineering applications is still an open and challenge issue.To this end,we propose the canonical correlation guided deep neural network(CCDNN),a novel deep learning architecture,to learn a correlated representation for multi-source data fusion.Unlike the linear canonical correlation analysis(CCA),kernel CCA and deep CCA,in the proposed method,the optimization formulation is not restricted to maximize correlation,instead we make canonical correlation as a constraint,which preserves the correlated representation learning ability and focuses more on the engineering tasks endowed by optimization formulation,such as reconstruction,classification and prediction.Furthermore,to reduce the redundancy induced by correlation,a redundancy filter is designed.We illustrate its data fusion ability via correlated representation learning and superior performance on various engineering tasks.In experiments on MNIST dataset,the results show that CCDNN has better reconstruction performance in terms of mean squared error and mean absolute error than deep CCA and deep canonically correlated autoencoders(DCCAE).Also,we present the application of the proposed network to industrial fault diagnosis and remaining useful life cases for the classification and prediction tasks accordingly.The proposed method demonstrates approving performance in both tasks when compared to existing methods.Extension of CCDNN to much more deeper with the aid of residual connection is also presented in Appendix.展开更多
High-resolution soil moisture(SM)data are critical for drought monitoring and flood forecasting.This paper describes the establishment of an interpretable machine learning(ML)-based framework for SM data fusion and ge...High-resolution soil moisture(SM)data are critical for drought monitoring and flood forecasting.This paper describes the establishment of an interpretable machine learning(ML)-based framework for SM data fusion and generates a dailyscale,1-km resolution,surface SM(0–10 cm)dataset over China(2000–2025).Four state-of-the-art ML models—Random Forest,XGBoost,LightGBM,and CatBoost—were trained based on in situ SM data from 2371 automatic observation stations across China.Model performance was optimized via Recursive Feature Elimination(RFE)and automated hyperparameter tuning using Optuna,while SHapley Additive exPlanations(SHAP)provided mechanistic interpretability of the ML models.The key findings of this study are as follows:(1)The fusion model primarily enhances SM estimation,exhibiting lower root-mean-square error than CLDAS(China Meteorological Administration Land Data Assimilation System)SM,despite marginally weaker daily temporal correlation;(2)RFE eliminated 57%of features while preserving predictive accuracy;(3)SHAP analysis revealed high-accuracy SM inputs as the most influential predictors,followed by static(terrain and soil properties)and meteorological variables.The SM fusion method developed in this study is transferable to multi-source satellite SM fusion and downscaling.The dataset is publicly available at http://gffzzd3cc09b8251d45dfsnwfqfv9cpqnx6ko5.ffgz.tsg.suse.edu.cn/10.11888/Terre.tpdc.302923.展开更多
To address the challenges of dusty,foggy and other complex construction site environments leading to the failure of visible light imaging and difficulties in small target detection,as well as the high resource consump...To address the challenges of dusty,foggy and other complex construction site environments leading to the failure of visible light imaging and difficulties in small target detection,as well as the high resource consumption hindering model deployment,an enhanced and lightweight algorithm is proposed.This algorithm employs a hybrid architecture,integrating red green blue(RGB)(visible light)and thermal infrared(RGBT)multi-modal images through a fusion framework based on you only look once(YOLO)version 8 and Mamba-Transformer(MT).We refer to this integrated model as YOLOv8-RGBT-MT.In terms of network improvements,a frequency enhancement module is first employed to enhance visible light and infrared images.And then,a module integrating Mamba and Transformer components is designed to replace base convolutional blocks in the backbone network,thereby expanding the receptive field of the model and improving feature extraction in complex backgrounds.Finally,a multi-modal feature fusion mechanism is introduced,through which complementary information from visible and infrared images is effectively integrated via an adaptive weighting strategy,so that both the detection accuracy and robustness for small targets are enhanced.Experimental results demonstrate that,compared to YOLOv8-RGBT,the enhanced algorithm achieves an improvement of 18.7%in mAP50,while reducing the number of inference time by 79.7%.展开更多
Indeed,a range of systems in the environment requires timely,spatially explicit,and credible information to support its environmental decision-making,but no one observing system can give the complete and reliable meas...Indeed,a range of systems in the environment requires timely,spatially explicit,and credible information to support its environmental decision-making,but no one observing system can give the complete and reliable measures of the Earth system across scales.This review summarizes how the realization of the Compute the Planet is underway in the form of machine learning,remote sensing,and sensor data fusion to generate decision-ready environmental insights.We use the application-first approach,which considers remote sensing,in situ and Internet of Things(IoT)sensing,and physics-based models as complementary streams of evidence with similar strengths and failures.We look critically at how an integrated system can convert heterogeneous observations to action products across three high impact application areas:atmosphere and air quality,water–land–ecosystem dynamics,and hazards.Rapid-response situational awareness,ecosystem condition metrics,drought and flood indicators,exposure maps,and hazard/extreme indicators are key products.The integrated systems to environment interface in three high impact application areas:atmosphere and air quality,water-land-ecosystem dynamics,and hazard Examine Our operational requirements can often determine real-life value such as latency,time stability,smooth degradation in the presence of missing or degraded inputs,and calibrated uncertainty usable in thresholdbased decisions.These pitfalls are common across fields:mismatch in the scale between a point sensor and a gridded product,objectives on proxies in remotely sensed measurements,domain shift in the extremes and changing baselines,and evaluation aspects,which overestimate generalization because of spatiotemporal autocorrelation.Based on these lessons,we present cross-domain proposals for strong validation,uncertainty quantification,provenance,and versioning,as well as fair performance evaluation.We conclude that the next era of environmental intelligence will see a reduction in average accuracy improvement and an increase in terms of robustness,transparency,and operational responsibility,thus allowing the integrated environmental intelligence system to be deployed,which may be relied on to monitor human health,resource allocation,and survival in a more climate-adapted world.展开更多
To address the difficulties in fusing multi-mode sensor data for complex industrial machinery, an adaptive deep coupling convolutional auto-encoder (ADCCAE) fusion method was proposed. First, the multi-mode features e...To address the difficulties in fusing multi-mode sensor data for complex industrial machinery, an adaptive deep coupling convolutional auto-encoder (ADCCAE) fusion method was proposed. First, the multi-mode features extracted synchronously by the CCAE were stacked and fed to the multi-channel convolution layers for fusion. Then, the fused data was passed to all connection layers for compression and fed to the Softmax module for classification. Finally, the coupling loss function coefficients and the network parameters were optimized through an adaptive approach using the gray wolf optimization (GWO) algorithm. Experimental comparisons showed that the proposed ADCCAE fusion model was superior to existing models for multi-mode data fusion.展开更多
As a core geospatial data product, Digital Orthophoto Map (DOM) plays a key role in many fields such as urban planning, disaster assessment, engineering surveying and mapping, and topographic mapping. With the increas...As a core geospatial data product, Digital Orthophoto Map (DOM) plays a key role in many fields such as urban planning, disaster assessment, engineering surveying and mapping, and topographic mapping. With the increasing requirements of various industries for the timeliness and accuracy of DOM, the traditional single-data-source update method can hardly meet the actual needs. Aiming at this problem, this paper deeply studies the rapid update and accuracy improvement methods of DOM driven by multi-source aerial data fusion. The research shows that multi-source aerial data fusion technology can effectively improve the efficiency and accuracy of DOM update, providing reliable support for the efficient application of geospatial data.展开更多
Near real-time maize phenology monitoring is crucial for field management,cropping system adjustments,and yield estimation.Most phenological monitoring methods are post-seasonal and heavily rely on high-frequency time...Near real-time maize phenology monitoring is crucial for field management,cropping system adjustments,and yield estimation.Most phenological monitoring methods are post-seasonal and heavily rely on high-frequency time-series data.These methods are not applicable on the unmanned aerial vehicle(UAV)platform due to the high cost of acquiring time-series UAV images and the shortage of UAV-based phenological monitoring methods.To address these challenges,we employed the Synthetic Minority Oversampling Technique(SMOTE)for sample augmentation,aiming to resolve the small sample modelling problem.Moreover,we utilized enhanced"separation"and"compactness"feature selection methods to identify input features from multiple data sources.In this process,we incorporated dynamic multi-source data fusion strategies,involving Vegetation index(VI),Color index(CI),and Texture features(TF).A two-stage neural network that combines Convolutional Neural Network(CNN)and Long Short-Term Memory Network(LSTM)is proposed to identify maize phenological stages(including sowing,seedling,jointing,trumpet,tasseling,maturity,and harvesting)on UAV platforms.The results indicate that the dataset generated by SMOTE closely resembles the measured dataset.Among dynamic data fusion strategies,the VI-TF combination proves to be most effective,with CI-TF and VI-CI combinations following behind.Notably,as more data sources are integrated,the model's demand for input features experiences a significant decline.In particular,the CNN-LSTM model,based on the fusion of three data sources,exhibited remarkable reliability when validating the three datasets.For Dataset 1(Beijing Xiaotangshan,2023:Data from 12 UAV Flight Missions),the model achieved an overall accuracy(OA)of 86.53%.Additionally,its precision(Pre),recall(Rec),F1 score(F1),false acceptance rate(FAR),and false rejection rate(FRR)were 0.89,0.89,0.87,0.11,and 0.11,respectively.The model also showed strong generalizability in Dataset 2(Beijing Xiaotangshan,2023:Data from 6 UAV Flight Missions)and Dataset 3(Beijing Xiaotangshan,2022:Data from 4 UAV Flight Missions),with OAs of 89.4%and 85%,respectively.Meanwhile,the model has a low demand for input featu res,requiring only 54.55%(99 of all featu res).The findings of this study not only offer novel insights into near real-time crop phenology monitoring,but also provide technical support for agricultural field management and cropping system adaptation.展开更多
This paper addresses the accuracy and timeliness limitations of traditional comprehensive prediction methods by proposing an approach of decision-level fusion of multisource data.A risk prediction indicator system was...This paper addresses the accuracy and timeliness limitations of traditional comprehensive prediction methods by proposing an approach of decision-level fusion of multisource data.A risk prediction indicator system was established for water and mud inrush in tunnels by analyzing advanced prediction data for specifi c tunnel segments.Additionally,the indicator weights were determined using the analytic hierarchy process combined with the Huber weighting method.Subsequently,a multisource data decision-layer fusion algorithm was utilized to generate fused imaging results for tunnel water and mud inrush risk predictions.Meanwhile,risk analysis was performed for different tunnel sections to achieve spatial and temporal complementarity within the indicator system and optimize redundant information.Finally,model feasibility was validated using the CZ Project Sejila Mountain Tunnel segment as a case study,yielding favorable risk prediction results and enabling effi cient information fusion and support for construction decision-making.展开更多
To address the challenge of missing modal information in entity alignment and to mitigate information loss or bias arising frommodal heterogeneity during fusion,while also capturing shared information acrossmodalities...To address the challenge of missing modal information in entity alignment and to mitigate information loss or bias arising frommodal heterogeneity during fusion,while also capturing shared information acrossmodalities,this paper proposes a Multi-modal Pre-synergistic Entity Alignmentmodel based on Cross-modalMutual Information Strategy Optimization(MPSEA).The model first employs independent encoders to process multi-modal features,including text,images,and numerical values.Next,a multi-modal pre-synergistic fusion mechanism integrates graph structural and visual modal features into the textual modality as preparatory information.This pre-fusion strategy enables unified perception of heterogeneous modalities at the model’s initial stage,reducing discrepancies during the fusion process.Finally,using cross-modal deep perception reinforcement learning,the model achieves adaptive multilevel feature fusion between modalities,supporting learningmore effective alignment strategies.Extensive experiments on multiple public datasets show that the MPSEA method achieves gains of up to 7% in Hits@1 and 8.2% in MRR on the FBDB15K dataset,and up to 9.1% in Hits@1 and 7.7% in MRR on the FBYG15K dataset,compared to existing state-of-the-art methods.These results confirm the effectiveness of the proposed model.展开更多
基金funded by the Natural Science Foundation of Chongqing Municipality,grant number CSTB2022NSCQ-MSX0503.
摘要Gait recognition is a key biometric for long-distance identification,yet its performance is severely degraded by real-world challenges such as varying clothing,carrying conditions,and changing viewpoints.While combining silhouette and skeleton data is a promising direction,effectively fusing these heterogeneous modalities and adaptively weighting their contributions in response to diverse conditions remains a central problem.This paper introduces GaitMAFF,a novelMulti-modal Adaptive Feature Fusion Network,to address this challenge.Our approach first transforms discrete skeleton joints into a dense SkeletonMap representation to align with silhouettes,then employs an attention-based module to dynamically learn the fusion weights between the two modalities.These fused features are processed by a powerful spatio-temporal backbone withWeighted Global-Local Feature FusionModules(WFFM)to learn a discriminative representation.Extensive experiments on the challenging CCPG and Gait3D datasets show that GaitMAFF achieves state-of-the-art performance,with an average Rank-1 accuracy of 84.6%on CCPG and 58.7%on Gait3D.These results demonstrate that our adaptive fusion strategy effectively integrates complementary multimodal information,significantly enhancing gait recognition robustness and accuracy in complex scenes and providing a practical solution for real-world applications.
基金supported by the Key Program of Natural Science Foundation of Tianjin(Grant No.21JCZDJC00770)the Tianjin Metrology Technology Project(Grant No.2024TJMT049).
摘要The traditional method of performance degradation prediction and maintenance of rolling bearings only considers a single sensor signal,which makes it difficult to automatically partition degradation stages and prone to over-detection.A new method of performance degradation evaluation and maintenance of rolling bearings based on data-level fusion,adaptive health state partitioning,and state maintenance is proposed.Firstly,considering the degradation and impact in the process of bearing deterioration,the multi-sensor signals are dynamically weighted to achieve data-level fusion.Secondly,a bearing health index was established based on fast spectral correlation,Wasserstein distance,and linear rectification techniques.On this basis,by combining the Bayesian information criterion and the elbow rule,the precise division of rolling bearing health state is realized through hidden Markov model regression.Then,random forest was used to classify and predict the data to verify the validity of the proposed data fusion method and health indicator.Finally,condition-based maintenance strategy based on the fourth moment,stress-strength interference model,and Gamma process is proposed to avoid excessive detection and reduce maintenance costs.Through accelerated degradation experiments and field validation tests on the rolling bearing test data set of Xi’an Jiaotong University and FEMTO(PRONOSTIA),the accuracy and superiority of the proposed method in the prediction and maintenance of bearing health state are verified.
基金supported in part by the National Natural Science Foundation of China under Grant 42171407 and Grant 42077242in part by the Key Program of National Natural Science Foundation of China under Grant 42330607。
摘要Outbreaks of the larch caterpillar(Dendrolimus superans)cause severe ecological and economic damage to boreal forests,underscoring the urgent need for effective monitoring and early warning systems.However,the utility of space-borne multispectral imagery(MSI)for this purpose is often constrained by either coarse spatial resolution or insufficient spectral bands,limiting the accurate classification of pest occurrence levels.To address this challenge,we developed an NDVI-constrained Dynamic Ridge Polynomial Neural Network(NDRPNN)to fuse Sentinel-2 MSI data with Gaofen-2(GF-2)panchromatic imagery,thereby enhancing spatial detail while preserving spectral integrity.Timeseries spectral,textural,and polarimetric features derived from Sentinel-1/2 imagery were subsequently integrated,and correlation analysis was applied to identify the most sensitive indicators.Four classification models—Random Forest,Light Gradient Boosting Machine,Stacking Ensemble,and Soft Voting Ensemble(SVE)—were evaluated for detecting infestation levels,with Shapley(SHAP)analysis employed to interpret feature contributions.The NDRPNN exhibited robust fusion performance in forested landscapes.Ensemble methods outperformed single classifiers,with the SVE model achieving the highest accuracy(overall accuracy=87.6%,Kappa=0.83).SHAP analysis identified the mean and maximum Normalized Difference Vegetation Index(NDVI),minimum Anthocyanin Reflectance Index(ARI),minimum Normalized Burn Ratio(NBR),and seasonal amplitude of Enhanced Vegetation Index(EVI)as key contributing features,highlighting the critical role of time-series vegetation indices and textural metrics in early pest detection.This study demonstrates that the integration of high-quality Sentinel-2 and GF-2 imagery with ensemble learning enables rapid and precise assessment of pest occurrence,offering a robust foundation for the early warning and scientific management of forest pests in mountain regions.
基金sponsored by the National Natural Science Foundation of China(Grant No.52178100).
摘要The spatial offset of bridge has a significant impact on the safety,comfort,and durability of high-speed railway(HSR)operations,so it is crucial to rapidly and effectively detect the spatial offset of operational HSR bridges.Drive-by monitoring of bridge uneven settlement demonstrates significant potential due to its practicality,cost-effectiveness,and efficiency.However,existing drive-by methods for detecting bridge offset have limitations such as reliance on a single data source,low detection accuracy,and the inability to identify lateral deformations of bridges.This paper proposes a novel drive-by inspection method for spatial offset of HSR bridge based on multi-source data fusion of comprehensive inspection train.Firstly,dung beetle optimizer-variational mode decomposition was employed to achieve adaptive decomposition of non-stationary dynamic signals,and explore the hidden temporal relationships in the data.Subsequently,a long short-term memory neural network was developed to achieve feature fusion of multi-source signal and accurate prediction of spatial settlement of HSR bridge.A dataset of track irregularities and CRH380A high-speed train responses was generated using a 3D train-track-bridge interaction model,and the accuracy and effectiveness of the proposed hybrid deep learning model were numerically validated.Finally,the reliability of the proposed drive-by inspection method was further validated by analyzing the actual measurement data obtained from comprehensive inspection train.The research findings indicate that the proposed approach enables rapid and accurate detection of spatial offset in HSR bridge,ensuring the long-term operational safety of HSR bridges.
基金funded by Beijing Natural Science Foundation,grant number L241078.
摘要The fasteners employed in the railway tracks are susceptible to defects arising from their intricate composition.Foreign objects are frequently observed on the track bed in an open environment.These two types of defects pose potential threats to high-speed trains,thus necessitating timely and accurate track inspection.The majority of extant automatic inspection methods are predicated on the utilization of single visible light data,and the efficacy of the algorithmic processes is influenced by complex environments.Furthermore,due to the single information dimension,the detection accuracy of defects in similar,occluded,and small object categories is low.To address the aforementioned issues,this paper proposes a track defect detectionmethod based on dynamicmulti-modal fusion and challenging object enhanced perception.First,in light of the variances in the representation dimensions ofmultimodal information,this paper proposes a dynamic weighted multi-modal feature fusion module.The fused multi-modal features are assigned weights,and thenmultiplied with the extracted single-modal features atmultiple levels,achieving adaptive adjustment of the response degree of fusion features.Second,a novel stepwise multi-scale convolution feature aggregation module is proposed for challenging objects.The proposed method employs depth separable convolution and cross-scale aggregation operations of different receptive fields to enhance feature extraction and reuse,thereby reducing the degree of progressive loss of effective information.The experimental results demonstrate the efficacy of the proposed method in comparison to eight established methods,encompassing both single-modal and multi-modal methods,as evidenced by the extensive findings within the constructed RGBD dataset.
基金National Natural Science Foundation of China(U2442216)Civilian Space Programme of China(D040305)。
摘要Global sea surface temperature(SST)is a crucial climate element owing to its significant heat capacity and extensive global ocean coverage.This study evaluates an SST fusion dataset derived from Fengyun-3 satellite observations(FY-3_fusion_SST;2011-2019),with emphasis on its accuracy and potential applications in climate monitoring and op-erational services.Validation against the operational SST and sea ice analysis(OSTIA_SST)dataset revealed that the accuracy of daily FY-3_fusion_SST data was 0.03±0.55℃,while that of monthly FY-3_fusion_SST data was 0.02±0.20℃.Spatially,FY-3_fusion_SST corresponded well with OSTIA_SST in mid-latitude regions and the tropical central-eastern Pacific region,whereas significant differences were observed in the tropical regions of the Atlantic and Indian Oceans and the western Pacific warm pool.In addition,FY-3_fusion_SST and OSTIA_SST data varied in different seasons.The Niño 3.4 index calculated using the FY-3_fusion_SST data effectively characterizes the onset,termination,and intensity of El Niño-Southern Oscillation cycles,although it exhibits a systematic warm bias relative to the index calculated from the Optimal Interpolation SST data.In the tropical regions of the Atlantic and Indian Oceans,the SST indices derived from the FY-3_fusion_SST data exhibited large biases that were comparable to the amplitudes of the SST indices for certain months.These larger SST biases may be due to the influence of strong convective clouds and weak precipitation in tropical regions.In addition,these larger biases may be related to the definitions of the SST indices.When the areas defined by the indices are small,the indices are more prone to large deviations,limiting the application of the FY-3_fusion_SST dataset in climate monitoring and prediction operations.
基金National Natural Science Foundation of China,12021002,Qian Zhang,12372186,QianZhang,Emerging Frontiers Cultivation Program of Tianjin University Interdisciplinary Center.
摘要The era of big data has profoundly transformed mechanics research,with data-driven approaches playing a vital role in modeling and optimization.This study focuses on tunnel boring machine(TBM),where the thrust-torque ratio is a key determinant of their tunneling energy efficiency.However,due to the complexity of experiments and the testing requirements,obtaining sufficient high-quality data under varying geological conditions remains a major challenge in optimizing the tunneling energy efficiency of TBM.To address this,multi-cutter rotary cutting machine experiments and numerical simulations were conducted on 22 different rock types.Comprehensive datasets of normal and rolling forces were systematically collected.Using specific energy(SE)as the rock-breaking efficiency metric,we integrated physical and numerical data through a CatBoost-based fusion framework.The predictive model was initially trained on simulation data to capture the relationships among penetration,uniaxial compressive strength,tensile strength,and SE,and was subsequently fine-tuned with experimental data to develop the final fused model.Compared to models trained solely on experimental or simulated data,the fused model reduced RMSE by 37.1%and 58.6%,respectively,and improved R2by 19.0%and 44.6%,thereby enhancing both prediction accuracy and generalization capability.Furthermore,Bayesian optimization was employed to minimize SE and identify the optimal penetration.The results indicate that as rock strength increases,the optimal penetration decreases,while the corresponding minimal SE increases.These findings provide theoretical and engineering insights for improving TBM energy efficiency and parameter optimization,while establishing a robust data fusion framework for mechanical data analysis.
基金supported by the National Natural Science Foundation of China(Grant numbers 52439004,U2243234,52309021,52109022,52169002)the Inner Mongolia Autonomous Region Science and Technology Leading Talent Team(Grant number 2022LJRC0007)+3 种基金the Ministry of Education of China Innovative Research Team(Grant number IRT_17R60)the Chinese Ministry of Science and Technology Innovative Research Team in Priority Areas(Grant number 2015RA4013)the Inner Mongolia Agricultural University Basic Research Project(Grant numbers BR221012 and BR221204)the First-class Academic Subjects Special Research Project of the Education Department of Inner Mongolia Autonomous Region(Grant numbers YLXKZX-NND-010 and YLXKZXNND-028).
摘要Continuous monitoring of high spatiotemporal resolution evapotranspiration(ET)is crucial for accurately assessing water resource management and irrigation efficiency at both global and regional scales.However,constraints such as satellite image transit time and cloud contamination can inhibit a single satellite to provide fine spatial resolution and continuous daily sequences of remote sensing data.In this study,we employed a coupled multi-scale fusion and interpolation method evapotranspiration model(CFIEM)framework integrated the remote sensing ET models,data fusion models,and the HANTS-GEE interpolation method to generate highresolution surface feature parameters and hydrological variables.Landsat remote sensing parameters and MODIS-Landsat fusion data were adopted to simulate daily regional ET,and the simulated results were subsequently validated.The results revealed the strong consistency between the simulated CFIEM-ET and Landsat-ET values and measured data,with the R2 values of 0.83 and 0.73.Among the simulations across the sand dunes,grasslands,rice,and maize ecosystems,maize exhibited the most accurate simulation(R2?0.84),while the sand dunes demonstrated certain deviation(R2?0.72).The geographical detector analysis identified the net radiation(Rn)as the primary driver of multi-year ET variation,followed by the land surface temperature(LST)and leaf area index(LAI).This study sheds light on the interannual ET variation and its driving mechanisms in arid regions,laying a theoretical foundation and offering technical support for regional water resource management and desertification control.
基金supported by the National Natural Science Foundation of China(No.82274352 to XDL)the Hubei Provincial Key Research and Development Program(No.2024BCB 038 to XDL)+1 种基金City University of Hong Kong(7006082,7020073,9609332,9609333,9678292,7020002)and the Research Grants Council(9048206,8799020 to BLK).
摘要Background and Aims:Metabolic dysfunction-associated fatty liver disease(MAFLD)represents a predominant cause of chronic liver disease,underscoring the demand for accessible,non-invasive diagnostic tools.Tongue diagnosis in Traditional Chinese Medicine provides a distinctive perspective on systemic health,though it remains largely subjective.This study aimed to develop an interpretable multimodal deep learning model for MAFLD screening by integrating quantitative tongue image features with routine clinical data.Methods:From 904 screened candidates,477 subjects(157 healthy,320 MAFLD)were included and randomly allocated to training,validation,and test sets in an 8:1:1 ratio.All participants underwent standardized tongue imaging(International Commission on Illumination L*a*b color features)and comprehensive clinical evaluation.We constructed a dual-stream deep learning model,combining a ConvNeXt-Tiny network for tongue images and a multilayer perceptron for clinical variables.Feature fusion was achieved via a Dynamic Affine Feature Transformation module,and the model was trained using weighted cross-entropy loss.Results:MAFLD patients showed significant metabolic abnormalities compared to healthy controls.A progressive decrease in tongue yellowness(b* value)was observed with advancing fibrosis.On an independent test set(n=48),the multimodal model achieved 97.92%accuracy,Quadratic Weighted Kappa of 0.9538,and 96.88%sensitivity,and 100%specificity,outperforming single-modality and serological models.Interpretability analyses confirmed the model’s focus on clinically relevant tongue regions and key metabolic drivers.Conclusions:We developed an accurate and interpretable multimodal model that synergizes tongue image features with metabolic indicators for MAFLD screening.This approach presents a promising,low-cost tool potentially well-suited for resource-limited settings.
基金funded by the National Natural Science Foundation of China(No.U2239205)the National Key Research and Development Programme of China(Nos.2020YFA0713400 and 2020YFA0713401).
摘要Accurate characterization of regional seismic activity is crucial for assessing earthquake hazards.Seismic activity depends on multiple factors,including tectonic loading,fault geometry and distribution,crustal and mantle structures,local topography,physical properties,and global environmental changes.Human influences,such as reservoir impoundment,enhanced geothermal systems,and shale gas extraction,further complicate these relationships.Establishing an integrated theoretical and methodological framework through data to analyze these natural and anthropogenic factors represents a frontier challenge in contemporary seismology and geodynamics.This study introduces three novel visualization methods for earthquake catalogs,efficiently capturing the complex relationships between magnitude,frequency,seismic origin time,and epicentral location.Utilizing these methods with comprehensive heterogeneous geophysical datasets from the Sichuan-Yunnan region,including over 420,000 earthquake records,160 three-dimensional active fault datasets,high-resolution topography,Moho depth,community velocity models,and crustal deformation data,the seismic characteristics of the region over the past 50 years were systematically analyzed.Results indicate that:(1)High seismic activity and hazard areas in the Sichuan-Yunnan region are primarily concentrated near deep major faults,block boundaries,and brittle transition zones with distinct low-and high-velocity anomalies,showing clear spatial banding,temporal clustering,and cyclicity;(2)Fault segments such as Longmenshan,Lijiang-Xiaojinhe,and Nujiang-Irrawaddy likely facilitate internal material exchange within the Qinghai-Xizang Plateau.Significant crustal thickening in their northwestern sections corresponds with lower seismic activity;(3)Crustal strain varies notably along the Xianshuihe-Anninghe-Zemuhe-Xiaojiang and Longmenshan fault zones,which delineate the boundary between regions of high-and low-velocity ratio anomalies.These zones host the majority of the regional earthquakes,requiring intensified monitoring due to frequent events despite moderate mainshock magnitudes.Overall,the proposed methodology provides a new reference for deepening our understanding of regional seismicity and developing an improved earthquake visualization technique.
基金supported in part by the National Natural Science Foundation of China(62173349)the Natural Science Foundation of Hunan Province(2025JJ10007)+1 种基金the Natural Science Foundation of Hunan Province(2022JJ20076)the Science and Technology Innovation Program of Hunan Province(2022RC1090)。
摘要With the rapid development of Industrial 4.0 and Industrial Internet of Things,the data collection with multisource has significantly improved.How to effectively fuse these data for various engineering applications is still an open and challenge issue.To this end,we propose the canonical correlation guided deep neural network(CCDNN),a novel deep learning architecture,to learn a correlated representation for multi-source data fusion.Unlike the linear canonical correlation analysis(CCA),kernel CCA and deep CCA,in the proposed method,the optimization formulation is not restricted to maximize correlation,instead we make canonical correlation as a constraint,which preserves the correlated representation learning ability and focuses more on the engineering tasks endowed by optimization formulation,such as reconstruction,classification and prediction.Furthermore,to reduce the redundancy induced by correlation,a redundancy filter is designed.We illustrate its data fusion ability via correlated representation learning and superior performance on various engineering tasks.In experiments on MNIST dataset,the results show that CCDNN has better reconstruction performance in terms of mean squared error and mean absolute error than deep CCA and deep canonically correlated autoencoders(DCCAE).Also,we present the application of the proposed network to industrial fault diagnosis and remaining useful life cases for the classification and prediction tasks accordingly.The proposed method demonstrates approving performance in both tasks when compared to existing methods.Extension of CCDNN to much more deeper with the aid of residual connection is also presented in Appendix.
基金supported by the National Natural Science Foundation of China(Grant No.U2342218)Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China(Grant No.JYB2025XDXM907)+4 种基金the National Key R&D Program of China(Grant No.2024YFC3012401)the GeoX Interdisciplinary Project(Grant No.20250304)of the Frontiers Science Center for Critical Earth Material Cycling,Nanjing UniversityKey Laboratory of Radar Meteorology,China Meteorology Administration,Nanjing,ChinaJiangsu Collaborative Innovation Center for Climate Changethe High-Performance Computing Center of Nanjing University.
摘要High-resolution soil moisture(SM)data are critical for drought monitoring and flood forecasting.This paper describes the establishment of an interpretable machine learning(ML)-based framework for SM data fusion and generates a dailyscale,1-km resolution,surface SM(0–10 cm)dataset over China(2000–2025).Four state-of-the-art ML models—Random Forest,XGBoost,LightGBM,and CatBoost—were trained based on in situ SM data from 2371 automatic observation stations across China.Model performance was optimized via Recursive Feature Elimination(RFE)and automated hyperparameter tuning using Optuna,while SHapley Additive exPlanations(SHAP)provided mechanistic interpretability of the ML models.The key findings of this study are as follows:(1)The fusion model primarily enhances SM estimation,exhibiting lower root-mean-square error than CLDAS(China Meteorological Administration Land Data Assimilation System)SM,despite marginally weaker daily temporal correlation;(2)RFE eliminated 57%of features while preserving predictive accuracy;(3)SHAP analysis revealed high-accuracy SM inputs as the most influential predictors,followed by static(terrain and soil properties)and meteorological variables.The SM fusion method developed in this study is transferable to multi-source satellite SM fusion and downscaling.The dataset is publicly available at http://gffzzd3cc09b8251d45dfsnwfqfv9cpqnx6ko5.ffgz.tsg.suse.edu.cn/10.11888/Terre.tpdc.302923.
摘要To address the challenges of dusty,foggy and other complex construction site environments leading to the failure of visible light imaging and difficulties in small target detection,as well as the high resource consumption hindering model deployment,an enhanced and lightweight algorithm is proposed.This algorithm employs a hybrid architecture,integrating red green blue(RGB)(visible light)and thermal infrared(RGBT)multi-modal images through a fusion framework based on you only look once(YOLO)version 8 and Mamba-Transformer(MT).We refer to this integrated model as YOLOv8-RGBT-MT.In terms of network improvements,a frequency enhancement module is first employed to enhance visible light and infrared images.And then,a module integrating Mamba and Transformer components is designed to replace base convolutional blocks in the backbone network,thereby expanding the receptive field of the model and improving feature extraction in complex backgrounds.Finally,a multi-modal feature fusion mechanism is introduced,through which complementary information from visible and infrared images is effectively integrated via an adaptive weighting strategy,so that both the detection accuracy and robustness for small targets are enhanced.Experimental results demonstrate that,compared to YOLOv8-RGBT,the enhanced algorithm achieves an improvement of 18.7%in mAP50,while reducing the number of inference time by 79.7%.
摘要Indeed,a range of systems in the environment requires timely,spatially explicit,and credible information to support its environmental decision-making,but no one observing system can give the complete and reliable measures of the Earth system across scales.This review summarizes how the realization of the Compute the Planet is underway in the form of machine learning,remote sensing,and sensor data fusion to generate decision-ready environmental insights.We use the application-first approach,which considers remote sensing,in situ and Internet of Things(IoT)sensing,and physics-based models as complementary streams of evidence with similar strengths and failures.We look critically at how an integrated system can convert heterogeneous observations to action products across three high impact application areas:atmosphere and air quality,water–land–ecosystem dynamics,and hazards.Rapid-response situational awareness,ecosystem condition metrics,drought and flood indicators,exposure maps,and hazard/extreme indicators are key products.The integrated systems to environment interface in three high impact application areas:atmosphere and air quality,water-land-ecosystem dynamics,and hazard Examine Our operational requirements can often determine real-life value such as latency,time stability,smooth degradation in the presence of missing or degraded inputs,and calibrated uncertainty usable in thresholdbased decisions.These pitfalls are common across fields:mismatch in the scale between a point sensor and a gridded product,objectives on proxies in remotely sensed measurements,domain shift in the extremes and changing baselines,and evaluation aspects,which overestimate generalization because of spatiotemporal autocorrelation.Based on these lessons,we present cross-domain proposals for strong validation,uncertainty quantification,provenance,and versioning,as well as fair performance evaluation.We conclude that the next era of environmental intelligence will see a reduction in average accuracy improvement and an increase in terms of robustness,transparency,and operational responsibility,thus allowing the integrated environmental intelligence system to be deployed,which may be relied on to monitor human health,resource allocation,and survival in a more climate-adapted world.
摘要To address the difficulties in fusing multi-mode sensor data for complex industrial machinery, an adaptive deep coupling convolutional auto-encoder (ADCCAE) fusion method was proposed. First, the multi-mode features extracted synchronously by the CCAE were stacked and fed to the multi-channel convolution layers for fusion. Then, the fused data was passed to all connection layers for compression and fed to the Softmax module for classification. Finally, the coupling loss function coefficients and the network parameters were optimized through an adaptive approach using the gray wolf optimization (GWO) algorithm. Experimental comparisons showed that the proposed ADCCAE fusion model was superior to existing models for multi-mode data fusion.
摘要As a core geospatial data product, Digital Orthophoto Map (DOM) plays a key role in many fields such as urban planning, disaster assessment, engineering surveying and mapping, and topographic mapping. With the increasing requirements of various industries for the timeliness and accuracy of DOM, the traditional single-data-source update method can hardly meet the actual needs. Aiming at this problem, this paper deeply studies the rapid update and accuracy improvement methods of DOM driven by multi-source aerial data fusion. The research shows that multi-source aerial data fusion technology can effectively improve the efficiency and accuracy of DOM update, providing reliable support for the efficient application of geospatial data.
基金supported by grants from the National Key Research and Development Program of China(2022YFD2001103)the National Natural Science Foundation of China(42371373)。
摘要Near real-time maize phenology monitoring is crucial for field management,cropping system adjustments,and yield estimation.Most phenological monitoring methods are post-seasonal and heavily rely on high-frequency time-series data.These methods are not applicable on the unmanned aerial vehicle(UAV)platform due to the high cost of acquiring time-series UAV images and the shortage of UAV-based phenological monitoring methods.To address these challenges,we employed the Synthetic Minority Oversampling Technique(SMOTE)for sample augmentation,aiming to resolve the small sample modelling problem.Moreover,we utilized enhanced"separation"and"compactness"feature selection methods to identify input features from multiple data sources.In this process,we incorporated dynamic multi-source data fusion strategies,involving Vegetation index(VI),Color index(CI),and Texture features(TF).A two-stage neural network that combines Convolutional Neural Network(CNN)and Long Short-Term Memory Network(LSTM)is proposed to identify maize phenological stages(including sowing,seedling,jointing,trumpet,tasseling,maturity,and harvesting)on UAV platforms.The results indicate that the dataset generated by SMOTE closely resembles the measured dataset.Among dynamic data fusion strategies,the VI-TF combination proves to be most effective,with CI-TF and VI-CI combinations following behind.Notably,as more data sources are integrated,the model's demand for input features experiences a significant decline.In particular,the CNN-LSTM model,based on the fusion of three data sources,exhibited remarkable reliability when validating the three datasets.For Dataset 1(Beijing Xiaotangshan,2023:Data from 12 UAV Flight Missions),the model achieved an overall accuracy(OA)of 86.53%.Additionally,its precision(Pre),recall(Rec),F1 score(F1),false acceptance rate(FAR),and false rejection rate(FRR)were 0.89,0.89,0.87,0.11,and 0.11,respectively.The model also showed strong generalizability in Dataset 2(Beijing Xiaotangshan,2023:Data from 6 UAV Flight Missions)and Dataset 3(Beijing Xiaotangshan,2022:Data from 4 UAV Flight Missions),with OAs of 89.4%and 85%,respectively.Meanwhile,the model has a low demand for input featu res,requiring only 54.55%(99 of all featu res).The findings of this study not only offer novel insights into near real-time crop phenology monitoring,but also provide technical support for agricultural field management and cropping system adaptation.
基金supported by the National Natural Science Foundation of China (grant numbers 42293351, and U2468221)。
摘要This paper addresses the accuracy and timeliness limitations of traditional comprehensive prediction methods by proposing an approach of decision-level fusion of multisource data.A risk prediction indicator system was established for water and mud inrush in tunnels by analyzing advanced prediction data for specifi c tunnel segments.Additionally,the indicator weights were determined using the analytic hierarchy process combined with the Huber weighting method.Subsequently,a multisource data decision-layer fusion algorithm was utilized to generate fused imaging results for tunnel water and mud inrush risk predictions.Meanwhile,risk analysis was performed for different tunnel sections to achieve spatial and temporal complementarity within the indicator system and optimize redundant information.Finally,model feasibility was validated using the CZ Project Sejila Mountain Tunnel segment as a case study,yielding favorable risk prediction results and enabling effi cient information fusion and support for construction decision-making.
基金partially supported by the National Natural Science Foundation of China under Grants 62471493 and 62402257(for conceptualization and investigation)partially supported by the Natural Science Foundation of Shandong Province,China under Grants ZR2023LZH017,ZR2024MF066,and 2023QF025(for formal analysis and validation)+1 种基金partially supported by the Open Foundation of Key Laboratory of Computing Power Network and Information Security,Ministry of Education,Qilu University of Technology(Shandong Academy of Sciences)under Grant 2023ZD010(for methodology and model design)partially supported by the Russian Science Foundation(RSF)Project under Grant 22-71-10095-P(for validation and results verification).
摘要To address the challenge of missing modal information in entity alignment and to mitigate information loss or bias arising frommodal heterogeneity during fusion,while also capturing shared information acrossmodalities,this paper proposes a Multi-modal Pre-synergistic Entity Alignmentmodel based on Cross-modalMutual Information Strategy Optimization(MPSEA).The model first employs independent encoders to process multi-modal features,including text,images,and numerical values.Next,a multi-modal pre-synergistic fusion mechanism integrates graph structural and visual modal features into the textual modality as preparatory information.This pre-fusion strategy enables unified perception of heterogeneous modalities at the model’s initial stage,reducing discrepancies during the fusion process.Finally,using cross-modal deep perception reinforcement learning,the model achieves adaptive multilevel feature fusion between modalities,supporting learningmore effective alignment strategies.Extensive experiments on multiple public datasets show that the MPSEA method achieves gains of up to 7% in Hits@1 and 8.2% in MRR on the FBDB15K dataset,and up to 9.1% in Hits@1 and 7.7% in MRR on the FBYG15K dataset,compared to existing state-of-the-art methods.These results confirm the effectiveness of the proposed model.