The binocular stereo vision is the lowest cost sensor for obtaining 3D information.Considering the weakness of long‐distance measurement and stability,the improvement of accuracy and stability of stereo vision is urg...The binocular stereo vision is the lowest cost sensor for obtaining 3D information.Considering the weakness of long‐distance measurement and stability,the improvement of accuracy and stability of stereo vision is urgently required for application of precision agriculture.To address the challenges of stereo vision long‐distance measurement and stable perception without hardware upgrade,inspired by hawk eyes,higher resolution perception and the adaptive HDR(High Dynamic Range)were introduced in this paper.Simulating the function from physiological structure of‘deep fovea’and‘shallow fovea’of hawk eye,the higher resolution reconstruction method in this paper was aimed at ac-curacy improving.Inspired by adjustment of pupils,the adaptive HDR method was proposed for high dynamic range optimisation and stable perception.In various light conditions,compared with default stereo vision,the accuracy of proposed algorithm was improved by 28.0%evaluated by error ratio,and the stability was improved by 26.56%by disparity accuracy.For fixed distance measurement,the maximum improvement was 78.6%by standard deviation.Based on the hawk‐eye‐inspired perception algorithm,the point cloud of orchard was improved both in quality and quantity.The hawk‐eye‐inspired perception algorithm contributed great advance in binocular 3D point cloud recon-struction in orchard navigation map.展开更多
Background Three-dimensional(3D)building models with unambiguous roof plane geometry parameters,roof structure units,and linked topology provide essential data for many applications related to human activities in urba...Background Three-dimensional(3D)building models with unambiguous roof plane geometry parameters,roof structure units,and linked topology provide essential data for many applications related to human activities in urban environments.The task of 3D reconstruction from point clouds is still in the development phase,especially the recognition and interpretation of roof topological structures.Methods This study proposes a novel visual perception-based approach to automatically decompose and reconstruct building point clouds into meaningful and simple parametric structures,while the associated mutual relationships between the roof plane geometry and roof structure units are expressed by a hierarchical topology tree.First,a roof plane extraction is performed by a multi-label graph cut energy optimization framework and a roof structure graph(RSG)model is then constructed to describe the roof topological geometry with common adjacency,symmetry,and convexity rules.Moreover,a progressive roof decomposition and refinement are performed,generating a hierarchical representation of the 3D roof structure models.Finally,a visual plane fitted residual or area constraint process is adopted to generate the RSG model with different levels of details.Results Two airborne laser scanning datasets with different point densities and roof styles were tested,and the performance evaluation metrics were obtained by International Society for Photogrammetry and Remote Sensing,achieving a correctness and accuracy of 97.7%and 0.29m,respectively.Conclusions The standardized assessment results demonstrate the effectiveness and robustness of the proposed approach,showing its ability to generate a variety of structural models,even with missing data.展开更多
【目的】环境感知是无人驾驶技术中的核心任务之一,高精度目标检测对于保障自动驾驶系统的安全性与稳定性具有重要意义。近年来,激光雷达(light detection and ranging,LiDAR)作为三维环境感知的关键传感器,凭借不受光照条件影响、测距...【目的】环境感知是无人驾驶技术中的核心任务之一,高精度目标检测对于保障自动驾驶系统的安全性与稳定性具有重要意义。近年来,激光雷达(light detection and ranging,LiDAR)作为三维环境感知的关键传感器,凭借不受光照条件影响、测距精度高等优势,在无人驾驶领域得到了广泛应用。【方法】本文首先回顾了传统基于相机的目标检测方法,并分析了其在复杂环境条件下的局限性;随后系统介绍了LiDAR的发展历程、工作原理、主要类型及关键参数,并对基于点云表示、体素表示以及多传感器融合策略的目标检测方法进行了综述。针对不同方法的网络结构特点、性能优势及面临的挑战进行了对比分析,并结合KITTI检测集的实验结果对相关算法进行了量化性能评估。此外,本文还介绍了基于鸟瞰图(bird′s eye view,BEV)视角的感知框架及多传感器融合的发展趋势,分析了现有算法在检测精度、实时性和环境适应性之间的权衡关系。【结果】总结了LiDAR目标检测方法的主要优势,并针对点云数据稀疏性、计算开销较大以及多模态融合复杂性等关键瓶颈问题,提出了未来研究的重点方向。【结论】持续优化算法与硬件,可提升复杂场景下LiDAR目标检测的精度、鲁棒性与实用性。展开更多
点云语义分割作为三维场景理解的重要任务之一,在智慧城市、智能化测绘等领域具有重要的应用价值。然而,现有分割网络在应对复杂城市场景时,易出现空间关系建模不准确、多尺度语义提取不充分等问题。因此,提出一种融合空间感知与多尺度...点云语义分割作为三维场景理解的重要任务之一,在智慧城市、智能化测绘等领域具有重要的应用价值。然而,现有分割网络在应对复杂城市场景时,易出现空间关系建模不准确、多尺度语义提取不充分等问题。因此,提出一种融合空间感知与多尺度特征的城市级点云语义分割方法LoGNet(local and global network)。通过联合编码点云的几何坐标、颜色属性与上下文语义关系,提升对地物形态差异、光谱特征与空间关联的表达能力;将可学习的空间距离权重与语义相似度共同引入邻域建模,实现基于结构特征与外观属性的精细聚合;构建轻量级的局部–全局双路径特征融合框架,通过点维度与通道维度的全局特征生成方式,强化跨尺度语义一致性与边界解析能力。在Toronto-3D、SensatUrban、STPLS3D公开数据集,与已有常用方法的比较实验表明:LoGNet在三个公开数据集的总体精度分别达97.5%、94.3%、75.0%,均表现最优;在SensatUrban数据集,相较于基线模型,LoGNet的OA、m IoU分别提升了4.5%、10%,在建筑、铁轨、马路等中大型结构性类别,取得了最高得分;对识别极小目标类别,也有显著优势。展开更多
针对PointRCNN(3D Object Proposal Generation and Detection from Point Cloud)在面对不规则点云时很难提取出有区别特征的问题,提出了一种Point-ANN(3D Object Proposal Generation and Aggregation Neural Network)的方法。整个框...针对PointRCNN(3D Object Proposal Generation and Detection from Point Cloud)在面对不规则点云时很难提取出有区别特征的问题,提出了一种Point-ANN(3D Object Proposal Generation and Aggregation Neural Network)的方法。整个框架分为2个阶段。第一个阶段是自下而上生成3D建议,第二阶段执行建议的RoI的感知点云汇集操作,对每个3D方案中的点云信息进行分组,并在坐标中改进3D建议。引入RoI感知点云汇集模块来消除点云上进行区域合并时的模糊性,从而更容易地提取出有区别的特征。通过在KITTI数据集上证明了改进的Point-ANN方法相比于其他网络在3D点云目标检测时精度更高。展开更多
基金funded by the National Natural Science Foundation of China(No.51979275)Key Laboratory of Spatial‐temporal Big Data Analysis and Application of Nat-ural Resources in Megacities,MNR(No.KFKT‐2022‐05)+3 种基金Open Fund of Key Laboratory of Urban Land Resources Monitoring and Simulation,Ministry of Natural Resources(No.KF‐2021‐06‐115)Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems,Bei-hang University(No.VRLAB2022C10)Jiangsu Province and Education Ministry Co‐sponsored Synergistic Innovation Center of Modern Agricultural Equipment(No.XTCX2002)2115 Talent Development Program of China Agricultural University and Chinese Universities Scientific Fund(No.2021TC105).
摘要The binocular stereo vision is the lowest cost sensor for obtaining 3D information.Considering the weakness of long‐distance measurement and stability,the improvement of accuracy and stability of stereo vision is urgently required for application of precision agriculture.To address the challenges of stereo vision long‐distance measurement and stable perception without hardware upgrade,inspired by hawk eyes,higher resolution perception and the adaptive HDR(High Dynamic Range)were introduced in this paper.Simulating the function from physiological structure of‘deep fovea’and‘shallow fovea’of hawk eye,the higher resolution reconstruction method in this paper was aimed at ac-curacy improving.Inspired by adjustment of pupils,the adaptive HDR method was proposed for high dynamic range optimisation and stable perception.In various light conditions,compared with default stereo vision,the accuracy of proposed algorithm was improved by 28.0%evaluated by error ratio,and the stability was improved by 26.56%by disparity accuracy.For fixed distance measurement,the maximum improvement was 78.6%by standard deviation.Based on the hawk‐eye‐inspired perception algorithm,the point cloud of orchard was improved both in quality and quantity.The hawk‐eye‐inspired perception algorithm contributed great advance in binocular 3D point cloud recon-struction in orchard navigation map.
基金Supported by the National Natural Science Foundation of China(41901405,41725005,41531177)and the National Key Research and Development Program of China(2016YFF0103501).
摘要Background Three-dimensional(3D)building models with unambiguous roof plane geometry parameters,roof structure units,and linked topology provide essential data for many applications related to human activities in urban environments.The task of 3D reconstruction from point clouds is still in the development phase,especially the recognition and interpretation of roof topological structures.Methods This study proposes a novel visual perception-based approach to automatically decompose and reconstruct building point clouds into meaningful and simple parametric structures,while the associated mutual relationships between the roof plane geometry and roof structure units are expressed by a hierarchical topology tree.First,a roof plane extraction is performed by a multi-label graph cut energy optimization framework and a roof structure graph(RSG)model is then constructed to describe the roof topological geometry with common adjacency,symmetry,and convexity rules.Moreover,a progressive roof decomposition and refinement are performed,generating a hierarchical representation of the 3D roof structure models.Finally,a visual plane fitted residual or area constraint process is adopted to generate the RSG model with different levels of details.Results Two airborne laser scanning datasets with different point densities and roof styles were tested,and the performance evaluation metrics were obtained by International Society for Photogrammetry and Remote Sensing,achieving a correctness and accuracy of 97.7%and 0.29m,respectively.Conclusions The standardized assessment results demonstrate the effectiveness and robustness of the proposed approach,showing its ability to generate a variety of structural models,even with missing data.
摘要点云语义分割作为三维场景理解的重要任务之一,在智慧城市、智能化测绘等领域具有重要的应用价值。然而,现有分割网络在应对复杂城市场景时,易出现空间关系建模不准确、多尺度语义提取不充分等问题。因此,提出一种融合空间感知与多尺度特征的城市级点云语义分割方法LoGNet(local and global network)。通过联合编码点云的几何坐标、颜色属性与上下文语义关系,提升对地物形态差异、光谱特征与空间关联的表达能力;将可学习的空间距离权重与语义相似度共同引入邻域建模,实现基于结构特征与外观属性的精细聚合;构建轻量级的局部–全局双路径特征融合框架,通过点维度与通道维度的全局特征生成方式,强化跨尺度语义一致性与边界解析能力。在Toronto-3D、SensatUrban、STPLS3D公开数据集,与已有常用方法的比较实验表明:LoGNet在三个公开数据集的总体精度分别达97.5%、94.3%、75.0%,均表现最优;在SensatUrban数据集,相较于基线模型,LoGNet的OA、m IoU分别提升了4.5%、10%,在建筑、铁轨、马路等中大型结构性类别,取得了最高得分;对识别极小目标类别,也有显著优势。
摘要针对PointRCNN(3D Object Proposal Generation and Detection from Point Cloud)在面对不规则点云时很难提取出有区别特征的问题,提出了一种Point-ANN(3D Object Proposal Generation and Aggregation Neural Network)的方法。整个框架分为2个阶段。第一个阶段是自下而上生成3D建议,第二阶段执行建议的RoI的感知点云汇集操作,对每个3D方案中的点云信息进行分组,并在坐标中改进3D建议。引入RoI感知点云汇集模块来消除点云上进行区域合并时的模糊性,从而更容易地提取出有区别的特征。通过在KITTI数据集上证明了改进的Point-ANN方法相比于其他网络在3D点云目标检测时精度更高。