1|Transformer for Chest X-Ray Report Analysis Natural language processing(NLP)has gained widespread use in computer-assisted chest X-ray(CXR)report analysis,particularly since the renaissance of deep learning(DL)in th...1|Transformer for Chest X-Ray Report Analysis Natural language processing(NLP)has gained widespread use in computer-assisted chest X-ray(CXR)report analysis,particularly since the renaissance of deep learning(DL)in the 2012 ImageNet challenge.While early endeavors predominantly employed recurrent neural networks(RNN)and convolutional neural networks(CNN)[1].展开更多
为了实现在城市内涝舆情信息中快速、精准地识别相关风险要素,首先基于新浪微博平台,对用户评论信息及媒体发布信息进行采集、整理及标注,构建了城市内涝灾害事件语料数据集。进而针对城市内涝舆情信息格式不统一、语义复杂且风险要素...为了实现在城市内涝舆情信息中快速、精准地识别相关风险要素,首先基于新浪微博平台,对用户评论信息及媒体发布信息进行采集、整理及标注,构建了城市内涝灾害事件语料数据集。进而针对城市内涝舆情信息格式不统一、语义复杂且风险要素识别的专业性、精准度要求较高等问题,结合自然灾害系统理论的风险要素框架,提出了一种基于双向编码器表征法-双向长短期记忆-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field,BERT-BiLSTM-CRF)的识别方法,并开展了一系列模型验证试验。对比试验结果表明,该模型在准确率、召回率、F1三项指标上均有较好表现,其中准确率为84.62%,召回率为86.19%,F1为85.35%,优于其他对比模型。消融试验结果表明,BERT预训练模型对于该模型性能有着更为显著的影响。综合上述试验结果,可以验证该模型能够有效识别城市内涝舆情信息中的各类风险要素,进而为城市内涝灾害风险管控的数智化转型提供研究依据。展开更多
聚焦国家电网客服中心客户诉求数据治理中存在的效率低、人工依赖性强等问题,提出基于Transformer的双向编码器表征(Bidirectional Encoder Representations from Transformer,BERT)和双向长短时记忆(Bi-directional Long Short-Term Me...聚焦国家电网客服中心客户诉求数据治理中存在的效率低、人工依赖性强等问题,提出基于Transformer的双向编码器表征(Bidirectional Encoder Representations from Transformer,BERT)和双向长短时记忆(Bi-directional Long Short-Term Memory,BiLSTM)融合技术的多阶段联合数据治理框架。通过构建有效性判断、语义增强、诉求监测及业务场景分类等核心模块,形成覆盖数据预处理、语义分析、分类预测及诉求应用的全链路治理体系。结果验表明,提出的BERT与BiLSTM融合技术具有较好的性能指标。所提框架通过动态语义特征提取与上下文建模的协同机制,实现客户诉求的细粒度分类和风险点识别,验证基于BERT和BiLSTM的融合模型在电力企业文本类数据处理和应用中的适用性和有效性,为构建自动化数据治理体系提供了更丰富的解决方案。展开更多
Background:The medical records of traditional Chinese medicine(TCM)contain numerous synonymous terms with different descriptions,which is not conducive to computer-aided data mining of TCM.However,there is a lack of m...Background:The medical records of traditional Chinese medicine(TCM)contain numerous synonymous terms with different descriptions,which is not conducive to computer-aided data mining of TCM.However,there is a lack of models available to normalize synonymous TCM terms.Therefore,construction of a synonymous term conversion(STC)model for normalizing synonymous TCM terms is necessary.Methods:Based on the neural networks of bidirectional encoder representations from transformers(BERT),four types of TCM STC models were designed:Models based on BERT and text classification,text sequence generation,named entity recognition,and text matching.The superior STC model was selected on the basis of its performance in converting synonymous terms.Moreover,three misjudgment inspection methods for the conversion results of the STC model based on inconsistency were proposed to find incorrect term conversion:Neuron random deactivation,output comparison of multiple isomorphic models,and output comparison of multiple heterogeneous models(OCMH).Results:The classification-based STC model outperformed the other STC task models.It achieved F1 scores of 0.91,0.91,and 0.83 for performing symptoms,patterns,and treatments STC tasks,respectively.The OCMH method showed the best performance in misjudgment inspection,with wrong detection rates of 0.80,0.84,and 0.90 in the term conversion results for symptoms,patterns,and treatments,respectively.Conclusion:The TCM STC model based on classification achieved superior performance in converting synonymous terms for symptoms,patterns,and treatments.The misjudgment inspection method based on OCMH showed superior performance in identifying incorrect outputs.展开更多
现有的无监督关键短语提取模型对复杂的上下文和多层次语义信息的捕获能力不足,无法获取多维度信息。因此,提出一种基于多视角信息增强和层次化权重的关键短语提取模型。首先,利用BERT(Bidirectional Encoder Representations from Tran...现有的无监督关键短语提取模型对复杂的上下文和多层次语义信息的捕获能力不足,无法获取多维度信息。因此,提出一种基于多视角信息增强和层次化权重的关键短语提取模型。首先,利用BERT(Bidirectional Encoder Representations from Transformers)预训练模型对文本和候选短语进行编码,获得嵌入表示;并且,通过加权平均池化优化文本嵌入,并计算它们与候选短语的全局相似度,以实现全局信息增强,提升对语义关联的理解。其次,提出基于图结构的边界感知局部中心性计算方法,以增强局部信息获取能力。最后,融合多因素计算权重,从多个维度评估候选短语的重要性。在Inspec、SemEval 2017和SemEval-2010等6个公开数据集上的实验结果表明,与基线模型PromptRank相比,所提模型的F1@5值提高了0.87~2.68个百分点,F1@10值提高了1.11~2.24个百分点,F1@15值提高了0.54~2.25个百分点。可见,所提模型的综合性能得到了有效提升。展开更多
在一些修船企业建立的修船结算系统和电子价格库中,人工匹配结算编码步骤易出错且耗时长,直接影响结算效率。为解决该问题,提出一种基于多特征融合的修船结算编码智能匹配复合模型。采用来自变换器的双向编码器表示(Bidirectional Encod...在一些修船企业建立的修船结算系统和电子价格库中,人工匹配结算编码步骤易出错且耗时长,直接影响结算效率。为解决该问题,提出一种基于多特征融合的修船结算编码智能匹配复合模型。采用来自变换器的双向编码器表示(Bidirectional Encoder Representations from Transformers,BERT)模型将工程内容文本表示为词向量,采用卷积神经网络(Convolutional Neural Network,CNN)模型提取文本的局部特征,采用双向长短期记忆网络结合注意力机制(Bidirectional Long Short-Term Memory with Attention Mechanism,BiLSTM-Attention)模型提取上下文特征,得到对应的结算编码。试验结果表明,所提出的复合模型在整体准确率方面实现显著提升,充分证明该复合模型在处理复杂文本分类任务中的优势。展开更多
摘要1|Transformer for Chest X-Ray Report Analysis Natural language processing(NLP)has gained widespread use in computer-assisted chest X-ray(CXR)report analysis,particularly since the renaissance of deep learning(DL)in the 2012 ImageNet challenge.While early endeavors predominantly employed recurrent neural networks(RNN)and convolutional neural networks(CNN)[1].
摘要为了实现在城市内涝舆情信息中快速、精准地识别相关风险要素,首先基于新浪微博平台,对用户评论信息及媒体发布信息进行采集、整理及标注,构建了城市内涝灾害事件语料数据集。进而针对城市内涝舆情信息格式不统一、语义复杂且风险要素识别的专业性、精准度要求较高等问题,结合自然灾害系统理论的风险要素框架,提出了一种基于双向编码器表征法-双向长短期记忆-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field,BERT-BiLSTM-CRF)的识别方法,并开展了一系列模型验证试验。对比试验结果表明,该模型在准确率、召回率、F1三项指标上均有较好表现,其中准确率为84.62%,召回率为86.19%,F1为85.35%,优于其他对比模型。消融试验结果表明,BERT预训练模型对于该模型性能有着更为显著的影响。综合上述试验结果,可以验证该模型能够有效识别城市内涝舆情信息中的各类风险要素,进而为城市内涝灾害风险管控的数智化转型提供研究依据。
摘要聚焦国家电网客服中心客户诉求数据治理中存在的效率低、人工依赖性强等问题,提出基于Transformer的双向编码器表征(Bidirectional Encoder Representations from Transformer,BERT)和双向长短时记忆(Bi-directional Long Short-Term Memory,BiLSTM)融合技术的多阶段联合数据治理框架。通过构建有效性判断、语义增强、诉求监测及业务场景分类等核心模块,形成覆盖数据预处理、语义分析、分类预测及诉求应用的全链路治理体系。结果验表明,提出的BERT与BiLSTM融合技术具有较好的性能指标。所提框架通过动态语义特征提取与上下文建模的协同机制,实现客户诉求的细粒度分类和风险点识别,验证基于BERT和BiLSTM的融合模型在电力企业文本类数据处理和应用中的适用性和有效性,为构建自动化数据治理体系提供了更丰富的解决方案。
摘要针对现有的中文命名实体识别算法没有充分考虑实体识别任务的数据特征,存在中文样本数据的类别不平衡、训练数据中的噪声太大和每次模型生成数据的分布差异较大的问题,提出了一种以BERT-BiLSTM-CRF(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field)为基线改进的中文命名实体识别模型。首先在BERT-BiLSTM-CRF模型上结合P-Tuning v2技术,精确提取数据特征,然后使用3个损失函数包括聚焦损失(Focal Loss)、标签平滑(Label Smoothing)和KL Loss(Kullback-Leibler divergence loss)作为正则项参与损失计算。实验结果表明,改进的模型在Weibo、Resume和MSRA(Microsoft Research Asia)数据集上的F 1得分分别为71.13%、96.31%、95.90%,验证了所提算法具有更好的性能,并且在不同的下游任务中,所提算法易于与其他的神经网络结合与扩展。
基金The National Key R&D Program of China supported this study(2017YFC1700303).
摘要Background:The medical records of traditional Chinese medicine(TCM)contain numerous synonymous terms with different descriptions,which is not conducive to computer-aided data mining of TCM.However,there is a lack of models available to normalize synonymous TCM terms.Therefore,construction of a synonymous term conversion(STC)model for normalizing synonymous TCM terms is necessary.Methods:Based on the neural networks of bidirectional encoder representations from transformers(BERT),four types of TCM STC models were designed:Models based on BERT and text classification,text sequence generation,named entity recognition,and text matching.The superior STC model was selected on the basis of its performance in converting synonymous terms.Moreover,three misjudgment inspection methods for the conversion results of the STC model based on inconsistency were proposed to find incorrect term conversion:Neuron random deactivation,output comparison of multiple isomorphic models,and output comparison of multiple heterogeneous models(OCMH).Results:The classification-based STC model outperformed the other STC task models.It achieved F1 scores of 0.91,0.91,and 0.83 for performing symptoms,patterns,and treatments STC tasks,respectively.The OCMH method showed the best performance in misjudgment inspection,with wrong detection rates of 0.80,0.84,and 0.90 in the term conversion results for symptoms,patterns,and treatments,respectively.Conclusion:The TCM STC model based on classification achieved superior performance in converting synonymous terms for symptoms,patterns,and treatments.The misjudgment inspection method based on OCMH showed superior performance in identifying incorrect outputs.
摘要在一些修船企业建立的修船结算系统和电子价格库中,人工匹配结算编码步骤易出错且耗时长,直接影响结算效率。为解决该问题,提出一种基于多特征融合的修船结算编码智能匹配复合模型。采用来自变换器的双向编码器表示(Bidirectional Encoder Representations from Transformers,BERT)模型将工程内容文本表示为词向量,采用卷积神经网络(Convolutional Neural Network,CNN)模型提取文本的局部特征,采用双向长短期记忆网络结合注意力机制(Bidirectional Long Short-Term Memory with Attention Mechanism,BiLSTM-Attention)模型提取上下文特征,得到对应的结算编码。试验结果表明,所提出的复合模型在整体准确率方面实现显著提升,充分证明该复合模型在处理复杂文本分类任务中的优势。