大型语言模型(LLM)已成为推进Text-to-SQL任务的强大工具。研究发现,基于LLM的模型在不同评估指标下,其性能表现与经过微调的模型存在显著差异。因此,文章分析了测试套件执行准确度(EXE)和精确集匹配准确度(ESM)在评估基于LLM的Text-to-...大型语言模型(LLM)已成为推进Text-to-SQL任务的强大工具。研究发现,基于LLM的模型在不同评估指标下,其性能表现与经过微调的模型存在显著差异。因此,文章分析了测试套件执行准确度(EXE)和精确集匹配准确度(ESM)在评估基于LLM的Text-to-SQL模型时的不足,并提出了改进指标EESM(Enhanced Exact Set Matching)。实验结果表明,EXE和ESM分别存在高达13.2%和10.8%的假阳性和假阴性率,而EESM的假阳性率和假阴性率分别仅为0.2%和1.8%,表明EESM能够提供更准确的评估。展开更多
Text-to-SQL is the task of translating a natural language query into a structured query language. Existing text-to-SQL approaches focus on improving the model’s architecture while ignoring the relationship between qu...Text-to-SQL is the task of translating a natural language query into a structured query language. Existing text-to-SQL approaches focus on improving the model’s architecture while ignoring the relationship between queries and table schemas and the differences in difficulty between examples in the dataset. To tackle these challenges, a two-stage curriculum learning framework for text-to-SQL(TSCL-SQL) is proposed in this paper. To exploit the relationship between the queries and the table schemas, a schema identification pre-training task is proposed to make the model choose the correct table schema from a set of candidates for a specific query. To leverage the differences in difficulty between examples, curriculum learning is applied to the text-to-SQL task, accompanied by an automatic curriculum learning solution, including a difficulty scorer and a training scheduler. Experiments show that the framework proposed in this paper is effective.展开更多
Recent advancements in large language models(LLMs)have significantly contributed to the progress of the Text-to-SQL task.A common requirement in many of these works is the post-correction of SQL queries.However,the ma...Recent advancements in large language models(LLMs)have significantly contributed to the progress of the Text-to-SQL task.A common requirement in many of these works is the post-correction of SQL queries.However,the majority of this process entails analyzing error cases to develop prompts with rules that eliminate model bias.And there is a weakness of execution verification for SQL queries.In addition,the prevalent techniques primarily depend on GPT-4 and few-shot prompts,resulting in expensive costs.To investigate the effective methods for SQL refinement in a cost-efficient manner,we introduce Semantic-Enhanced Text-to-SQL with Adaptive Refinement(SEA-SQL),which includes Adaptive Bias Elimination and Dynamic Execution Adjustment,aims to improve performance while minimizing resource expenditure with zero-shot prompts.Specifically,SEA-SQL employs a semantic-enhanced schema to augment database information and optimize SQL queries.During the SQL query generation,a fine-tuned adaptive bias eliminator is applied to mitigate inherent biases caused by the LLM.The dynamic execution adjustment is utilized to guarantee the executability of the bias eliminated SQL query.We conduct experiments on the Spider and BIRD datasets to demonstrate the effectiveness of this framework.The results demonstrate that SEA-SQL achieves state-of-the-art performance in the GPT-3.5 scenario with 9%-58% of the generation cost.Furthermore,SEA-SQL is comparable to GPT-4 with only 0.9%-5.3% of the generation cost.Our code is available at the website of github.com/545999961/SEA-SQL.展开更多
The increasing complexity of modern power systems,driven by factors such as the large-scale integration of renewable energy and the proliferation of distributed generation,has placed unprecedented demands on power dis...The increasing complexity of modern power systems,driven by factors such as the large-scale integration of renewable energy and the proliferation of distributed generation,has placed unprecedented demands on power dispatching operations.Ensuring grid stability and safety in this new environment requires real-time monitoring and swift,data-driven decision-making.Consequently,efficient and accurate data querying capabilities have become paramount.This study introduces Intelli-Dispatch-SQL,a novel agent-based Text-to-SQL framework that leverages the Large Language Model(LLM)to enhance the accuracy and reliability of generated SQL queries in the context of power dispatching.By integrating intent recognition and SQL validation modules,Intelli-Dispatch-SQL ensures that generated queries are not only syntactically correct but also semantically aligned with user intent and executable within the operational context.Through comprehensive experiments,including ablation studies and cross-model evaluations,we demonstrate that Intelli-Dispatch-SQL significantly outperforms existing Text-to-SQL models,achieving substantial improvements in both Exact Match(EM)and Execution Accuracy(EX).Notably,the incorporation of intent recognition and SQL validation modules is shown to be critical for performance enhancement.The framework’s effectiveness was further validated across various LLMs,confirming its robustness and applicability across diverse scenarios.Intelli-Dispatch-SQL offers a performance high-and generalizable solution for Text-to-SQL in power dispatching,paving the way for more efficient and intelligent power system management.展开更多
大语言模型(large language model,LLM)为数据库性能带来了极大的提升,将LLM与数据库相结合成为当前的研究热点。然而,目前大部分高校数据库实验课程仍停留于数据库基础操作层面,学生普遍缺乏数据库前沿技术相关知识。文本转结构化查询...大语言模型(large language model,LLM)为数据库性能带来了极大的提升,将LLM与数据库相结合成为当前的研究热点。然而,目前大部分高校数据库实验课程仍停留于数据库基础操作层面,学生普遍缺乏数据库前沿技术相关知识。文本转结构化查询语言(text to structured query language,Text-to-SQL)技术为基于LLM的数据库设计的重要研究方向之一。首先,设计实验课程系统地介绍基于LLM的Text-to-SQL方法的教学意义,以盘古大模型为例对实验课程相关技术进行介绍;其次,分析并划定基于LLM的数据库设计实验课程的教学目标,阐述具体的教学设计和实施;最后,对教学方法设计的各方面进行系统总结。展开更多
This study examines the potential of open-source foundation models for structured data analytics,with particular emphasis on SQL generation and business-oriented interpretation in single-agent and multi-agent large la...This study examines the potential of open-source foundation models for structured data analytics,with particular emphasis on SQL generation and business-oriented interpretation in single-agent and multi-agent large language model(LLM)systems.The proposed framework addresses a practical problem in analytics-intensive environments,where natural-language requests must be translated into executable,semantically appropriate SQL queries and subsequently interpreted in a form useful for business decision-making.The system is evaluated in two complementary settings:a custom SQL test suite designed around realistic marketing and e-commerce analytics tasks,and the public Spider benchmark,which supports comparison with prior text-to-SQL research and enables assessment of cross-domain generalization.The analysis includes Mistral,Devstral,Qwen2.5-Coder,and Qwen3.On the custom SQL test suite,performance was assessed using exact match,safe SQL rate,and an independent semantic judge score.Qwen2.5-Coder achieved the strongest overall result,reaching an independent semantic score of 90.14%while maintaining a 98.59%safe SQL rate.Qwen3 followed with a semantic score of 77.46%and completely safe SQL generation.These results indicate that in domain-specific analytics settings,strict query-level matching alone is too conservative to capture practical model usefulness,since semantically appropriate SQL queries may differ substantially from the reference formulation.The Spider benchmark results provide complementary evidence regarding broader model behavior.Qwen2.5-Coder achieved the highest single-agent execution accuracy(72.44%),whereas Devstral obtained the strongest single-agent exact-match score(28.14%).Qwen3 remained competitive and delivered the lowest single-agent latency(0.41 s)among the evaluated models.At the architectural level,the effect of multi-agent decomposition was not uniform:it yielded modest gains in execution accuracy for some model families,but reduced performance for others,while consistently increasing latency and token consumption.Taken together,the findings show that open-source LLM agents can provide effective support for structured analytics,but that their performance depends strongly on model family,prompting strategy,and agent architecture.More broadly,the study demonstrates that the evaluation of text-to-SQL systems benefits from combining benchmark-based metrics,domain-oriented semantic assessment,and efficiency-aware analysis,thereby offering a more realistic basis for the deployment of open-source LLM systems in analytics-intensive environments.展开更多
摘要大型语言模型(LLM)已成为推进Text-to-SQL任务的强大工具。研究发现,基于LLM的模型在不同评估指标下,其性能表现与经过微调的模型存在显著差异。因此,文章分析了测试套件执行准确度(EXE)和精确集匹配准确度(ESM)在评估基于LLM的Text-to-SQL模型时的不足,并提出了改进指标EESM(Enhanced Exact Set Matching)。实验结果表明,EXE和ESM分别存在高达13.2%和10.8%的假阳性和假阴性率,而EESM的假阳性率和假阴性率分别仅为0.2%和1.8%,表明EESM能够提供更准确的评估。
基金Fundamental Research Funds for the Central Universities,China (No. 2232023D-19)。
摘要Text-to-SQL is the task of translating a natural language query into a structured query language. Existing text-to-SQL approaches focus on improving the model’s architecture while ignoring the relationship between queries and table schemas and the differences in difficulty between examples in the dataset. To tackle these challenges, a two-stage curriculum learning framework for text-to-SQL(TSCL-SQL) is proposed in this paper. To exploit the relationship between the queries and the table schemas, a schema identification pre-training task is proposed to make the model choose the correct table schema from a set of candidates for a specific query. To leverage the differences in difficulty between examples, curriculum learning is applied to the text-to-SQL task, accompanied by an automatic curriculum learning solution, including a difficulty scorer and a training scheduler. Experiments show that the framework proposed in this paper is effective.
基金supported by the National Natural Science Foundation of China(Grant Nos.62272054,62192784,62172056)the Beijing Nova Program(No.20230484319)the Xiaomi Young Talents Program.
摘要Recent advancements in large language models(LLMs)have significantly contributed to the progress of the Text-to-SQL task.A common requirement in many of these works is the post-correction of SQL queries.However,the majority of this process entails analyzing error cases to develop prompts with rules that eliminate model bias.And there is a weakness of execution verification for SQL queries.In addition,the prevalent techniques primarily depend on GPT-4 and few-shot prompts,resulting in expensive costs.To investigate the effective methods for SQL refinement in a cost-efficient manner,we introduce Semantic-Enhanced Text-to-SQL with Adaptive Refinement(SEA-SQL),which includes Adaptive Bias Elimination and Dynamic Execution Adjustment,aims to improve performance while minimizing resource expenditure with zero-shot prompts.Specifically,SEA-SQL employs a semantic-enhanced schema to augment database information and optimize SQL queries.During the SQL query generation,a fine-tuned adaptive bias eliminator is applied to mitigate inherent biases caused by the LLM.The dynamic execution adjustment is utilized to guarantee the executability of the bias eliminated SQL query.We conduct experiments on the Spider and BIRD datasets to demonstrate the effectiveness of this framework.The results demonstrate that SEA-SQL achieves state-of-the-art performance in the GPT-3.5 scenario with 9%-58% of the generation cost.Furthermore,SEA-SQL is comparable to GPT-4 with only 0.9%-5.3% of the generation cost.Our code is available at the website of github.com/545999961/SEA-SQL.
基金supported by the Guangdong Power Grid Com-pany(Grant Number:GDKJXM20231024)the National Natural Sci-ence Foundation of China(Grant Number:72331009,72171206 and 92270105)the Shenzhen Key Laboratory of Crowd Intelligence Em-powered Low-Carbon Energy Network(Grant number:ZDSYS20220606100601002).
摘要The increasing complexity of modern power systems,driven by factors such as the large-scale integration of renewable energy and the proliferation of distributed generation,has placed unprecedented demands on power dispatching operations.Ensuring grid stability and safety in this new environment requires real-time monitoring and swift,data-driven decision-making.Consequently,efficient and accurate data querying capabilities have become paramount.This study introduces Intelli-Dispatch-SQL,a novel agent-based Text-to-SQL framework that leverages the Large Language Model(LLM)to enhance the accuracy and reliability of generated SQL queries in the context of power dispatching.By integrating intent recognition and SQL validation modules,Intelli-Dispatch-SQL ensures that generated queries are not only syntactically correct but also semantically aligned with user intent and executable within the operational context.Through comprehensive experiments,including ablation studies and cross-model evaluations,we demonstrate that Intelli-Dispatch-SQL significantly outperforms existing Text-to-SQL models,achieving substantial improvements in both Exact Match(EM)and Execution Accuracy(EX).Notably,the incorporation of intent recognition and SQL validation modules is shown to be critical for performance enhancement.The framework’s effectiveness was further validated across various LLMs,confirming its robustness and applicability across diverse scenarios.Intelli-Dispatch-SQL offers a performance high-and generalizable solution for Text-to-SQL in power dispatching,paving the way for more efficient and intelligent power system management.
摘要大语言模型(large language model,LLM)为数据库性能带来了极大的提升,将LLM与数据库相结合成为当前的研究热点。然而,目前大部分高校数据库实验课程仍停留于数据库基础操作层面,学生普遍缺乏数据库前沿技术相关知识。文本转结构化查询语言(text to structured query language,Text-to-SQL)技术为基于LLM的数据库设计的重要研究方向之一。首先,设计实验课程系统地介绍基于LLM的Text-to-SQL方法的教学意义,以盘古大模型为例对实验课程相关技术进行介绍;其次,分析并划定基于LLM的数据库设计实验课程的教学目标,阐述具体的教学设计和实施;最后,对教学方法设计的各方面进行系统总结。
基金supported by the Croatian Science Foundation under the project number IP-2025-02-1267by the EuropeanUnion’sHorizon Europe research and innovation programme under GrantNo.101086179.
摘要This study examines the potential of open-source foundation models for structured data analytics,with particular emphasis on SQL generation and business-oriented interpretation in single-agent and multi-agent large language model(LLM)systems.The proposed framework addresses a practical problem in analytics-intensive environments,where natural-language requests must be translated into executable,semantically appropriate SQL queries and subsequently interpreted in a form useful for business decision-making.The system is evaluated in two complementary settings:a custom SQL test suite designed around realistic marketing and e-commerce analytics tasks,and the public Spider benchmark,which supports comparison with prior text-to-SQL research and enables assessment of cross-domain generalization.The analysis includes Mistral,Devstral,Qwen2.5-Coder,and Qwen3.On the custom SQL test suite,performance was assessed using exact match,safe SQL rate,and an independent semantic judge score.Qwen2.5-Coder achieved the strongest overall result,reaching an independent semantic score of 90.14%while maintaining a 98.59%safe SQL rate.Qwen3 followed with a semantic score of 77.46%and completely safe SQL generation.These results indicate that in domain-specific analytics settings,strict query-level matching alone is too conservative to capture practical model usefulness,since semantically appropriate SQL queries may differ substantially from the reference formulation.The Spider benchmark results provide complementary evidence regarding broader model behavior.Qwen2.5-Coder achieved the highest single-agent execution accuracy(72.44%),whereas Devstral obtained the strongest single-agent exact-match score(28.14%).Qwen3 remained competitive and delivered the lowest single-agent latency(0.41 s)among the evaluated models.At the architectural level,the effect of multi-agent decomposition was not uniform:it yielded modest gains in execution accuracy for some model families,but reduced performance for others,while consistently increasing latency and token consumption.Taken together,the findings show that open-source LLM agents can provide effective support for structured analytics,but that their performance depends strongly on model family,prompting strategy,and agent architecture.More broadly,the study demonstrates that the evaluation of text-to-SQL systems benefits from combining benchmark-based metrics,domain-oriented semantic assessment,and efficiency-aware analysis,thereby offering a more realistic basis for the deployment of open-source LLM systems in analytics-intensive environments.