随着大数据时代的到来,各种企业和组织在日益增长的业务拓展以及信息需求的推动下积累了海量的数据。这些数据通常以结构化或半结构化的形式存储在对应的关系型数据库中。SQL作为一种结构化的数据库查询语言,长期以来被广泛应用于数据...随着大数据时代的到来,各种企业和组织在日益增长的业务拓展以及信息需求的推动下积累了海量的数据。这些数据通常以结构化或半结构化的形式存储在对应的关系型数据库中。SQL作为一种结构化的数据库查询语言,长期以来被广泛应用于数据的检索和处理。它为相关专业技术人员提供了一种高效的方式来与数据库进行交互,从而能够更快捷方便地实现对于数据的分析。随着大语言模型(large language models,LLMs)的快速发展,上下文学习(in-context learning,ICL)在文本生成SQL(textto-SQL)任务中展现出巨大潜力。ICL能够使大语言模型通过利用参考示例生成准确的SQL查询。为了充分利用大语言模型在ICL机制下进行文本生成SQL,提出了一个新的SQL生成流程,该流程包括3个部分:首先,提出了迭代式的模式增强,以强化大语言模型对解决问题相关信息的关注。其次,利用SQL的骨架结构进行语义相似度筛选,从而辅助生成伪SQL(pseudo-SQL)查询。最后设计了一种参考示例选择策略,该策略结合问题与伪SQL骨架之间的相似性,以提高生成SQL查询的准确性。同时,在2个关键阶段中都引入了迭代优化过程,以逐步优化示例选择策略,直至达到稳定状态。在多个大语言模型和数据集上进行了广泛的实验,验证了所提出方法的有效性。展开更多
This study examines the potential of open-source foundation models for structured data analytics,with particular emphasis on SQL generation and business-oriented interpretation in single-agent and multi-agent large la...This study examines the potential of open-source foundation models for structured data analytics,with particular emphasis on SQL generation and business-oriented interpretation in single-agent and multi-agent large language model(LLM)systems.The proposed framework addresses a practical problem in analytics-intensive environments,where natural-language requests must be translated into executable,semantically appropriate SQL queries and subsequently interpreted in a form useful for business decision-making.The system is evaluated in two complementary settings:a custom SQL test suite designed around realistic marketing and e-commerce analytics tasks,and the public Spider benchmark,which supports comparison with prior text-to-SQL research and enables assessment of cross-domain generalization.The analysis includes Mistral,Devstral,Qwen2.5-Coder,and Qwen3.On the custom SQL test suite,performance was assessed using exact match,safe SQL rate,and an independent semantic judge score.Qwen2.5-Coder achieved the strongest overall result,reaching an independent semantic score of 90.14%while maintaining a 98.59%safe SQL rate.Qwen3 followed with a semantic score of 77.46%and completely safe SQL generation.These results indicate that in domain-specific analytics settings,strict query-level matching alone is too conservative to capture practical model usefulness,since semantically appropriate SQL queries may differ substantially from the reference formulation.The Spider benchmark results provide complementary evidence regarding broader model behavior.Qwen2.5-Coder achieved the highest single-agent execution accuracy(72.44%),whereas Devstral obtained the strongest single-agent exact-match score(28.14%).Qwen3 remained competitive and delivered the lowest single-agent latency(0.41 s)among the evaluated models.At the architectural level,the effect of multi-agent decomposition was not uniform:it yielded modest gains in execution accuracy for some model families,but reduced performance for others,while consistently increasing latency and token consumption.Taken together,the findings show that open-source LLM agents can provide effective support for structured analytics,but that their performance depends strongly on model family,prompting strategy,and agent architecture.More broadly,the study demonstrates that the evaluation of text-to-SQL systems benefits from combining benchmark-based metrics,domain-oriented semantic assessment,and efficiency-aware analysis,thereby offering a more realistic basis for the deployment of open-source LLM systems in analytics-intensive environments.展开更多
摘要随着大数据时代的到来,各种企业和组织在日益增长的业务拓展以及信息需求的推动下积累了海量的数据。这些数据通常以结构化或半结构化的形式存储在对应的关系型数据库中。SQL作为一种结构化的数据库查询语言,长期以来被广泛应用于数据的检索和处理。它为相关专业技术人员提供了一种高效的方式来与数据库进行交互,从而能够更快捷方便地实现对于数据的分析。随着大语言模型(large language models,LLMs)的快速发展,上下文学习(in-context learning,ICL)在文本生成SQL(textto-SQL)任务中展现出巨大潜力。ICL能够使大语言模型通过利用参考示例生成准确的SQL查询。为了充分利用大语言模型在ICL机制下进行文本生成SQL,提出了一个新的SQL生成流程,该流程包括3个部分:首先,提出了迭代式的模式增强,以强化大语言模型对解决问题相关信息的关注。其次,利用SQL的骨架结构进行语义相似度筛选,从而辅助生成伪SQL(pseudo-SQL)查询。最后设计了一种参考示例选择策略,该策略结合问题与伪SQL骨架之间的相似性,以提高生成SQL查询的准确性。同时,在2个关键阶段中都引入了迭代优化过程,以逐步优化示例选择策略,直至达到稳定状态。在多个大语言模型和数据集上进行了广泛的实验,验证了所提出方法的有效性。
基金supported by the Croatian Science Foundation under the project number IP-2025-02-1267by the EuropeanUnion’sHorizon Europe research and innovation programme under GrantNo.101086179.
摘要This study examines the potential of open-source foundation models for structured data analytics,with particular emphasis on SQL generation and business-oriented interpretation in single-agent and multi-agent large language model(LLM)systems.The proposed framework addresses a practical problem in analytics-intensive environments,where natural-language requests must be translated into executable,semantically appropriate SQL queries and subsequently interpreted in a form useful for business decision-making.The system is evaluated in two complementary settings:a custom SQL test suite designed around realistic marketing and e-commerce analytics tasks,and the public Spider benchmark,which supports comparison with prior text-to-SQL research and enables assessment of cross-domain generalization.The analysis includes Mistral,Devstral,Qwen2.5-Coder,and Qwen3.On the custom SQL test suite,performance was assessed using exact match,safe SQL rate,and an independent semantic judge score.Qwen2.5-Coder achieved the strongest overall result,reaching an independent semantic score of 90.14%while maintaining a 98.59%safe SQL rate.Qwen3 followed with a semantic score of 77.46%and completely safe SQL generation.These results indicate that in domain-specific analytics settings,strict query-level matching alone is too conservative to capture practical model usefulness,since semantically appropriate SQL queries may differ substantially from the reference formulation.The Spider benchmark results provide complementary evidence regarding broader model behavior.Qwen2.5-Coder achieved the highest single-agent execution accuracy(72.44%),whereas Devstral obtained the strongest single-agent exact-match score(28.14%).Qwen3 remained competitive and delivered the lowest single-agent latency(0.41 s)among the evaluated models.At the architectural level,the effect of multi-agent decomposition was not uniform:it yielded modest gains in execution accuracy for some model families,but reduced performance for others,while consistently increasing latency and token consumption.Taken together,the findings show that open-source LLM agents can provide effective support for structured analytics,but that their performance depends strongly on model family,prompting strategy,and agent architecture.More broadly,the study demonstrates that the evaluation of text-to-SQL systems benefits from combining benchmark-based metrics,domain-oriented semantic assessment,and efficiency-aware analysis,thereby offering a more realistic basis for the deployment of open-source LLM systems in analytics-intensive environments.