“Find the best-selling product from last year.” When an AI system attempts to answer a question like this by querying a company database, even a single reference to a nonexistent item can cause the query to fail. Until now, correcting such an error often required regenerating the entire SQL query from scratch. A KAIST research team has developed a technology that instead identifies and fixes only the erroneous part. The technology is expected to make AI-powered data retrieval faster and more accurate, accelerating the adoption of AI work assistants in enterprise environments.

KAIST (President Choongsik Bae) announced that a research team led by Professor Min-Soo Kim from the School of Computing has developed SafeQL, a technology that detects and corrects errors that arise when natural-language questions are translated into Structured Query Language (SQL).

Text-to-SQL technology enables AI systems to convert everyday questions, such as “Which product saw the largest increase in sales last year?” or “Which items are running low in stock?”, into SQL queries. This allows users to retrieve sales, customer, and inventory data using natural language without having to understand complex database commands.

However, AI systems can make mistakes when generating SQL. For example, they may refer to a table or column that does not exist or join tables incorrectly. These errors can prevent the query from running and leave the AI unable to retrieve the requested data.

Conventional correction methods send the database error back to a large language model (LLM) and ask it to regenerate the entire query. This is similar to rewriting an entire report to correct a single word. In the process, parts that were already correct may be altered, new errors may be introduced, and repeated calls to the LLM increase both cost and processing time.

SafeQL takes a different approach. Instead of discarding and regenerating the entire query after an execution failure, it interprets feedback from the database management system to precisely locate the faulty component, such as a relation, attribute, function, or value. It then incrementally repairs that component while preserving the valid structure and logic of the original query.

To achieve this, the research team developed a “safe query space” approach. Among the candidate corrections that can be executed on the database, SafeQL searches for the one closest to the query originally generated by the AI. The system prioritizes the most promising candidates and filters out unsuitable ones in advance, reducing the time required for correction.

The team implemented SafeQL as a PostgreSQL extension and integrated it with the database system’s parser, binder, and type analyzer. This enables SafeQL to precisely locate errors even in complex SQL queries. The system calls the LLM again when search-based refinement cannot resolve the error within a predefined threshold, thereby reducing unnecessary AI use.

The research team evaluated SafeQL using BIRD and Spider, two widely used benchmarks for assessing the database querying capabilities of AI systems.

On the BIRD benchmark, SafeQL resolved execution errors in up to 87.4% of initially erroneous SQL queries and improved execution accuracy by up to 5.8 percentage points over the unrefined baseline. Compared with regenerating the entire query, SafeQL reduced token use by a factor of up to 15.1 and refinement latency by a factor of up to 29.6.

SafeQL is expected to be particularly useful for enterprises that handle large volumes of data requests. If an error occurs while an AI system is searching internal sales, customer, or inventory data, SafeQL can repair only the affected part instead of regenerating the entire query. This can reduce the cost and time required to operate enterprise AI systems and support reliable work automation powered by AI agents and corporate data.

Professor Min-Soo Kim said, “For AI to perform real-world tasks in enterprise environments, it must be able to accurately retrieve the data it needs.” He added, “When AI makes an error during a database search, SafeQL fixes only the affected part instead of starting over from scratch. By reducing errors, costs, and processing time, we expect the technology to accelerate reliable AI-powered work automation.”

Geonho Lee, a PhD student in the KAIST School of Computing, participated in the study as first author, with Professor Min-Soo Kim serving as corresponding author. The findings will be presented at the International Conference on Very Large Data Bases (VLDB), a leading international database conference, to be held in Boston, USA, from September 1 to 5.

※ Paper title: SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL
DOI: 10.14778/3819518.3819545
※ Authors: Geonho Lee (KAIST, first author) and Min-Soo Kim (KAIST, corresponding author)

This research was supported by the National Research Foundation of Korea (NRF) and the SW Star Lab program of the Institute of Information & Communications Technology Planning & Evaluation (IITP), both funded by the Ministry of Science and ICT (MSIT).