Can AI fine-tuning, that once took 10 hours, be completed in just 3 hours? KAIST researchers have developed a technology that selects the most useful training data and distributes it evenly across multiple GPUs. By reducing repetitive learning of similar data, the method shortened training time by up to 69.6% while maintaining nearly the same AI performance. The technology is expected to help develop AI tailored to specific tasks, such as corporate customer service or assisting medical professionals with information retrieval, more quickly and at lower cost.

KAIST (President Choongsik Bae) announced on November 2 that a research team led by Professor Jae-Gil Lee from the School of Computing developed “CluSTER,” a large language model fine-tuning technology that groups and selects data with similar effects on learning and distributes them evenly across multiple GPUs.

To use a large language model in a specific field such as coding or medicine, it must be additionally trained on relevant data. This process is called “fine-tuning.” It is similar to teaching a broadly knowledgeable AI coding problems and solutions or medical questions and answers so that it can perform tasks in that particular field more effectively.

This process requires substantial computation, so multiple GPUs divide the data and process it in parallel. A GPU is a semiconductor device capable of rapidly performing the calculations required for AI training. Just as several people can divide up a large set of problems and solve them together, multiple GPUs split the training data, process it, and combine the results to improve a single AI model.

However, if the data are distributed randomly, multiple GPUs may repeatedly process data that have similar learning effects. This is like a student repeatedly solving problems that are very similar to ones they have already mastered, resulting in relatively little additional learning compared with the amount of computation required.

The research team therefore focused not only on “how much data to train on,” but also on “which data to select and how to distribute them.” The goal was to reduce repetitive learning while ensuring that the AI did not miss diverse information it needed to learn.

CluSTER first analyzes how each piece of data affects AI learning and groups together data with similar effects. The grouping is not based simply on whether the data cover the same topic or use similar words. Even problems that look different on the surface are classified into the same group if they teach the AI similar problem-solving abilities.

For example, when teaching a student addition, a problem about combining two apples with three apples and a problem about combining two pencils with three pencils tell different stories, but the calculation being learned is the same. In this way, the research team identified and grouped data that had similar effects on learning even when they were expressed differently.

The system then selects data from each group and distributes them so that multiple GPUs process a diverse range of learning information evenly. It is similar to reducing repetitive problems in a workbook while making sure that all important types of problems remain and are divided among several people to solve. The key feature is that the method does not merely reduce the amount of data, but also enables multiple GPUs to use the selected data efficiently.

The research team fine-tuned large language models with approximately 7 billion and 13 billion parameters using three training datasets from the coding and medical fields. Parameters are internal numerical values adjusted as an AI model learns and are used to reflect acquired knowledge in its responses. The team evaluated the effectiveness of the technology through a total of eight performance benchmarks.

In the experiments, the models maintained nearly the same performance even when trained on only about 31% of the full dataset, while training time was reduced by up to 69.6%. Applying the maximum reduction rate, training that previously took 10 hours could be completed in about 3 hours. The research team primarily conducted experiments in an environment using four GPUs and also confirmed better performance than conventional methods in an eight-GPU environment.

This study shows that AI training efficiency can be improved without adding more GPUs if the training data are selected and distributed effectively. In the future, the technology is expected to help companies and research institutions reduce the time and GPU costs required for additional training when developing AI for specific purposes such as coding assistance or medical question answering.

Professor Jae-Gil Lee said, “To improve the efficiency of AI training, it is important not only to use more GPUs, but also to intelligently select which data should be used for learning,” adding, “This study is significant because it shows that existing GPU resources can be used more efficiently to reduce the training burden of large-scale AI models.”

Hyunjin Kim, a Ph.D. student in the KAIST School of Computing, participated in the study as the first author, with Professor Jae-Gil Lee as the corresponding author. Youngeun Nam, a postdoctoral researcher at the University of Toronto, and Jaemin Han and Wonhyeok Choi, master’s students at the KAIST Graduate School of Data Science, participated as co-authors. The research findings were presented in October 2026 at the Conference on Empirical Methods in Natural Language Processing (EMNLP 2026), a major international conference in natural language processing.
※ Paper title: Clustering-Based Balanced Sampling and Allocation with Data Parallelism for High-Performance Fine-Tuning, https://arxiv.org/pdf/2609.12584
※ Author information: Hyunjin Kim (Ph.D. student, KAIST School of Computing, first author), Youngeun Nam (postdoctoral researcher, University of Toronto; KAIST alumnus, co-author), Jaemin Han (master’s student, KAIST Graduate School of Data Science, co-author), Wonhyeok Choi (master’s student, KAIST Graduate School of Data Science, co-author), Jae-Gil Lee (Professor, KAIST School of Computing, corresponding author)

This research was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP), funded by the Ministry of Science and ICT, through the SW Computing Industry Core Technology Development Program’s SW Star Lab project. The project title is “DB4DL: High-Usability and Performance In-Memory Distributed DBMS for Deep Learning” (RS-2020-II200862).