The rapid growth of LLMs has transformed applications from machine translation to text generation. However, their size is a major barrier. For example, some models have hundreds of billions of parameters, demanding significant computational and memory resources that are unavailable in resource-constrained environments like smartphones or personal computers. To address this, researchers use several model compression techniques such as pruning and quantization. While these methods are effective individually, there has been little guidance on how they can be combined for maximum impact until now.

A research team led by Zhaoguo Wang published their study on the composability of these techniques—or how they can be effectively combined. While previous surveys have typically focused on single compression methods, this work is the first to systematically analyze the interactions, benefits, and challenges of layering them together. The findings were published on 15 September 2026, in Frontiers of Computer Science co-published by Higher Education Press and Springer Nature.

The team systematically examined thirty-two research papers to understand the cumulative effects of combining methods. Their analysis reveals how certain combinations, such as pairing pruning with quantization, can dramatically reduce a model's memory footprint. They also provide insights into how knowledge distillation or low-rank approximation can be used after other compression steps to recover lost performance, leading to models that are both smaller and still highly accurate.

As a result of their comprehensive analysis, the researchers also proposed a proof-of-concept formal framework to help automate the process of finding the optimal combination order. This framework emerges from the study's core findings and represents a step toward simplifying the complex task of designing efficient LLMs. The study provides a foundational guide for the research community and outlines promising directions for future work in the field.

For future work, they suggest focusing on proposing joint optimization frameworks that simultaneously optimize multiple compression techniques while considering their interactions. Additionally, establishing mathematical foundations to predict how different compression methods interact could guide optimal application order and combinations.
DOI:10.1007/s11704-025-50814-1