Researchers have developed a novel multi-constraint optimization method that significantly improves the efficiency of reinforcement learning in complex environments. This new algorithm, called First-Order Projection-based Multi-Constraint Optimization (FPMCO), provides groundbreaking advancements in managing multiple constraints, which is crucial for applications like robotic control with safety limitations. The findings were published on 15 August 2025 in Frontiers of Computer Science co-published by Higher Education Press and Springer Nature.

FPMCO leverages first-order optimization, which is more efficient and easier to implement than conventional second-order methods. Unlike previous approaches that primarily focus on single-constraint problems, FPMCO addresses the common and more challenging multi-constraint scenarios. It does so by decomposing the multi-constraint optimization problem into manageable sub-problems and then using a projection mechanism based on Kullback-Leibler (KL) divergence to ensure compliance with each constraint set. This innovative approach not only makes the algorithm more effective but also reduces computational overhead.
The research team, including Sheng Han, Hengrui Zhang, Hao Wu, Youfang Lin, and Kai Lv from Beijing Jiaotong University, also introduced the Safe Constrained Isaac Gym (SCIG) benchmark to evaluate FPMCO’s performance against other existing reinforcement learning algorithms. The results demonstrated FPMCO's ability to achieve higher rewards while adhering to multiple safety constraints in a variety of complex environments.
DOI: 10.1007/s11704-024-40682-6