It’s important for a robot, not just to learn a task optimally, but to complete it without going off the rails of reasonable expectations. Until now, roboticists could have one or the other. Aerospace engineering researchers in The Grainger College of Engineering, University of Illinois Urbana-Champaign combined reinforcement learning with control theory to guarantee robot stability and improve the tracking performance by up to 15% from the existing methods.

“Reinforcement learning offers a promising way for robots to learn effective behavior, but it often lacks stability guarantees,” said Ph.D. student Minjae Cho, whose advisors are Huy Tran and Hiro Tsukamoto. “Without confidence that a robot will remain stable, deploying it in practice is difficult. In a worst-case scenario, it could break and require costly repairs. So, guaranteeing stability of the robot as well as ensuring it can learn effective behavior is the key idea of this project.”

Reinforcement learning is like the hot/cold game you may have played as a child. One person knows where an object is in a room. When the other person, who is blindfolded, moves in a direction, the first says “hot” if they are getting closer to the object and “cold” if they’re moving away from the object, providing real-time feedback until the person is successful.

With Cho’s new method called CARL, which stands for Contraction-Aware Reinforcement Learning, the robot consistently stays on track and reaches its goal faster.

“CARL combines reinforcement learning with contraction theory to provide stability guarantees for its performance. Small tracking errors may occur, but the method provides mathematical limits on how large they can become. This allows reinforcement learning to serve as the robot’s main controller with stability guarantees.”

Cho began by simulating CARL then moved to the aerospace engineering flight lab to test it with multiple real robots.
“There is a difference between the model we train in the computer world to the physical robot in the real world. For example, a model in the simulator has a mass of 1 kilogram. In the real world, it may be 1.2 kilograms. This is called the sim-to-real gap, but our algorithm was shown to be better at handling those uncertainties.”

Cho ran experiments with three types of robots: a wheeled TurboBot 3 Burger, a flapping aerial drone, and a six-legged Jethexa robot. He created a video that overlays the robots’ progress with previous algorithms and CARL.

Watch a video of the robot testing at https://yutu.be/sOJ4hulbop0

“There was no guarantee that our method would work better than the existing methods, particularly with the increasing complexity of the robots we tested. But we found that, not only was integrating those two methods novel, but the performance achieved in the computer world is consistently better in the real-world testing as well, by up to 15%.”

Cho said one example of how this method can be applied is with wildfire management. First responders could send in robots to help identify hotspots or even deliver supplies.

CARL has open-access availability, with code also provided in GitHub.

The study, “Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustness,” written by Minjae Cho, Hiroyasu Tsukamoto and Huy T. Tran, is published in IEEE Transactions on Robotics. DOI:10.1109/TRO.2026.3727138