Introducing Learned Natural Walking
- Revolutionizing Humanoid Locomotion: We present an end-to-end neural network trained with reinforcement learning (RL) to enable humanoid robots to walk naturally like humans.
- Sim-to-Real Success: By leveraging high-fidelity simulations and domain randomization, our policies transfer seamlessly from simulation to real-world robots without additional tuning.
- Scalable and Robust: Our approach enables consistent, human-like walking across a fleet of robots, paving the way for scalable and efficient humanoid robotics.
Humanoid robots have long been a symbol of technological progress, but achieving natural, human-like walking has remained a significant challenge. At Figure, we’ve developed an innovative approach to humanoid locomotion using rHumanoid robots have long been a fascinating area of research, but achieving natural, human-like walking has remained a significant challenge. Recently, researchers have made a major breakthrough by using reinforcement learning (RL) to teach robots how to walk in a way that closely mimics human locomotion.
This innovative approach involves training robots in high-fidelity simulations and transferring the learned behaviors directly to real-world hardware. The result is a scalable and robust solution for natural humanoid walking, with potential applications in industries ranging from automation to healthcare.
Reinforcement Learning: The Key to Natural Walking
Reinforcement learning is a cutting-edge AI technique where a controller learns through trial and error, guided by a reward system. In this case, researchers used RL to teach a humanoid robot, known as Figure 02, how to walk.
The training process took place in a GPU-accelerated physics simulator, where thousands of virtual humanoid robots were trained in parallel. Each robot was exposed to a variety of scenarios, such as uneven terrains, slips, and external disturbances. By simulating years of walking data in just a few hours, the RL controller learned robust and adaptive walking strategies.
The outcome is a single neural network policy capable of controlling all these robots, ensuring they can handle a wide range of real-world challenges.
Engineering Robots That Walk Like Humans
The goal of humanoid robots is to perform tasks in human environments, making natural walking a critical feature. However, teaching a robot to walk naturally involves more than just forward motion—it requires replicating the stylistic nuances of human gait.
To achieve this, researchers incorporated human walking reference trajectories into the RL framework. These trajectories guide the robot’s movements, ensuring it walks with heel strikes, toe-offs, and synchronized arm swings, just like a human.
In addition to style, the training process optimized for practical factors such as energy efficiency, velocity tracking, and robustness to external disturbances. This combination of stylistic and functional rewards ensures the robots not only look human-like but also perform reliably in real-world conditions.
Bridging the Sim-to-Real Gap
One of the most significant challenges in robotics is transferring behaviors learned in simulation to real hardware. Simulated robots, no matter how advanced, are only approximations of their physical counterparts. To address this, researchers employed two key techniques:
- Domain Randomization: In simulation, the physical properties of the robots, such as mass, friction, and actuator dynamics, were randomized. This exposed the RL policy to a wide range of conditions, enabling it to generalize to real-world variations.
- High-Frequency Torque Feedback: On the physical robot, a kHz-rate closed-loop torque control system was implemented. This compensates for any discrepancies between the simulated and real robots, ensuring smooth and reliable walking.
These techniques allow the learned policies to transfer “zero-shot” from simulation to real robots, meaning no additional tuning is required. This is a significant breakthrough, as it enables rapid deployment across an entire fleet of robots.
Scaling to a Fleet of Humanoid Robots
The scalability of this approach is one of its most impressive features. The same RL policy powers multiple Figure 02 robots, all walking naturally and consistently without any individual adjustments. This scalability is critical for commercial applications, where fleets of robots must operate reliably in diverse environments.
By reducing the need for manual tuning and engineering effort, this RL-driven training process accelerates development cycles and ensures robust performance across the board. This positions the technology to scale to thousands of robots in the near future.
The Future of Humanoid Robotics
This development represents a significant step forward in humanoid robotics. By combining reinforcement learning, high-fidelity simulation, and advanced control techniques, researchers have created a system that enables natural, human-like walking.
While this is a major milestone, it is only the beginning. The technology has the potential to expand further, enabling robots to handle a wide range of human-like scenarios, from navigating crowded spaces to climbing stairs. The possibilities for humanoid robotics are vast, with potential applications in industries such as manufacturing, logistics, and healthcare.
This breakthrough highlights the power of reinforcement learning and its ability to solve complex challenges in robotics. As the technology continues to evolve, it could redefine what humanoid robots are capable of achieving.