Google’s DeepMind uses Deep Reinforcement Learning to teach robots soccer skills and strategies
- DeepMind trained miniature humanoid robots with 20 controllable joints to play soccer using Deep Reinforcement Learning (Deep RL).
- The robots developed motor skills such as rapid fall recovery, walking, turning, kicking, and basic strategic understanding.
- The agents learned target joint angles using only the onboard computer and were tracked using a real-time motion capture system.
- The trained soccer agents exhibited agile and reactive behavior during gameplay, transitioning smoothly between various skills.
- Compared to scripted baseline skills, the Deep RL-trained agents walked 156% faster, took 63% less time to get up, and kicked 24% faster.
Google’s DeepMind has made a remarkable breakthrough by training miniature humanoid robots with 20 controllable joints to play soccer using Deep Reinforcement Learning (Deep RL). The robots developed motor skills such as rapid fall recovery, walking, turning, kicking, and even acquired a basic strategic understanding of the game.
The agents learned target joint angles solely from the onboard computer, while a real-time motion capture system was used to observe the robot’s and ball’s locations. The agents were rewarded for scoring goals and penalized for getting too close to the other robot. They were then trained in simulation using a deep reinforcement learning algorithm, where they learned essential skills like getting up from the floor and scoring goals.
During gameplay, the trained soccer agents displayed agile and reactive behavior, including getting up, turning, kicking a moving ball, and blocking defensively. These agents were able to transition quickly between various emergent skills and combine them effectively.
The performance of the Deep RL-trained agents was compared with scripted baseline skills in both simulation and real environments. The results showed that the agents walked 156% faster, took 63% less time to get up, and kicked 24% faster. These significant improvements in performance demonstrate the potential of Deep RL training.
DeepMind is now exploring the possibility of agents learning directly from vision. With the evident success of deep reinforcement learning in producing high-quality individual skills, the potential for its application extends beyond soccer to other real-world industries and tasks.