This article is part of our coverage of the latest in AI research.
While there has been impressive progress in humanoid robotics, most advanced platforms are either too expensive or unavailable for research labs. This makes it difficult to experiment with and improve on the state-of-the-art.
To address this challenge, a team of researchers at UC Berkeley has developed Berkeley Humanoid, a mid-scale humanoid robot platform designed for affordability, modularity, ease of repair, and a tight sim-to-real gap, making it suitable for research and experimentation in academic settings.
The need for better humanoid research platforms
Commercial humanoid robots, though highly advanced, are usually too expensive for academic budgets and provide limited access to modify low-level hardware and software. At the same time, building a new humanoid robot from scratch is an expensive and time-consuming endeavor.
“We need better robots for research that are easy to work with and easy to maintain,” Qiayuan Liao, a PhD student at UC Berkeley and lead author of the Berkeley Humanoid paper, told TechTalks. “We believe we can build robots with better performance, transparent low-level actuators and transmission, and lower cost.”
This motivated the Berkeley team to create a mid-scale humanoid platform tailored for research into learning-based control algorithms.
The Berkeley Humanoid is a 16-kilogram, fully electric, mid-scale humanoid robot built almost entirely from custom-designed parts, including actuators, transmissions, sensors, and the control board.
“This comprehensive understanding of the whole system enables us to explore new control strategies with a narrow sim-to-real gap, as well as accounting for specific requirements on the hardware from learning-based algorithms,” the researchers write.
The researchers removed complex components from the robot’s design, such as springs, dampers, and closed kinematic chains, which add to the cost and complexity of simulating the robot’s physics and movements.
The simple mechanical design enables the researchers to run simulations of the robot at very high speed. This significantly accelerates the process of training learning-based control models. On an Nvidia A4500 GPU, the robot can be simulated at a speed of more than 90,000 steps per second.
Without arms, the Berkeley Humanoid costs around $10,000 to manufacture, which is significantly cheaper than other comparable robots, which can cost upward of $100,000.
“The Berkeley Humanoid holds significant value for research in academic labs, allowing researchers to quickly develop and test their algorithms,” Liao said. “Its design is tailored to support rapid experimentation and iteration, making it a valuable asset for advancing learning-based control techniques.”
Another important design goal of the Berkeley Humanoid was to make it easy and safe to operate. The robot can be easily maneuvered and reset by a single operator for indoor environments.
The Berkeley Humanoid uses a simple model-free reinforcement learning (RL) model that takes the robot’s sensor data as input and outputs motor commands. They further simplified the training process by minimizing the dimensions of the observation and action spaces of the RL agent, which reduces the computational cost of training.
The researchers trained the model in the Isaac simulated environment, using proximal policy optimization (PPO), a popular RL algorithm.
According to the researchers, the simple hardware and software design of the Berkeley Humanoid helped reduce the sim-to-real gap. The sim-to-real gap is the difference between the simulation environment where the robotics control models are trained and the real world where it is deployed. It remains one of the biggest challenges of applying learning-based control models in real-world settings.
“Our policy relies on the synergy of the hardware and learning algorithm to achieve a narrow sim-to-real gap, ensuring that the robust and agile locomotion performance in training can be fully demonstrated on the real-world robot,” the researchers write.
Berkeley Humanoid in action
To evaluate the Berkeley Humanoid, the researchers tested its locomotion capabilities in both indoor and outdoor environments. The robot was able to successfully walk on different surfaces, including grass fields, sidewalks, unpaved trails, asphalt roads, and bridges. It was also able to navigate inclines and stairs.
If you enjoyed this article, please consider supporting TechTalks with a paid subscription (and gain access to subscriber-only posts)
“We are surprised to find that our robot is able to climb a relatively steep and narrow unpaved trail covered with dust and rocks,” the researchers write. “This trail is a bit steep to climb even for adults, let alone our robot which resembles only a 5-year-old child in size.”
The robot also demonstrated robust recovery capabilities. During tests, it was able to regain its balance and continue walking after being kicked in different directions, both in indoor and outdoor environments.
The tight sim-to-real gap observed during testing confirmed the researchers’ hypothesis about the effectiveness of the robot’s simple hardware and software design.
“We have demonstrated the feasibility and methodology of designing and constructing a robust, high-performance robot platform within an academic laboratory setting,” Liao said. “We hope many of the academic labs will find this helpful and be excited to see such a platform.”
The researchers plan to further enhance the Berkeley Humanoid by adding arms for manipulation tasks, integrating cameras for perception, and improving its power system to allow for longer operation times and more dynamic movements.
“As a reliable, low-cost research platform, the ultimate goal is to deploy scalably for learning in the real world,” the researchers write.