By Víctor Mayoral-Vilches
When developing a robot, system integration consumes arguably the largest proportion of the project’s resources; much more so than developing the end application. With the advent of lower-end industrial collaborative robots, companies have emerged that focus solely on developing software to run on existing hardware. However, there is a critical relationship between the hardware and the software capabilities in a robot. Retaining design control over the computing hardware is needed to create more specialized, power-efficient, secure, and high-performing robots.
Hardware Challenge. Software Skills
But there is an obstacle that must be overcome if roboticists are to deliver the better and faster robots that will be required in the future. In today’s post-Moore computing world, upgrading hardware to adopt the latest-generation microprocessor cannot deliver the desired application-performance upgrade. The way forward no longer lies with waiting for the latest chip lithography. Hardware acceleration is often the only way to achieve the necessary gains.
This hardware challenge complicates life for developers in disciplines like robotics, whose skills tend to be software biased. It means they must face the prospect of designing adaptive computational hardware if they are to satisfy market demands for new industrial robots. End-users – the companies that operate robots to raise productivity in areas like production lines and warehouses – are looking for equipment that offers extra dexterity, finer positional control, superior vision-based capabilities, improved data capture, and lower power consumption.
The main principle underlying hardware acceleration in robotics is that a mixed control- and data-driven approach for software development, unlike the traditional control-driven approach, lets teams design custom compute architectures that allocate the optimal amount of hardware resources for an application.
As far as implementation is concerned, a heterogeneous computational model is needed. This takes advantage of the strengths of CPUs and GPUs, which excel in control-flow computations, while leveraging the strengths of FPGAs to handle data-flow computations. This approach simultaneously delivers the flexibility and full control of CPUs/GPUs to implement complex computations, with the low power, high performance, low latency, and deterministic nature of hardware acceleration. Various vendors are now offering adaptive System-on-Chips (SoCs) and System-on-Module (SOM) devices – like the AMD-Xilinx Kria™ SOM and its associated Kria Robotics Stack – that deliver the advantages of such a mixed computational model. The table compares these different models.
The adaptive SoCs and SOMs allow roboticists to build machine behaviors by programming an architecture that creates the right data paths and control mechanisms. However, complex engineering skills are needed to program such architectures using established tools and techniques.
Suitable expertise in hardware and embedded design is scarce among roboticists, who are accustomed to building behaviors in the form of computational graphs that solve the robotic task at hand. They often use C++ to create complex real-time deterministic systems through advanced software engineering practices.
Building on ROS
A different approach is now needed to help roboticists take advantage of the hardware acceleration technologies available. Ideally, such an approach should let them create their custom hardware working within a familiar development environment – such as ROS – and using familiar tools – such as Gazebo for simulation.
ROS is the de facto industry standard for robot application development and even more so since the arrival of ROS 2 in 2020. This has become the default Software Development Kit (SDK) for robotic applications across industries. Many groups are now using ROS and Gazebo.
Previous initiatives to integrate adaptive computing into ROS have tackled the challenge from a hardware engineer’s perspective. They have assumed that the user has previous experiences with embedded and hardware flows and therefore is familiar with concepts like RTL, HDL, and HLS, and the design tools used to manipulate them. Similarly, deployment into embedded targets requires one to be somewhat familiar with Yocto, OpenEmbedded, and related tools.
Understanding that most roboticists do not come from this background, the ROS 2 Hardware Acceleration Working Group (HAWG) is taking a ROS-centric approach to integrate embedded flows directly into the ROS ecosystem. Its goal is to provide an experience similar to that enjoyed by roboticists when they build ROS workspaces in their desktop workstations.
The HAWG work builds on published research into optimizing the ROS computational graph to leverage adaptive computing, as well as proposals for tools and methodologies to accelerate parts of the graph in programmable logic. Adding to this, HAWG is now putting forward an architecture (figure below) that focuses on familiar languages like C++ and OpenCL.
The proposed architecture aims to be not only platform-agnostic, and therefore suitable for edge, workstation, data center, and/or cloud targets, but also technology-agnostic to allow targeting FPGAs, CPUs, and GPUs as well as easily portable to various modules and boards.
Ultimately, this work should enable the majority of roboticists to take advantage of the opportunities for hardware acceleration to realize future generations of advanced and sophisticated robots.
About the author
Víctor Mayoral-Vilches is the Robotics System Architect at Adaptive & Embedded Computing Group, AMD. With a strong technical background in robotics, embedded systems and cybersecurity, he spent the last 10 years building robots. Prior to joining AMD-Xilinx, he founded and led three robotics startups building teams of 30+ engineers and leading them across research initiatives and projects in the fields of robotics, cybersecurity and artificial intelligence. Víctor was selected as one of the ten most innovative individuals under 35 in Spain by the MIT Technology Review in 2017 and holds multiple national expert positions representing Spain in ISO and IEC committees for standards in robotics, safety and cybersecurity.