NVIDIA Corp. today announced that it is working with Open Robotics to improve ROS 2 performance on its Jetson platform for artificial intelligence at the edge and its GPU-based systems. The company said this initiative will enable robotics developers to more easily and quickly incorporate computer vision and machine learning functionality into systems based on the Robot Operating System (ROS).
“We've been working closely with Open Robotics to bring a host of NVIDIA AI technologies to the ROS community,” said Murali Gopalakrishna, head of product management, autonomous machines, and general manager for robotics at NVIDIA. “We want to make it seamless for developers to leverage the GPU and deep learning accelerators, along with other compute resources on NVIDIA hardware, in ROS.”
The global market for robots will be $210 billion by 2025, according to Statista. “That a 26% to 30% clip,” Gopalakrishna told Robotics 24/7. “We believe this to be a conservative estimate based on the growth we have seen driven by COVID-19 and labor shortages.”
“At the same time, robotics developers face challenges with system complexity, unstructured environments, and complex sensor configurations,” Gopalakrishna said. “NVIDIA is working to make it easier for them to build realistic and scalable models, train machine learning algorithms using synthetic data, and deploy models on the edge.”
Open Robotics to extend ROS for NVIDIA AI
“As more ROS developers leverage hardware platforms that contain additional compute capabilities designed to offload the host CPU, ROS is evolving to make it easier to efficiently take advantage of these advanced hardware resources,” stated Brian Gerkey, CEO of Open Robotics.
Open Robotics said it will enhance ROS 2 to enable efficient management of data flow and shared memory across graphics processing units (GPUs) and other processors on the NVIDIA Jetson edge AI platform. This will improve the performance of applications that have to process high-bandwidth data from sensors such as cameras and lidars in real time, it said.
In addition, Open Robotics and NVIDIA are working to enable simulation interoperability between Open Robotics’s Ignition Gazebo and NVIDIA Isaac Sim on Omniverse. Isaac Sim already supports ROS 1 and 2 out of the box and features an ecosystem of 3D content with its connection to popular applications such as Blender and Unreal Engine 4.
With the two simulators interoperable, ROS developers can easily move their robots and environments between Ignition Gazebo and Isaac Sim to run large-scale simulations, said NVIDIA. They can also take advantage of each simulator’s advanced features such as high-fidelity dynamics, accurate sensor models, and photorealistic rendering to generate synthetic data for training and testing of AI models.
“Developers could take domain-specific data for, say, a hospital, a warehouse, or a store,” Gopalakrishna explained. “They could use synthetic data generated in Isaac Sim, combine it with real-world data, and train a model to be robust in the robot's operating environment. We already offer pre-trained models to reduce the time and expense of training machine learning models.”
NVIDIA and Open Robotics said they expect software resulting from this collaboration to be released in the spring of 2022.
NVIDIA releases Isaac GEMs for ROS
Isaac GEMs for ROS are hardware-accelerated packages intended to make it easier for ROS developers to build high-performance solutions on the Jetson platform.
The focus of these GEMs is on improving throughput on image processing and on perception models based on deep neural networks (DNNs). These packages can reduce the load on the host CPU while providing significant performance gain, said NVIDIA.
“DNN models can be used as a plug-in to develop robots,” said Gopalakrishna. “Developers could take domain-specific data for, say, a hospital, a warehouse, or a store. They could deploy a diverse set of models in real time.”
The new Isaac GEMs for ROS include:
- SGM stereo disparity and point cloud
- Color space conversation and lens distortion correction
- AprilTags detection
“We now have over 850,000 developers on Jetson, whihch has become the de facto edge AI compute platform for many mobile robot applications,” said Gopalakrishna. “From our $99 Jetson Nano all the way up to Jetson AGX Xavier for higher-end autonomous machines and on up to the cloud, they can leverage the same algorithms.”
New Isaac Sim features for ROS developers
The latest release of Isaac Sim includes significant support for the ROS developer community. Some examples of this are the ROS2 Navigation stack and the MoveIt Motion Planning Framework. These examples are available today and can be found in the Isaac Sim documentation. Here are examples of ROS in Isaac Sim:
- ROS April Tag
- ROS stereo camera
- ROS navigation
- ROS TurtleBot 3 sample
- ROS manipulation and camera sample
- ROS services
- MoveIT motion-planning framework
- Native Python ROS usage
- ROS 2 navigation
Isaac Sim generates synthetic data for perception training
In addition to being a robotic simulator, Isaac Sim can generate synthetic data to train and test perception models, said NVIDIA. These capabilities will be more important as roboticists incorporate more perception features into their platforms. The better a robot can perceive its environment, the more autonomous it can be, the company said.
Once Isaac Sim generates synthetic datasets, they can be fed directly into NVIDIA’s TAO Toolkit to adapt perception models for a robot’s specific working environment. The task of ensuring that a robot’s perception stack is going to perform in a given working environment can be started well before any real-world data is ever collected from the target surroundings, according to the company.
Roboticists have faced challenges in connecting and integrating robotic tasks like navigation to AI-based perception stacks, said NVIDIA. Isaac Sim addresses this workflow challenge by being simultaneously a full robotics and synthetic data-generation tool with streamlined integration with the TAO training platform.
“Anything built will have a digital twin, anything that moves will be autonomous, and anything autonomous will be simulated,” said Gopalakrishna.
NVIDIA to share more at events
Gopalakrishna said NVIDIA is preparing for ROS World, which was originally planned for Las Vegas but will now be online on Oct. 21 and 22. The company is planning to release more new GEMs for Jetson developers, including several popular DNNs.
NVIDIA also plans to announce features in Isaac Sim to support the ROS developer community. The company will have a virtual booth, a NVIDIA ROS roundtable, and a technical presentation on Isaac Sim.
“We recently had a Robotics Day for customers, and 5,000 people including startups registered for it,” said Gopalakrishna. “Startups can apply to join the NVIDIA Inception accelerator, which has 8,500 AI and data science startups. We register new ones every week.”
Over the 12 months ending in March 2021, venture firms poured $6.3 billion into robotics companies, up nearly 50% from the $4.3 billion they invested in the prior 12-month period, according to an analysis by venture-capital database PitchBook for Forbes.
“The funding for robotics startups is unprecedented,” Gopalakrishna said. “I believe that startups need to lean more into AI. Then they can solve problems in a more meaningful way.”
NVIDIA will also discuss various technologies at its GPU Technology Conference (GTC) scheduled for Nov. 8 to 11. A track for robotics developers will include a presentation by Open Robotics' Gerkey. In addition, the company will have presentations covering NVIDIA Jetson, Isaac ROS, Isaac Sim, and Isaac GYM.
About the Author
Follow Robotics 24/7 on Linkedin
Article topics
Email Sign Up
Related Simulation News
Related Companies
Related Resources
