Supply chain challenges and labor shortages have driven more industries to adopt automation, but unfamiliarity and complexity are often barriers to deployment. At CES in Las Vegas today, NVIDIA Corp. announced major updates to Isaac Sim, its tool for building and testing virtual robots across varied operating conditions in simulated environments.
The installed base of industrial and commercial robots will grow by more than sixfold—from 3.1 million in 2020 to 20 million in 2030, according to ABI Research.
“Broadly speaking, the challenge [for robotics developers] falls into two categories,” said Deepu Talla, vice president and general manager for embedded and edge computing at NVIDIA. “The first is continuous development and operations of robots.”
“The second is runtime on physical robots,” he added. “DevOps is many times more difficult, and we've been adding tools for simulation, which is the biggest part. We've been working on the Isaac robotics platform for over five years.”
About Isaac Sim
“We believe simulation is key to robotics, providing a virtual proving ground,” said Gerard Andrews, senior product marketing manager for robotics at NVIDIA. “We provide the Jetson compute platform and support reference robots and the cloud.”
“Isaac Sim is our end-to-end robotics platform that we have designed to accelerate all aspects of robotics development and deployment,” he explained. “The latest version is focused on improving performance and functionality for manufacturing and logistics robotics use cases.”
The NVIDIA Omniverse application enables developers to import robot models and build realistic environments to validate both physical designs and software stacks for performance. Isaac Sim can also generate synthetic datasets to train the AI models for robotic perception.
Developers can then use Isaac Sim's reinforcement learning API to train models in a robot's control stack, said NVIDIA during its special address at CES.
Adding people to realistic simulations
As more people work alongside collaborative robot arms or autonomous mobile robots (AMRs), it's critical that common behaviors are added to simulations, NVIDIA claimed.
The Isaac Sim 2022.2 release gives users the ability add people to environments for safety simulations. They can task characters with executing pre-supported behaviors such as stacking packages or pushing carts.
Isaac Sim now includes new 3D assets for physically accurate simulated environments, including warehouse parts and conveyor belts.
The Isaac ecosystem includes 1,000 organizations and 1 million developers, according to Talla. It includes robots, computers, and sensors used in agriculture, manufacturing, and retail.
“Too often, 3D assets are built for visualization, not physics accuracy,” Talla noted. “By augmenting real-world data with labeled synthetic data, you can vastly accelerate development and rapidly train, adapt, and optimize models.”
Isaac Sim can also render accurate data from RTX-simulated sensors, including those from Ouster and Hesai, in real time. For instance, “in the case of an RTX-simulated lidar, ray tracing provides more accurate sensor data under various lighting conditions or in response to reflective materials.” said NVIDIA.
The company said it has added capabilities to Isaac Gym for reinforcement learning and Isaac Cortex for cobot programming. In addition, the new Isaac ORBIT tool provides simulated operating environments and benchmarks for robot learning and motion planning.
“What was research a year or two ago will bleed into production,” Andrews said. “This is a real feature-rich release of Isaac Sim.”
Supporting the community with ROS and the cloud
For the Robot Operating System (ROS) community, Isaac Sim also comes with native support for ROS 2 Humble and Windows. All of the Isaac ROS software is now available for simulation, said NVIDIA.
NVIDIA noted that since Isaac Sim is available in the cloud, multidisciplinary teams across locations can more easily collaborate on testing and training of virtual robots.
With Isaac Replicator, robotics developers can generate synthetic data and use Isaac Sim to “create massive ground-truth datasets that mimic the physics of real-world environments,” said Andrews.
NVIDIA cuOpt is a real-time, route-planning engine and fleet task-assignment tool for increasing operational efficiencies, he added.
“Our goals are to close the simulation-to-reality gap, be good at more than one thing, embrace AI as a key enabler of autonomy, and simulate fleets of robots at scale,” Andrews said.
Isaac ecosystem expands
Isaac Sim 2022.2 includes two new mobile robots and 10 virtual manipulators, including popular models from Clearpath and Kawasaki, said Andrews.
“As robots get more complex, they need to be constandly rechecked and validated in configuration,” he added. “We're working with leading-edge innovators to figure out new use cases.”
NVIDIA cited several examples of companies across industries using Isaac Sim. For instance, Deutsche Bahn is training AI models to handle “corner cases” that happen rarely in reality but are still important, such as luggage falling on a train track.
Amazon Robotics is using Isaac Sim to train soft robot grippers and Omniverse for warehouse digital twins, while ETH Zurich is training a wheeled quadruped robot, said Andrews.
Fraunhofer is developing advanced AMRs using the physically accurate and full-fidelity visualization features of Isaac Sim.
Festo uses Isaac Cortex to simplify programming for cobots and transfer simulated skills to the physical robots. Flexiv is using Isaac Replicator for synthetic data generation to train AI models.
While training robots is important, simulation is playing a critical role in training the human operators to work with and program robots, said NVIDIA. Ready Robotics is teaching how to program industrial robots with Isaac Sim.
Sarcos Robotics is developing robots to pick and place solar panels in renewable energy installations. Telexistence deployed beverage-restocking robots across 300 Family Mart convenience stores in Japan.
Cobot leader Universal Robotics is using Isaac Sim for workforce development to train end operators from the cloud.
NVIDIA also touted six companies that have used its Jetson edge AI platform, including Agrist for its bell pepper-harvesting robot, John Deere for its autonomous tractor, Skydio for its Scout drone, Neubility for its delivery robot, and Seoul Robotics for its self-driving car “control tower.”
These companies have also received recognition with CES 2023 Innovation Awards, Talla said.
NVIDIA makes more simulation tools available
Also at CES, NVIDIA also announced a major update for Omniverse Enterprise to improve performance, improve RTX ray and path tracing, and streamline workflows. It has opened Omniverse portals with more Blender AI add-ons, Ada Lovelace support, and thousands of free 3D USD assets for GeForce RTX users.
The company also announced the NVIDIA Omniverse Avatar Cloud Engine, or ACE, as well as RTX 40 Series laptops. It noted that Mercedes Benz is using Omniverse digital twins to plan and build more efficient factories as the automotive industry transitions to electric and autonomous vehicles.
Speaking of autonomous vehicles, NVIDIA announced a partnership with Foxconn to build DRIVE Orin computers and integrate the Hyperion sensor architecture into electric vehicle fleets. The company is also relying on its relationship with video game developers for in-vehicle experiences. NVIDIA said its simulation tools will help developers of both autonomous vehicles and robotics.
“Isaac Sim provides the ability to simulate tens or even thousands of robots, with multiple instances of virtual robots in the cloud,” said Talla. “Soon, it will also address simulation of multiple robots in one instance.”
“Digital twins are becoming a must-have developer tool,” he concluded. “There's something for everyone in the new Isaac Sim. NVIDIA is arguably investing more than anyone else in the world, and we look forward to sharing more.”
About the Author
Follow Robotics 24/7 on Linkedin
Article topics
Email Sign Up
Related Simulation News
Related Companies
Related Resources
