As developers build robots that adapt to their environments, handle diverse items, or interact safely with people, they need tools that facilitate the training of machine learning systems. NVIDIA Corp. today announced the availability of the 2022.1 release of NVIDIA Isaac Sim.
“With Isaac Sim, developers can generate production-quality datasets to train AI perception models,” wrote Gerard Andrews, senior product marketing manager for robotics at NVIDIA, in a blog post. “Developers will also be able to simulate robotic navigation and manipulation, as well as build a test environment to validate robotics applications continually.”
NVIDIA works to close 'reality gap'
With its NVIDIA Omniverse applications, the Santa Clara, Calif.-based company has focused on ever more realistic simulation and digital twins to accelerate artificial intelligence for robots.
“This follows our strategy for end-to-end robotics,” Andrews told Robotics 24/7. “We segment the intersection of robotics and AI into four phases of development: No. 1, train with Isaac Replicator and synthetic datasets. No. 2, simulate with digital twins, which are the focus of today's release.”
“No. 3, build using Jetson Orin, Isaac ROS, SDKs [software development kits], and conversational AI, and No. 4, deploy and manage using our suites for fleets of robots in warehouses and other industrial settings,” Andrews said. “It's really about safer simulation for hazardous environments, faster training in parallel and with GPU-accelerated GEMs, and lower cost than testing in the real world.”
Why don't more roboticists use simulation? “Often, it doesn't model the behavior of a robot closely enough,” replied Andrews. “Once the reality gap is too large, developers would rather go to the lab, even if that's slower, smaller-scale, and more expensive.”
“With this release of NVIDIA Isaac Sim, we're delivering on incrementally closing that reality gap,” he said.
Isaac Sim 2022.1 release features
Isaac Cortex is a decision framework for training collaborative robots. “Program cobot tasks as easily as programming game AI,” said Andrews.
“Leverage this decision framework for cobots to develop task-aware and adaptive skills,” he said. “Using its belief representation of the world, analogous to the robot’s brain, real or simulated data can be used as inputs, and the resulting actuations will be generated.”
Andrews cited the example of working with Festo on a picking system (see video below) and said it's now available.
Isaac Gym is a reinforcement learning framework accelerated by NVIDIA's graphics processing units (GPUs; see video above). It keeps the entire learning workflow on the GPU to reduce training time, Andrews wrote.
“ETH Zurich/Swiss Mile created a wheeled version of ANYbotics' ANYmal and trained a new locomotion policy in Gym,” he said. “We had a standalone version for reinforcement learning, but it's now available as part of Isaac Sim.”
Omnigraph is a visual programming and scripting environment intended to simply application development and debugging.
“Connecting nodes with different inputs and outputs to drive compute tasks translates well to robotics applications, which already tend to be modular,” noted Andrews. “As with passing ROS [Robot Operating System] nodes, this works well with visual programming.”
Isaac Sim/Gazebo Connector is a feature to allow ROS developers to move between Gazebo and Isaac Sim for tasks such as generating synthetic datasets or high-fidelity rendering, said NVIDIA. In addition, users can keep multiple Gazebo simulations synchronized by connecting to Omniverse's nucleus server.
“The community can use any simulator as needed,” Andrews said. “The option of the high-fidelity Isaac Sim gives flexibility to developers.”
Isaac Replicator is a set of synthetic data-generation (SDG) tools, application programming interfaces (APIs), and workflows. It allows robots to be trained in minutes instead of weeks, claimed NVIDIA.
In its latest release, the company added the SceneBlox SDG feature to Replicator to generate industrial environments procedurally. It includes modules for conveyors and adds examples on how to generate synthetic data and train a pose-estimation model.
Developers to get more support
In addition, NVIDIA Isaac Sim 2022.1 includes limited support for Windows. It previously supported only Linux, but some customers asked for Windows, explained Andrews.
The Omniverse application also supports new robots, including the A1, GO1, and ANYmal quadrupeds, as well as the Fraunhofer Institute's O3dyn (formerly “Obelix”) indoor/outdoor pallet-moving system.
“Fraunhofer's AMR [autonomous mobile robot] is omnidirectional, can move fast, and can pick up heavy loads,” said Andrews. “It has plans to open-source the platform.”
“Our panel on digital twins at Automate last week was well-received,” he recalled. “The panelists from companies like Kroger and our newly announced partnership with READY Robotics demonstrate that the technology is becoming really important to diverse users.”
“Our discussions with them and people at our booth showed that it's not just the technology providers—end users are starting to think about how digital twins and simulation can help them improve efficiency, save money and energy, and make a smaller climate impact,” said Andrews.
“The overall theme of this release is that NVIDIA is providing a rich toolbox for developers to close the simulation-to-reality gap,” he said.
About the Author
Follow Robotics 24/7 on Linkedin
Article topics
Email Sign Up
Related Simulation News
Related Companies
Related Resources
