Growing numbers of mobile robots in warehouses and elsewhere have raised concerns about interoperability among systems from multiple vendors, as well as with facility infrastructure and enterprise software. One effort has its roots in the open-source community and the healthcare industry. The Open Robotics Middleware Framework, or Open-RMF, is designed to coordinate fleets of robots with typical use cases and integrate them with elevators and other systems.
“As more diverse robots are being deployed around the world, there is a growing need for specialized robots from different developers to be able to communicate with each other. A lingua robotica, if you will,” said Open Robotics in a blog post today.
Mountain View, Calif.-based Open Source Robotics Foundation Inc. was founded in 2012 and forms Open Robotics with its Singapore subsidiary. The nonprofit has developed the Robot Operating System (ROS) and the Ignition simulation software formerly known as Gazebo.
Hospital pain points lead to Open-RMF
Open-RMF arose out of efforts in 2018 to get autonomous mobile robots (AMRs) for cleaning, delivery, security, and other applications to operate in the same space within hospitals in Singapore. Productivity depends on such robots being able to communicate with one another and be orchestrated by human managers or higher-level systems.
“Our initial customer request was to coordinate the behavior of multiple AMRs from different vendors,” recalled Brian Gerkey, co-founder and CEO of Open Robotics. “Their top-level pain point was that they needed to share space more effectively. They wanted to put in more AMRs to move food and linens, as well as surgical equipment to the autoclave, but there's the constraint of space.”
“The traditional way to do it is to divide fleets by time or space,” he told Robotics 24/7. “You could put different robots in different corridors, but that's obviously inefficient. This extends to elevator banks, and one customer was even thinking of tunneling through the building to add elevators, even when they're not fully utilized.”
“The robots couldn't share them for mundane reasons—vendors have proprietary ways of working with elevators,” Gerkey said. “You could have a bolt-on kit or bridge to talk with the elevator, but robots waiting to get in or out will still perceive each other as obstacles, unless they're from the same vendor.”
“We spent a lot of effort on that initial request, and we have a well-defined specification for writing fleet adaptors,” he added. “We expect vendors and operators to put in fleets of robots. What we can tell the fleet management software and what it can tell us can vary quite a bit.”
Open Robotics sets hierarchy of interfaces
Open-RMF classifies AMRs based on how much data access each vendor is willing to provide.
“Whether it was technical limitations or a lack of willingness from AMR providers, we got very limited information,” said Gerkey. “We set a low bar—just tell me where the robots are at some frequency. We called this 'Red light, green light' in the API [application programming interface].”
“Then there's the different levels of information they can give back —position, trajectory, and time parameter or schedule,” he explained. “We can't tell the robots what to do, but they can show us where they're planning to go.”
“The next step up is to give commands, such as pause and resume. Then there's offering waypoints and telling the robots where to go,” Gerkey said. “At the highest level is direct teleoperation for troubleshooting.”
“On top of that, we built a traffic manager, with the ability to delegate tasks, depending on the fleet's ability to expose data,” he added. “Then we can get to more efficient use of the shared space. We'll roll out actual deployments in healthcare settings next year.”
Interoperability isn't just about fleets speaking with planning software or human operators; robots must also communicate with things like smoke alarms, Gerkey noted.
“If a fire alarm goes off, we need robots to go to the sides of hallways,” he said. “It will happen very rarely, but we absolutely have to build it in. That kind of thing will be easier once the industry has a standardized interface.”
In addition, the traffic manager can account for other equipment, such as a bed pushed by a human staffer. “It rolls into our system, so that robots know where it is and can plan to go around it,” said Gerkey.
Open-RMF and simulation
Simulation is necessary, not only for demonstrating the capabilities of Open-RMF, but also for training systems to work together.
“The biggest challenge for us on the simulation side is scaling up Gazebo to be able to model large, multifloor buildings with all kinds of detail and all kinds of people,” said Gerkey. “The point of simulation is that application developers can do tests and rely on the models as a base for future decisions.”
“Having simulation is absolutely vital to being able to test systems, especially when we're talking about interoperability,” he added. “We'll always have to test the physical hardware, but on the way there, the logistical and operational cost of any large-scale test is tremendous. In hospitals or warehouses, which are busy 24/7, if you can get an operator to carve out a corner of the building for half a day, you're lucky.”
Open Robotics' Ignition can help train robots to recognize large, diverse populations of people, such as those in wheelchairs or on crutches, as well as one another.
“UV disinfection robots are relatively new, but floor-cleaning robots have been available for a while,” observed Gerkey. “We're working with customers that have large enough facilities that they're running robots from different vendors for different cleaning needs. They're using Open-RMF, which doesn't care if it's a scrubbing or mopping robot or a vehicle carrying supplies.”
Other interoperability efforts
Other organizations such as MassRobotics, the Advanced Robotics for Manufacturing (ARM) Institute, the Association for Advancing Automation (A3), and the Robot Operations Group (ROG) have been discussing robot interoperability in heterogeneous environments. However, Open Robotics has been taking an open-source approach for a few years already, said Gerkey.
“Compared with companies in the robot operations, or RobOps software space, some of them are focused on ways to manage a single kind of fleet,” he said. “Other groups are looking to establish a standard. We're not really looking to dictate standards, which don't always work in the fast-moving world of robotics.”
“We are a member of MassRobotics and participated in its effort around a communications standard, which was demonstrated at A3's AMR & Logistics Conference in Memphis,” added Gerkey. “They're starting with having robots broadcast their locations. There are some technical differences in how they define their spec.”
“We're building a hierarchical interface, so developers can add algorithms on top of it,” he said. “There will be adapters to go back and forth between what we've got and they've got, but it's early days.”
An open-source approach has the advantage of wide community involvement, Gerkey said. “Open Robotics has shared its version of the interface, and others can adapt the working code and evolve it, without a standards committee consensus,” he said. “Open-RMF must work across as many systems as possible, and my hope is that companies will provide feedback.”
“It's like how AMRs were built on top of ROS,” Gerkey said. “We'd like to see companies build their analytics and dashboards on top of Open-RMF. We're providing the industry building blocks.”
“We're thrilled that people are talking about interoperability, which was our raison d'etre since the ROS project,” he said. “We have a well-defined set of abstractions, such as an interface for lidar. Rather than deal with individual drivers, people can mix and match lidar sensors, and the code doesn't change. It's the same with cameras and SLAM [simultaneous localization and mapping] systems.”
Users must demand interoperability
“The ROS user community hasn't paid as much attention to interoperability as I would like,” acknowledged Gerkey. “Having major customers such as FedEx talking about the need will drive new companies and existing AMR vendors to serve that specific need.”
“At the 2019 ROScon, when a hospital from Singapore delivered a keynote, it was one of the first times a customer got on stage and said, 'I want to use all these robots and can't because they won't talk with one another,'” he said. “That same year, we met with an AMR vendor and a major automotive manufacturer that had said the same thing. The AMR vendor said, 'We have no interest in sharing maps.' That mindset showed that many AMR vendors hadn't seen an incentive to working together.”
“Bigger robotics players will say, 'Buy our entire integrated product line,' but even they can't supply every kind of robot,” Gerkey said. “Nobody wants to be locked in. The pull from customers is changing the situation for the vendors.”
“It reminds me of the early days of the ROS Industrial effort, which started at the Southwest Research Institute,” he observed. “They were tackling the problem of robot arms in industrial settings, which all had different interfaces. Companies had to rewrite each application in different programming languages and needed an interface on top.”
“ROS provided abstraction so that more powerful applications could be portable,” said Gerkey. “Smaller players like Yaskawa said, 'Yes, please,' because they saw it as a way to gain market share. Now, 10 years later, every major robot vendor can work with ROS.”
Algorithms to operate across fleets
Can Open-RMF be applied to other domains? “Absolutely,” Gerkey replied. “We're prototyping with customers in airports for cleaning and delivery robots and with seaports for larger-scale equipment outdoors. It's basically the same problems of autonomy at different levels. Companies need the best robots for each task, but they need to combine them to work together efficiently.”
“We're going a step higher—it doesn't matter what brand an AMR is; we just want it to report its location and status and then take fleet-level commands,” he said. “Algorithms are what's really powerful in the long run.”
“The first thing people want is situational awareness, like in the demo in Memphis,” said Gerkey. “The next step is to tell robots what to do and make better use of all of them at once. Our traffic management system has great potential, but companies could do more on top of that.”
“Open Robotics wants to reduce risk for operators and lower the barriers of entry for vendors,” he said. “We can't ask them to rewrite the software stack, and our adapter assumes nothing about the existing system. The fleet manager uses ROS 2 to communicate in a standard way across fleet managers and to higher-level algorithms like traffic management.”
“With Open-RMF, we want to make interactions planned and reasoned,” Gerkey said. “A robot should never be surprised to turn a corner and see a different robot.”
About the Author
Follow Robotics 24/7 on Linkedin
Article topics
Email Sign Up
Related Interoperability News
