When people think of cobots, they usually picture people and robots working side by side in a shared space. But collaborative robotics also includes the fields of supervised autonomy and teleoperation.
Plus One Robotics creates machine vision and AI computing systems for parcel picking applications. In addition to its AI platform PickOne, the company provides its customers with a remote supervision service called Yonder.
Plus One describes Yonder as a human-in-the-loop automation software suite for robotic applications. Through its Yonder robotics-as-a-service (RaaS) offering, Plus One’s crew chiefs serve as remote supervisors who can step in 24/7 when automated systems need human assistance.
Erik Nieves, CEO of Plus One Robotics, recently discussed the nuances of remote monitoring for autonomous robot arms with Robotics 24/7. Nieves explained what degree of freedom Plus One gives to its AI control systems, as well as the types of situations that warrant human-in-the-loop intervention.
A human in the loop to help confused robots
Plus One Robotics recently celebrated one billion picks performed by its machine vision systems. Yet even with a billion picks worth of data to feed into its AI models, Nieves said “We're still the ones telling you AI is still going to disappoint you on occasion.
“We built the company on the premise that the AI - as good as it ever was going to be - would not be good enough,” Nieves said. “And so you needed to have a human in the loop.”
Plus One’s AI software, PickOne, runs on edge iPCs, allowing the robot arms to perform the vast majority of their picks autonomously. When the autonomous system does not reach a confidence threshold, it invokes Yonder to request human operator intervention. A crew chief will see what the robot sees and tell it which parcel to pick next.
“When a robot stops, seldom is it broken. Most of the time, it's just confused,” Nieves said.
Nieves said there are two ways to improve efficiency and allow one crew chief to remotely monitor more robots. One way is to reduce the duration of each intervention. Interventions by Yonder crew chiefs usually take about six seconds. “They're not going to get much shorter than they are now,” he added.
The other way to improve efficiency is to reduce the number of intervention requests from the robots. With each intervention, Plus One collects data. “The most high-value pieces of data any system can have are the edge cases,” Nieves said.
Plus One then curates those human interventions, and adds data tags and attributes to help its AI model learn. When they release their next model, the AI might have learned how to process types of packages it hadn’t encountered before. As a result, the robots will be less likely to request intervention when they encounter those types of packages again.
Robots shouldn’t solve every problem on their own
With intervention rates coming down and the number of robots overseen by each crew chief going up, warehouses could be on a path toward complete autonomy. While Plus One has deployed its systems globally to solve real-world warehouse problems, Nieves said achieving a “dark warehouse” without a human in the loop is not sustainable.
Suppose a warehouse trained their AI vision system until it learned every SKU in the building, then started running entirely autonomously.
“This is going to work for about two days,” Nieves said. “And then there's going to be some new package that you have to deal with, or there's going to be the same SKU - the same item - but now it's wrapped in different promotional packaging.”
Nieves explained the rate of change in a warehouse never reaches zero, which means an AI model is not likely to achieve full compliance. “Variability is the rule in the warehouse. Therefore, thou shalt have a human in the loop.”
Variability is not the only reason Plus One doesn’t feed every intervention back into its AI model. In fact, they cull their data so the AI doesn’t learn how to do things it shouldn’t.
“We do not want the AI to learn from every one of these interventions.” Nieves said. “When you get into real life, there are instances of errors in the world that you would not want the robot to learn from.”
For example, if a box opens up and spills its contents across the workcell, the robot might decide to pick up each individual item and put them onto the output conveyor. But if the downstream sortation equipment is designed for shippable parcels - not individual items - the mispicked items will cause problems.
Nieves said Plus One’s service model is to use people in the workflow where it makes sense. “That's why the company is called Plus One - because you add a human being to make the system more robust. The uptime, reliability, all of that goes up because you put smart people in charge.”
Adaptable, purpose-built AI for parcel picking to drive cycle time
Plus One Robotics provides machine vision systems exclusively for picking and depalletizing packaged shippables, not individual eaches. Nieves said Plus One’s systems are used by major parcel couriers, including UPS, FedEx and DHL.
Plus One's customers want faster cycle times for their applications. “In an each picking situation, you might do seven orders a minute,” Nieves said. “We have to live at 25 plus cycles per minute. So the robot never stops moving, which means the vision processing has to happen at that cadence.”
Rather than develop a generalized AI model to accomplish tasks, Nieves said Plus One has created a purpose-built, “adaptable” AI specifically for parcel picking. “A generalized model has to learn a bunch of stuff it may never use. An adaptable model learns what it needs.”
To achieve faster cycle times, each PickOne iPC runs off one GPU, allowing the system to process images faster than generalized AI. “We deal on the order of 10s of milliseconds,” Nieves said. “I don't have to have a constellation of computational power.”
But it takes more than Goldilocks-sized computing to speed up pick cycles. Plus One exclusively deploys camera-off-robot hardware placement for image acquisition.
After the robot arm picks a piece and moves out of the picking area, the camera captures the next image so the software can start analyzing it. “By the time the robot comes back, it already knows the next pick to take.”
Nieves said that while some applications may benefit from a camera-on-hand arrangement, the machine vision system is not imaging while the arm is in motion, which serializes cycle time delays.
Placement verification can keep downstream processes flowing
Plus One’s machine vision systems are hardware vendor agnostic. After examining the picking area image, PickOne can deliver a six-degree-of-freedom pose to motion control platforms through a register, position variable, or fetchable array.
But PickOne doesn’t just take an image and send a command. Using a second camera over the placement area, PickOne validates that the robot arm placed the parcel in the correct position and orientation, a process called placement verification.
One reason PickOne checks placement and orientation is to prevent disruptions to downstream processes. Nieves said parcels need to be loaded at an angle on crossbelts that feed into high-speed sorting loops for the packages to merge correctly.
Another reason is to check for double inducts, or when the robot arm accidentally picks up two parcels at the same time. For some applications - such as depalletization - robot arms can increase throughput by picking multiple items simultaneously. However, singulation requires picking one piece at a time.
When a robot arm singulates packages, one objective is to orient labels so that scanners in sortation systems can identify packages. If a package is placed in an orientation with its label concealed and it enters a sorting loop alongside another parcel, the stray package could be missorted with the other package.
Plus One offers an individual suction cup control gripper that can match the shape of the target parcel and avoid accidentally grasping adjacent parcels. But even with the best AI, machine vision, and gripper hardware, double inducts cannot be completely eliminated.
While human-in-the-loop robotics is about setting limits on which problems robots should not attempt to solve on their own, Plus One has devised double induct solutions that robot arms can execute autonomously. One option is for the robot arm to “recover” one of the parcels, hold it while the other parcel clears the placement area, and then set it back down.
For parcel carriers such as DHL, Plus One developed an offering that keeps a human in the loop, but not through teleoperation. When the system sees a double induct, rather than attempt to solve the problem itself, PickOne instructs the conveyor to run backwards into a reject bin. Nieves said the robot can continue picking without impacting cycle time while an employee handles the double induct manually.
Want to learn more about cobot arms? This article was featured in the April 2024 Robotics 24/7 Special Focus Issue titled “Collaborating with robot arms, platforms.”
Supervised autonomy for robots
Erik Nieves, CEO of Plus One Robotics, explains why the company chose a human-in-the-loop robotics service model.About the Author
Follow Robotics 24/7 on Linkedin
Article topics
Email Sign Up
Related Supervised Autonomy News
Related Companies
Related Resources
