For nearly 100 years, researchers at Bell Labs have developed countless technological innovations. The research institution is credited with inventing numerous electronics components that are commonplace today, including the transistor, photovoltaic solar cells, and charge-coupled device (CCD) imaging sensors.
Now Nokia Bell Labs, the New Jersey-based research subsidiary focuses on commercialization opportunities for new technologies.
One project at Nokia Bell Labs involved flying drones equipped with cameras within vertical farming facilities. Drones enabled by AI and machine learning technologies allowed farmers to more efficiently monitor plant health without manual inspection.
As the project continued, Nokia identified warehouse inventory monitoring as a commercialization opportunity, forming an internal startup to market its Autonomous Inventory Monitoring Service (AIMS).
Paul Heitlinger, general manager of Nokia AIMS, said the small standalone business unit is now working with warehouse operators across the U.S. to automate inventory cycle counting using drones.
Minimizing impacts to customers
Nokia AIMS launched commercially last year, announcing a deployment with Graybar in October 2023. Heitlinger said most AIMS customers count overnight during off hours, meaning that the average closure time of 30 minutes won’t delay their normal operations.
“There’s huge value for those customers,” Heitlinger added. “They’re doing it much faster. We’re getting more coverage than the human workers. They don’t have to get on a scissor lift. They’re not making mistakes. They’re not having to stop to write things down. It’s much more accurate.”
While the best time to cycle is during off hours, not all customers can dedicate time to count.
“The other issue with flying while picking is that the data will change,” Heitlinger said. “The best use case for this is where there's downtime while the drone is flying, because otherwise the data that the drone has captured is already stale by the time it's consumed by the user.”
One AIMS customer with fast-moving inventory needed to start picking immediately after counting. Even though items moved before the customer could view their data, Heitlinger said they were still able to benefit from uncovering pre-count inventory mismatch trends.
Off-the-shelf drones with customized attachments
AIMS uses a customized drone platform, based on U.S.-made processors. Nokia implements its proprietary navigation, autonomy, localization, and processing software - along with some specialized attachments - so the drones can accomplish their missions.
AIMS cycle-counting drones travel in saw-tooth patterns up and down storage racks and aisles, hovering to capture images of each location before flying off to the next one.
The drones are relatively small and lightweight, which allows them to fly in aisles as narrow as four feet. Lighter drones also use less energy, allowing AIMS to maximize flight time. “The smaller the drone, the more agile it is, the less intimidating it is for a customer,” Heitlinger added.
Cameras mounted on the drones are outfitted with customized lenses. The focal length is set to capture clear images at a distance from the rack where the entire location is visible in a single image.
The lenses are also designed to ensure pictures taken from multiple angles in focus, enabling AIMS to count items on pallets. They are also outfitted with a light bar to illuminate the tops and sides of pallets.
Edge computing to combate latency
Together, localization and autonomy software form a feedback loop that allows AIMS drones to fly to the correct positions at the correct speeds, pausing long enough to take clear pictures. Localization tells the drone where it is, autonomy tells it where to move, and then localization checks to ensure the drone is where it should be.
Low latency originally enabled AIMS to rely on cloud computing for real-time processing. But customer connectivity was not reliable enough to run navigation through the cloud. So AIMS recently introduced edge computing.
“The benefit of the drone that we use is that it has a very high level of compute capacity on board, even in a tiny little package,” Heitlinger said. “We actually ended up moving all that compute capability that we needed to make the drone autonomous and localized onto the drone.”
Now AIMS drones can autonomously fly round-trip missions from a launch spot without network connectivity. They only need a connection for the command to launch, and after landing will upload images and metadata to an on-location server as soon as they reconnect.
Structure-Based Visual Localization
Because indoor navigation cannot rely on GPS, some robots reference scannable fiducial tags placed around a facility for localization. “But we wanted to move away from that because we wanted to minimize any impact that the customer would have to do to get onboarded,” Heitlinger said.
Nokia AIMS developed a proprietary computer vision system called Structure-Based Visual Localization (SAVLOC). Customer onboarding requires a schematic of the warehouse.
“We look for certain structures in the warehouse based on a map that we generate,” Heitlinger said. “As it sees particular types of structures that are known, it can localize where it is in three dimensional space.”
After an onboarding period for the system to learn and map a facility, SAVLOC enables AIMS drones to fly at the correct distances from racking and capture images from the middle of each pallet location, providing clear images for both end users to view and for AI analysis.
Extracting and presenting data from images
AI and machine learning algorithms extract information from photos and present it to customers through a user interface (UI). Heitlinger said Nokia has developed barcode-detection and optical character recognition (OCR) software that can translate images into human-readable text.
“We can also start counting what is visible from the aisle,” Heitlinger said. “Even if it's stacked within a pallet size, even if there's multiple layers and they can't be seen, we can count it. If a human can reasonably count it, then our system can reasonably do the same thing.”
AIMS can pull inventory data from a warehouse management system (WMS) or enterprise resource planner (ERP) to create a differential report against its count.
“We position this as we provide a source of truth to the customer, saying, ‘Regardless of what you think is there, here's what's actually there,’” Heitlinger said.
New inventory counting capability
Nokia AIMS recently launched an inventory counting capability. Unlike inferred case counting, the feature requires knowledge of how boxes are stacked on pallets and a sequence of images taken from multiple angles to count how many boxes have been removed.
“It’s the holy grail of cycle counting. We’re putting the ‘count’ in inventory counting,” Heitlinger said. “Not only can we tell you if the product is in the right place or the wrong place, but here’s how many of that item there are in that location.”
After recently testing with one customer, the inventory counting feature is now in production and available to all AIMS customers.
“We estimate about a 95% accuracy,” Heitlinger said. “Combined with empty bin detection, plus their product locations, and the very quick and efficient way that we can cycle count - the value and the ROI to customers is really huge.”
Calculating items within cases a future goal
While drones can’t count items within cases, Heitlinger said autonomous counting allows warehouse operators to assign human counters to picking locations with open, partial cases that can’t be counted autonomously rather than the whole warehouse, which can improve labor productivity. “Usually those open cartons are down lower on the racking,” he added. “It’s much easier and quicker for them to get to those locations.”
AIMS plans to release an update that can pull quantities within cases from WMS, allowing the software to calculate individual item counts.
Despite AIMS being a new venture, Heitlinger gave his kudos to the Nokia Bell Labs development team for their accomplishments thus far. “With a small team, we've actually built a lot of really good tech in a very short amount of time,” he said. “I’m very excited to see what that next year or two will bring.”
Want to learn more about data management in robots? This article was featured in the September 2024 Robotics 24/7 Special Focus Issue titled “Robots as mobile data platforms.”
Nokia Autonomous Inventory Monitoring Service (AIMS)
About the Author
Follow Robotics 24/7 on Linkedin
Article topics
Email Sign Up
Related Inventory Management News
Related Companies
Related Resources
