Building the model
In August of 2020, the organization reached out to its partners at Insight to help design a solution. Members of Insight’s team began with a proof of concept, developing two computer vision models using Microsoft’s Custom Vision Cognitive Services.
These models were trained to identify the products on a single pallet in a given image. But accurately identifying the correct products was only the first step. Using a front-facing image to count objects in a three-dimensional formation meant recognizing not only how many products were visible, but also inferring how many were not visible.
To solve this problem, Insight developed a proprietary algorithm to model a 3D virtual pallet of boxes based on the 2D image provided. By calculating the size of each product relative to the known pallet depth, the algorithm is able to accurately predict the number of products layered behind the visible rows.
With the groundwork in place, Insight integrated these solutions into a simple application which could be operationalized within the existing environment. The app allows users to snap a quick picture of a bin as they complete picking or restocking tasks. These images are automatically uploaded to Azure, where the computer vision model and corresponding algorithm calculate the number of products on the pallet.
This data is then correlated with the company’s ERP system to ensure this number of products in the image aligns with the expected inventory. If a discrepancy is detected, the application flags the pallet for a manual count. If the output matches anticipated count, no further action is needed.
Accuracy and accessibility at scale
Phase one of the project focused on pallets containing only one type of product, which simplified some aspects of the initial training process. But with the success of the pilot, Insight and the client rapidly shifted into phase two, extending the solution to pallets containing multiple types of products in multiple box sizes. With this additional training and optimization, the computer vision solution has now reached 93% accuracy.
As a result, the organization rolled out full production in January 2021. The app is now in use at eight warehouses across the U.S., eliminating the need for workers to perform manual cycle-counting and freeing up more time for meaningful, revenue-oriented tasks.
The client’s CTO expressed the significance of this solution and the value of Insight’s partnership.
“Our goal with these technologies is to give all our employees, with or without disabilities, the same opportunities to perform the same jobs in our warehouses — and to show other organizations this is possible.” He said. “Together, [our company] and Insight have been able to move quicker, collaborate better and accomplish so much more than we would separately.”