Ever since Amazon released a promotional video of Amazon Go in late 2016, there has been a lot of buzz among retailers about what this means for the future. The video demonstrated how a consumer would enter an Amazon “grocery” store, grab what they wanted and leave: no cashier, no self-checkout and no need to scan the product, the hang tag or the shelf label. But it offered no details on the underlying technologies, begging the question: “How does it work?”
Amazon Go’s planned opening for May had to be delayed because of “technical issues.” News reports have indicated the technology (more on that later) malfunctioned when more than 20 people were shopping in the store. From conversations with partners and customers, we understand innovating in the retail space presents challenges. But our team at Digimarc has been watching the Amazon experimentation with great interest and we applaud their boldness.
First, it’s important to keep in a mind a few basic facts with the Amazon experiment. The store is located on the Amazon “campus” and is only open to Amazon employees. The many issues that affect a grocery store in the real world are not in play here. Any retailer can tick off a host of challenges with this scenario, not least of all shrinkage via theft, and the consequent loss prevention efforts a store would have to implement.
Based on the current state of object recognition, activity tracking and deep learning, one can extrapolate how several of the key elements of the store work and what the related challenges might be.
The video begins by showing consumers entering the store and scanning their phones via an electronic check-in station. I suspect this station is enabled to read a static barcode from the mobile screen that identifies the user or possibly a one-time token for the shopping session.
Once the shopper is identified, they can be tracked in the store using depth-sensing cameras, and infrared sensors that specialize in traffic analysis, all fairly common techniques. Amazon’s patent filings reference facial recognition which, given the constrained size of the store (1,800 sq. ft.), and the fact that the shoppers are all known (Amazon employees), this model is certainly achievable. Yet even without facial recognition, the accuracy provided by a traffic analysis system is sufficient to segment shapes and track individuals through the limited traffic patterns of this store.
Another tracking method could be installing pressure sensitive tiles in the floor. This is certainly a scalable option, given the tiles only cost approximately $10 each.
As a technologist, I find the most interesting aspect of the interaction occurs at the shelf. Many industry watchers know Microsoft has been demonstrating the use of Red-Green-Blue (RGB) plus depth cameras in the retail space for multiple years now.
The theory is that with an array of inexpensive cameras, you can create and monitor a “plane” in front of retail shelving. If a shopper disrupts the plane, you can see where on the shelf, and how high off the floor (depth aspect of the camera) the object was selected. Utilizing this mechanism with highly enforced planograms (the plastic rails on the shelves), it is possible—with a high level of accuracy—to tell which item was picked. And when you appreciate that Amazon is able to combine their deep experience with machine learning/vision with virtually unlimited computing resources, this is a solvable problem.
The more robust challenge, in my opinion, is removing uncertainty from a purchase made from the same location, at the same time, by people who have taken the same path in the store and are visually similar. Yet again, in a subsidized company store, challenges are greatly reduced, and any loss or shrinkage will be minimal.
As a technology company in the retail space, we want Amazon Go to be successful. We’re excited about the kinds of questions they are raising and we believe Amazon Go’s initial attempts will lead to new ideas and more innovations.
The real challenge with the Amazon Go proposition is how do you scale the model while keeping it economically viable? Certain elements of the implementation, such as facial recognition, are either problematic from a privacy perspective, or not technically viable at scale (i.e. image recognition). With a larger store, comes a larger image recognition database that will need to manage 20,000+ SKUs (Stock Keeping Units), not several hundred.
There are also other challenges with large SKUs. Many SKUs form visually similar families to re-inforce branding elements. In the Image Recognition community, such product families are considered “degenerate cases” because the technology cannot differentiate between visually identical packaging. For instance, picture a family of boxed detergents where the only difference is the scent variant, represented as a small graphic on the front label. If the differentiating word or element is obscured, or not presented to the camera, it results in confusion regarding which product was pulled from the shelf. At scale, the results of such mis-identifications initiate a sequence of geometrically growing costs, such as stock-outs, excess inventory, shrinkage and frustrated customers. Digimarc Barcode offers a convenient solution to these challenges by enabling fast, accurate identification—at enormous scale—and even down to the level of a single package.
Another key advantage of Digimarc Barcode is that it works with the current retail and supply chain infrastructure as it currently stands, allowing retailers to invest incrementally and harvest benefits accordingly. Retailers don’t have to radically change their store, retail model, or relationship with their customer, which may be necessary with the Amazon Go model. The opportunity for retail with Digimarc Barcode includes easier checkout, consumers scanning packages for rich content, real-time inventory data, increased supply chain efficiencies, and opportunities for retailers and brands to have a “connected package” and more easily communicate with consumers.
The Amazon Go story is very much a work in progress and we plan to follow their developments with great interest. We’re convinced that technological investments and innovation will continue to be a crucial factor in helping grocers and retailers distinguish themselves in an intensely competitive space.
Share this Blog
