Apple’s track record of innovation needs little introduction: the graphical user interface, desktop publishing, iPod/iTunes and the iPhone. These technological leaps are now the stuff of legend. These innovations were profound, not because they were novel, but because Apple took the time to study the ecosystem, fill in the gaps and provide a holistic solution with value from first use.
These technologies have restructured markets, shuffled suppliers and, in the end, fundamentally changed the dominant human/computer interface paradigms. As a technologist and a developer, I have watched with great interest as Apple’s developments have given developers, both tools, and an audience for them to reach.
The End of Bifurcation
It is not an overstatement to say Apple has done it again with the release of ARKit, a coordinated set of tools in iOS 11. These tools will lead to a new wave of apps fundamentally changing how we engage with the physical world. And after a decade of looking down at our phones (our realities bifurcated between the digital and physical worlds), Apple has created a compelling foundation to merge our worlds again, and in the process, encouraged us to re-engage with the physical world, which is the foundation of the human experience.
Augmented reality (AR), as distinct from virtual reality, grounds the human computer interface in the context of the physical, allowing apps to leverage the most powerful user interface we have, our senses. Before we can crawl, we know how to manipulate the physical world around us, from picking up toys, to shaking keychains and seeing how they interact—we are wired to explore and discover.
ARKit and the underlying release of the Vision and ML layers, each of which warrant its own post, taps into this innate human ability and desire to explore; yet ARKit is exceptional because now we get to explore on our own terms using the sensors on the platform, as opposed to being relegated to an intermediary between our desires and the keyword paradigm of search.
When fully realized by the Apple developer community, a sustainable model for human-computer interaction will emerge, one that rightfully puts the people, places and things that define the human experience at the center again. It will also place the mobile device one step closer to its predicted destination as a profound technology that disappears as the cognitive load to use it evaporates; the precise destination Mark Weiser, the father of ubiquitous computing, predicted many years ago.
ARKit leverages the advances in sensors and sensor fusion to deliver a powerful pose tracking mechanism, freeing developers to focus on how apps are engaging the physical world, and not on the nuances of Simultaneous Location and Mapping (SLAM) or other algorithms. This peace of mind frees developers to focus instead on creating compelling experiences that matter for users. Underlying this functionality is an extensible architecture powered by the machine learning abstractions provided by Apple’s Metal. The same way ARKit simplifies matters for the developer, Metal allows entities, such as Digimarc, to quickly leverage our expertise in machine learning and convolutional nets, without having to directly write the infrastructure to run our neural-networks. The final piece of the puzzle is the underlying architecture that leverages the GPU (Graphical Processing Unit) instead of the CPU (Computer Processing Unit), the rightful home for visual discovery (may Moore’s law rest in peace).
Accurately Identifying Objects
For all the polish demonstrated on stage at this year’s WWDC17, there is a missing element in making this ecosystem take off—and that is the accurate identification of objects.
All the examples of ARKit to-date, and there are many, are based on moving the device through space and using it as a lens for entertainment purposes, not as a tool to investigate those tangible objects that define our everyday lives. To put a fine point on it, ARKit doesn’t innately know what it is looking at, only where it is in 3D space.
Our focus at Digimarc for over 20 years has been to ensure the devices and networks around us can identify, with absolute confidence, the identity of images, media and, more recently, packaged goods. Our early foray into augmented reality dates back over 15 years, and has taught us a lot about the importance of providing fast, accurate and reliable identification of objects. This has proved particularly true when it comes to understanding how people engage with the products we all buy and consume on a daily basis.
As an example, there has been a global surge of interest in food safety and a desire to know the pedigree of the products and medicines we consume. And while consumers may think their phone or wearable device can be the tool to learn more, if identification accuracy is poor—providing the wrong instructions for an over-the-counter medicine, for example, or incorrectly listing allergens—there can be real-life consequences.
Empowering the Apple User
Our focus at Digimarc is to ensure that devices such as an iPhone, or a future wearable device, can provide additional information in the form of AR experiences that are grounded on deterministic identity for consumer packaged goods. This includes the ability to identify instances of visually identical packages, allowing product pedigree (related to a specific package) to be shared with the consumer. Historically, this has been done in support of serialization efforts and supply-chain visibility, but the real value is empowering the Apple user to discover the world around them, by providing a wealth of actionable information.
Digimarc’s expertise in accurately identify objects, combined with the tools in ARKit, truly opens up a future of unlimited applications, where developers can deliver real enduring value to app users. And when you combine this with the efficiencies provided by Metal, Vision and the other features forthcoming in iOS 11, it becomes clear we are living in a true golden age, where developers can create new and meaningful experiences that address the real needs and desires of consumers.