Josh Clark headshot image

Magical UX and the internet of things

“Any sufficiently advanced technology is indistinguishable from magic.” – Arthur C. Clarke

That insight from one of science and technology’s most visionary thinkers started Josh Clark on a quest for a new approach to design for the internet of things. After all, what could be more magical than commanding unseen forces through the use of everyday objects?

The founder of design agency Big Medium (formerly Global Moxie), Josh specializes in designing multi-device experiences that blend function and inspiration in equal measure. We spoke with Josh to learn more about how he developed his “magical” approach to design—the focus of his upcoming keynote and workshop at Delight.

Josh: “It’s a challenging thing to create something that is genuinely magical. The tricky thing about magic is that it decays. What’s magic today is old news tomorrow—after a while it just becomes part of the fabric of things. For example, today we’re succeeding at getting information into all these different devices and places, even into our physical environment—our homes—with things like Nest and other home automation tools. The challenge is less and less ‘how do we create great individual experiences,’ but how do we create experiences across all of these devices. And it’s in these gaps between devices that I think there is some really magical opportunity.

“We can actually with a gesture or a word or sometimes combination of the two move information from this device to that in a way that really does feel more than just a technology, but as a kind of a wizardry and control over our environment.

A dance of devices

Hearing Josh speak about the relationship between devices, actors and environments got us thinking about choreography—and Josh found it an apt metaphor for the new role of the designer.

Josh: “There are many physical arts that come into play here. Choreography is certainly a piece of it. How do you make all of these moving pieces work together to create one delightful experience? Particularly as we start to create physical interfaces to digital systems—which is really what the internet of things is all about—moving interaction away from the screen and into our immediate physical environment. As you start to do that there are aspects of motion through space, requiring designers to plan and coordinate the movements that define an interaction. And that’s choreography.

Leading with imagination

Magical UX is Josh’s way of allowing creative forces like inspiration and imagination to define a future that is increasingly dominated by technology.

Josh: I think Google Glass is a great example of what happens when you allow technology to lead at the expense of imagination: ‘We’ve got a camera, a processor, and these little tiny screens, and what if we put it all on your face?’ Rather than just saying, ‘What if these glasses were magic? What are the possibilities of creating a magic vision or a magic lens for things?’ And I think you get very different answers to those questions because it’s really about what’s the fundamental characteristic about the glasses or about vision that we can enhance?

That’s why magic is such an interesting thing. We have centuries of experience of myths and legend, of telling ourselves stories that ask what it would be like if part of the physical world was magic. What if this mirror was magic? What if this carpet was magic? These regular, everyday, mundane objects, what if they were lit up with a little bit of smarts and power? They still work as their original object, and we manipulate them like the original object, but they can also do something special. And we have design patterns in place, because we already know what a magic mirror is supposed to do. We know what magic rings are for. It’s these things that we’ve been telling ourselves in stories forever. We have a shared idea of what they are. It’s a very different vision of the future from the more science fiction-based, technology-focused future—where screens are everywhere. When you think of magic, it leads to a more organic, humane vision of what the future might be.

It’s life, amplified

Central to Josh’s point of view is a belief that the role of technology is to give us more control over our lives, not less.

Josh: Amplify is a great word to describe how I want to think about this. Magic is about extending my will into my environment or making me more of who I am. It’s not Harry Potter’s wand that’s magic, after all, it’s Harry. That’s a bit of an illusion, sleight of hand or at least stage magic, making these systems we manipulate feel like they are coming from us. But it will have the effect of amplifying us—and the objects around us—through our interaction.

A lot of people are nervous about an automated home, a staple of dystopia or dystopic science fiction.  Because you know the machines take over or we lose agency to the machines—but I think that’s overdrawn. There will always be a partnership between humans and machines. But it’s got to start with people and not technology. The technology has to bend to our lives, not to the reverse.

School for magicians?

While there’s no Hogwarts for magical UX yet, Josh will be sharing design techniques in a workshop on the second day of our Delight Conference. He offered some insights into what designers—or aspiring magicians—can learn.

Josh: We are in an area of discovery and experimentation where we’re figuring out exactly how magical objects should behave. We don’t have standards yet, but we have notions and ideas, and we can get started from those. There are hints and cues and things that are cultural norms about how an enhanced object, a magical object—or an enchanted object—would work. It’s a couple of things: One, I think we start with how the object is used already. What is the natural interface for an object or a place and how can we lean into that and build upon that? Another part of it is realizing that in some cases we only want to have a very light touch with these things, and the enhancements will be really modest. Some will be really modest magical powers; it’s not all fancy technological wizardry.

Whether it’s modest or grandiose, Josh’s vision for a creative, imaginative, magical future leaves us simply… spellbound.

Watch Josh on the Delight 2015 main stage then subscribe to Delight Digest for fresh conference news.

About Josh Clark

Josh Clark is an interaction designer specializing in connected devices, multi-device design, product strategy, and user experience. He’s the author of four books, including “Tapworthy: Designing Great iPhone Apps” (O’Reilly, 2010) and the forthcoming “Designing for Touch” (A Book Apart, 2015). Josh’s agency Big Medium offers design services, strategic consulting and training to help creative organizations build exceptional interfaces for apps, websites and connected devices. His clients include AOL, Time Inc., eBay, O’Reilly Media and many others.