What is physical interaction?

As it turns out, the term interaction seems to be a pretty slippery one.  But from what I can gather, and in relation to the interaction between a human and a machine, it requires both action and reaction from each party.  The human must do something, which causes the machine to not only react, but react in a way that represents the particular action of the human.  Then present that information back to it’s human counterpart.  The human then reacts, and the cycle continues.  Physical interaction in particular, as inspired by Bret Victor’s rant, is how we use our bodies to give input to a machine.  He focuses a lot on hands, and rightly so; but why couldn’t we leverage any and all of our human senses and capabilities to provide input into a machine, be it visual, aural, or tactile?  Good physical interaction comes about when the modes of input are natural and discoverable and the reaction from the machine is accurate, timely, and representative.  Good physical interaction should focus on the human needs, not the needs of the machine.  It should account for human error, and take into account human psychology and culture.  I can imagine a scenario where a wave gesture might be used to dismiss a window, but what if, in a different culture, a wave gesture of the hand means something radically different than “go away”? Maybe a poor example, but the point stands that designed interactions must take the audience into account.  Good physical interaction is expressive and deep, offering a range of modes of interaction.  Good physical interaction makes you rethink how you’ve interacted with technology in the past because it presents a refined or more natural way to accomplish something.

The first thing that came to mind was Siri.  We all know and love/hate Siri.  She never seems to “get it”.  But Apple has attempted to change the way we interact with our phones, and the results are, well, hit or miss.  The pinnacle of this type of interaction might exist similarly to what’s presented in the film Her, where everyone interacts with their devices mainly through speech.  In the movie, though, the machines respond in remarkably human ways, understanding context and intention.  Siri obviously doesn’t.  Which is why she can be so damn frustrating.  The fact that I don’t personally know anyone who uses Siri for more than setting alarms probably speaks to the fact that this way of interacting with tech just isn’t that useful, at least as it currently stands.  As Crawford might say, Siri isn’t a great thinker.  She is sometimes a decent listener (as long as you speak in a stereotypical standard American accent) but rarely produces the information you’re looking for.  It’s a relatively poorly designed interaction because Siri fails to “think” well.

Screen Shot 2015-09-09 at 2.52.39 PM

Leave a Reply

Your email address will not be published. Required fields are marked *