What is interaction? Or, a series of questions to consider during my study at ITP.
What is it not? Interaction is not a reaction. On several occasions, upon turning the corner at a museum gallery, I’ve found myself unexpectedly facing a Rothko. My heart jumps and I stop breathing. It is always involuntary and thrillingly intense. But the painting never acknowledges me with a corresponding greeting. It never saturates nor mutes itself in response to my suspended state, and so I settle and sink into the color field.
In an interaction, gestures influence and they are reciprocated. The etymology of the word, reciprocal, according to my laptop dictionary, indicates that it’s from the 16th century Latin word, reciprocus, based on re- ‘back’ and pro- ‘forward’. Back and forth. Back and forth. Retreating to create space and allowing pause to receive a response, and then moving ahead with an answer. Interaction as a dance, a dance between [insert two or more nouns* here].
*living beings, objects, places, systems
But what triggers an interaction? There must be some impetus. Is it a question, a need, a desire? Is that question somehow coded in the physicality of the object or environment (as we discussed in class last week)? Is it declared in the tone and volume of an intimate partner or in the glance of a neighbor on the subway?
And once contact is made, how does it feel? Does it perpetuate further exchange or does it push for a hasty retreat. Is it friendly, inclusive, unexpected, spirited? Or standoffish, exclusive, predictable, dull?
If you enter into the dance, is it a guided back-and-forth or spontaneous and open for exploration? Does it restrict movement or speech or invite free play? And specifically for physical interaction, how do the five senses respond? Is it loud, bright, soft, pungent, sweet? Is there a congruous connection being how it feels (form) and what you’re actually doing (function)? If not, what needs to change to offer a much richer, potentially more satisfying experience? And I can’t help but wonder: is playful interaction the embodiment of joyful curiosity and learning for all involved parties?
In 2016, I experienced two digital works during NYC’s Creative Tech Week, both labeled interactive. The first, Man A, by Gibson/Martelli was described on CTW's website as “an interactive downloadable app and augmented reality experience that sees life and movement burst from a surface of distorted patterns.” With the app in hand, I set upon their sculptural piece and indeed an animated geometric character sprang to digital life and scrambled over and around. Was it delightful and fun? Absolutely! But was it interactive? Crawford might assign it a very low degree for the nitpicky. My view of the digital performer shifted as I moved, but the cartoon did not adjust its own movements in response to my gaze. I, or rather the app on my phone, merely activated its lively contortions.
I also met Louise Foo and Martha Skou at that year’s expo alongside their Format No. 1 and Format No. 2 installations. On their site they describe their work as “an exploration of the connection between image and sound,” and with the required app, my phone became “an optical sound device” that sang notes as I waved over marks on the wall and objects dangling from above. The artists “combine analog techniques with digital technology to create new experiences,” and “not only to see or hear but also to have an experience in physical space” which I certainly did. It was playful and soothing to the ears, and the resulting composition completely of my own. And therein lies the key difference with Man A: I created something new with my choices. No doubt visitors before and after me composed their own individual songs, as well. So were the pieces interactive? As much as any piano. Sure the same sound played each time an app-rigged phone scanned a specific location, but the option of when and how often to activate that note in the context of many others was a rare gift for the experiential art seeker.
Inspired by this week’s readings:
Alexa, Please Kill Me Now: Thoughts on Conversational UI by Alan Cooper
Things that don’t work yet: On Roli Blocks, musical technologies, and designing connected objects in general by Dan Hill
A Brief Rant on the Future of Interaction Design by Bret Victor
The Art of Interactive Design, Chapters 1-2, by Chris Crawford