EDUCATING FOR INTERACTION
By Dan Collins
A first version of this paper was given at the Performative Sites
conference at Penn State in October 2000.
A published version appeared in New Art Examiner in February 2001.
What is interaction? How can we begin to make sense of the avalanche of educational toys, computer programs, and artworks that claim to be "interactive?" What would a new pedagogy structured around the rules of interactivity look like?
Some theorists have argued that all "art is interactive," effectively shutting the door on the discrete contributions of new digital/computer-based interactivity. Conversely, other theorists such as Simon Penny have insisted that traditional artworks such as painting and sculpture are simply "instances of representation" and as such should not be defined as truly "interactive systems." For Penny, interactive artworks are "virtual machines which...produce instances of representation based on real time inputs." (Penny 1996) Still other theorists distinguish between the relationship of system interactivity (the enabling hardware and software) and the nature of the interaction (the actual exchange--be it aesthetic, educational, political, etc.) (Hillman et al, 1994). For our purposes here, a high level of interaction equals "mutual reciprocity"--a state of dialectical exchange between two or more entities. Ideally, interactive systems--whether a high tech computer game or a Socratic dialogue--can be tools for learning providing intelligent feedback that refines and amplifies user input.
While the demand for "interactivity" is a relatively recent phenomenon in the arts, the culture at large has long been obsessed with the idea of machines that learn. The evidence is mounting. From media spectacles such as Big Blue's defeat of World Chess Champion Garry Kasporov in May of 1997 to quieter revolutions in teaching autistic children, computers that master the behaviors of their users are beginning to find a place in the culture. There is more than a hint of narcissism in our desire to be personally reflected in the machines we make. We don't want simply "dumb" tools, we want "intelligent" machines that respond and learn by interacting with their owners. Even our cooking appliances and car radios are "programmable" to reflect individual tastes.
Few art schools provide courses for producing let alone interpreting or critiquing "interactive artworks." Though the borderline between the fine arts and other cultural practices (such as science, technology, and entertainment) is becoming increasingly blurred, it is clear that the development of "interactive art" is largely dependent on "non-art" traditions. From a technical and theoretical perspective, such strange bedfellows as computer gaming, combat simulation, and medical diagnostics have more in common with much recent digital and interactive art practice than main stream art history or criticism. Theorizing this territory is less a matter of mining, say, the Art Index, and more a matter of conducting systematic research into areas such as communications theory, human computer interaction, educational technology, and cognitive science. With this in mind, it may be helpful to briefly review how other disciplines are looking at the issues surrounding interaction.
THEORIES OF INTERACTION
"Interaction" is a useful construct in helping to understand the complex relationships occurring in a computerized learning environment. Educational technologist Ellen Wagner defines interaction as " reciprocal events that require at least two objects and two actions. Interactions occur when these objects and events mutually influence one another." (Wagner, 1994)
Wagner points to historical examples of communication theory to illustrate the move from "one-way" systems of communication to "multi-directional" systems. C.E. Shannon's mathematical theory of communication (Shannon, 1948), for example, was a highly linear engineering model of information transfer involving the one-way transmission of information from a source to a destination using a transmitter, a signal, and a receiver. Later theorists built upon Shannon's model to include the concepts of interactivity and feedback. It is only recently that truly interactive systems that support both synchronous and a-synchronous exchanges among multiple users have been available. Common examples of synchronous exchange include live satellite uplinks, telephones, and chat rooms on the Net. E-mail is the prototypical example of an "asynchronous" exchange system.
Other trends supporting the development of interactive systems come from research in artificial intelligence and cognitive science. If "mutual influence" and reciprocity are criteria for true interactivity, then the system needs to be capable of delivering more than pre-existing data on demand. Interactive systems need to be able to generate "custom" responses to input and queries. In short, the system needs to be smart enough to produce output that is not already part of the system. Interactivity must be more than following predetermined prompts to preprogrammed conclusions like in a video game
While most natural and living systems are "productive" in the sense of creating new "information," human-made machines that can respond with anything more than simple binary "yes/no" responses are a relatively recent phenomenon. To paraphrase media artist Jim Campbell, most machines are simply "reactive," not interactive. "Intelligent" machines, being developed with the aid of "neural networks" and "artificial intelligence," can interact by learning new behaviors and changing their responses based upon user input and environmental cues. Over time, certain properties begin to "emerge" such as self-replication or patterns of self-organization and control. These so-called "emergent properties" represent the antithesis of the idea that the world is simply a collection of facts waiting for adequate representation. The ideal system is a generative engine that is simultaneously a producer and a product.
Creating an experience for a participant in an interactive artwork must take into account that interactions are, by definition, not "one-way" propositions. Interaction depends on feedback loops that include not just the messages that preceded them, but also the manner in which previous messages were reactive. When a fully interactive level is reached, communication roles are interchangeable, and information flows across and through intersecting fields of experience that are mutually reciprocal. The level of success at attaining mutual reciprocity could offer a standard by which to critique interactive artwork.
Many artists have developed unique attempts at true interaction, addressing problems of visual display, user control processes, navigation actions, and system responses. Different works have varying levels of audience participation, different ratios of local to remote interaction, and either the presence or absence of emergent behaviors. Moreover, different artistic attempts at interactivity suggest different approaches to interaction could be used for diverse kinds of learners in a variety of educational settings. Understanding experiments with interaction in an art context may help us to better understand interaction in pedagogical settings.
Carol Flax: Journeys: 1900/2000
Detail showing a viewer turning a page of the book and triggering
a video projection.
Photo credit: Patricia Clark.
Arizona artist Carol Flax has created an "interactive book" entitled, Journeys: 1900/2000 at the Institute for Studies in the Arts at Arizona State University, where I am the Interim Director. At the heart of the project is a reproduction of a 19th century travel album that trades in fragments of memory, pieces of voyages, and bits of history. It uses single images from various existing albums, reproducing and recontextualizing them to create a completely original "voyage." Movement sensing technology (computerized tracking devices) sense the presence of a viewer. "Bend sensors" embedded in the pages cause an electrical signal to be sent to a computer when a page is turned. Video and audio clips are in turn triggered that support, amplify, or contest the veracity of the photographic prints of idealized ancient settings through the simple juxtaposition of contemporary imagery and sound with historical photographs. For example, a black and white image of a Middle Eastern market--a classic example of the "exotic" and the "picturesque"--is overlaid with a video closeup in color of an orange being passed from the hand of one person to another in an Arizona back yard. Text is used throughout both descriptively and ironically to throw into question the truth value of what we are seeing. The message is deliberately multi-leveled and ambiguous, but one thing is clear: we are creating a journey in which we are complicit, not simply voyeurs.
The work enables a unique method of navigating the content and scores high in providing a seamless encounter between the user and the subject matter. While the artwork is described as being an interaction between a single user and a variable content, the entire set of options remains fixed in the computer's database. The work does not claim any degree of "reciprocity" between the object and the user. It does not "learn" the reader's habits. Therefore, it is not, strictly speaking, interactive. This does not diminish the project's ability inspire repeated visits and reward the user with unexpected discoveries. Given its non-linear organization and randomized sequences of multiple video clips, each users experience of the book is actively engaging and unique.
As a model for a different "textbook" perhaps, the project points toward a new class of books that are constructed with the individual user in mind and that respond with some intelligence to reader's choices. The fact that Flax insists on preserving the essential kinesthetic aspects of reading--the feel of the paper, the turning of the pages--implies that certain direct forms of knowing just cannot be improved upon. However, the use of moving images, scrolling text, and audio clips that spill beyond the boundaries of the book have more in common with immersive experiences such as VR than reading. This is neither a conventional book nor an over-scaled e-book, but rather a hybrid of traditional forms and "reactive" hi tech processes.
Christa Sommerer and Laurent Mignonneau: Interactive Plant Growing (1993)
Austrian-born Christa Sommerer and French-born Laurent Mignonneau teamed up in 1992, and now work at the ATR Media Integration and Communications Research Laboratories in Kyoto, Japan. In nearly a decade of collaborative work, Sommerer and Mignonneau have built a number of unique virtual ecosystems, many with custom viewer/machine interfaces. Their projects allow audiences to create new plants or creatures and influence their behavior by drawing on touch screens, sending e-mail, moving through an installation space, or by touching real plants wired to a computer.
Artist's rendering of the installation showing the five pedestals with plants and the video screen.
Interactive Plant Growing is an example of one such project. The installation connects actual living plants, which can be touched or approached by human viewers, to virtual plants that are grown in real-time in the computer. In a darkened installation space, five different living plants are placed on 5 wooden columns in front of a high-resolution video projection screen. The plants themselves are the interface. They are in turn connected to a computer that sends video signals from its processor to a high resolution video data projection system. Because the plants are essentially antennae hard wired into the system, they are capable of responding to differences in the electrical potential of a viewer's body. Touching the plants or moving your hands around them alters the signals sent through the system. Viewers can influence and control the virtual growth of more than two dozen computer-based plants.
Screen shot of the video projection during one interactive session.
Viewer participation is crucial to the life of the piece. Through their individual and collective involvement with the plants, visitors decide how the interactions unfold and how their interactions are translated to the screen. Viewers can control the size of the virtual plants, rotate them, modify their appearance, change their colors, and control new positions for the same type of plant. Interactions between a viewer's body and the living plants determine how the virtual three-dimensional plants will develop. Five or more people can interact at the same time with the five real plants in the installation space. All events depend exclusively on the interactions between viewers and plants.
The artificial growing of computer-based plants is, according to the artists, an expression of their desire to better understand the transformations and morphogenesis of certain organisms (Sommerer et al, 1998).
What are the implications of such works for education? How can we learn from this artistic experimentation to use technological systems to be better teachers? Educators have long recognized the importance of two-way or multi-directional communication. Nevertheless, many educators perpetuate the mindset of the one-way "broadcast"--a concept that harks back to broadcast media such as radio and echoes the structure of the standard lecture where teacher as "source" transmits information to passive "receivers." The notion of a "one-to-many" model that reinforces a traditional hierarchical top-down approach to teaching is at odds with truly democratic exchange. In Interactive Plant Growing, Sommerer and Mignonneau invert this one to many model by providing a system for multiple users to collaborate on the creation of a digital wall projection in real time. The system in effect enables a real time collaboration that takes many diverse inputs and directs them to a common goal. And this is exactly what good teaching is. This conceptualization of critical pedagogy has been developed in many different situations, but here is combined with technology that mirrors its structure.
Sommerer and Mignonneau: Verbarium (1999)
In a more recent project the artists have created an interactive "text-to-form" editor available on the Internet. At their Verbarium web site, on-line users are invited to type text messages into a small pop up window. Each of these messages functions as a genetic code for creating a visual three-dimensional form. An algorithm translates the genetic encoding of text characters (i.e., letters) into design functions. The system provides a steady flow of new images that are not pre-defined but develop in real-time through the interaction of the user with the system. Each different message creates a different organic form. Depending on the composition of the text, the forms can either be simple or complex. Taken together, all text images are used to build a collective and complex three-dimensional image. This image is a virtual herbarium, comprised of plant forms based on the text messages of the participants. On-line users help to not only create and develop this virtual herbarium, but also have the option of clicking on any part of the collective image to de-code earlier messages sent by other users.
Screen shot of the Verbarium web page showing the collaborative
image created by visitors to the site.
The text to form algorithm translated "purple people eater" into the image at the upper left.
This image was subsequently collaged into the collective "virtual herbarium."
In both the localized computer installations and web-based projects realized by Sommerer and Mignonneau, the interaction between multiple participants operating through a common interface represents a reversal of the topology of information dissemination. The pieces are enabled and realized through the collaboration of many participants remotely connected by a computer network. In an educational setting, this heightened sense of interaction needs to be understood as crucial. Students and instructors alike become capable of both sending and receiving messages. Everyone is a transmitter and a receiver, a publisher and a consumer. In the new information ecology, traditional roles may become reversed--or abandoned. Audience members become active agents in the creation of new artwork. Teachers spend more time facilitating and "receiving" information than lecturing. Students exchange information with their peers and become adept at disseminating knowledge.
Ken Rinaldo: Autopoiesis (2000)
Overview of all fifteen robotic arms of the Autopoiesis installation.
Photo credit: Yehia Eweis.
A work by American artist Ken Rinaldo was recently exhibited in Finland as part of "Outoaly, the Alien Intelligence Exhibition 2000," curated by media theorist Erkki Huhtamo. Rinaldo, who has a background in both computer science and art, is pursuing projects influenced by current theories on living systems and artificial life. He is seeking what he calls an "integration of organic and electro-mechanical elements" that point to a "co-evolution between living and evolving technological material."
Rinaldo's contribution to the Finnish exhibition was an installation entitled Autopoiesis, which translates literally as "self making." The work is a computer-based installation consisting of fifteen robotic sound sculptures that interact with the public and modify their behaviors over time. These behaviors change based on feedback from infrared sensors which determine the presence of the participant/viewers in the exhibition, and the communication between each separate sculpture.
The series of robotic sculptures--mechanical arms that are suspended from an overhead grid--"talk" with each other (exchange audio messages) through a computer network and audible telephone tones. The interactivity engages the participants who in turn effect the system's evolution and emergence. This interaction, according to the artist, creates a system evolution as well as an overall group sculptural aesthetic. The project presents an interactive environment which is immersive, detailed, and able to evolve in real time by utilizing feedback and interaction from audience members.
What are the pedagogical implications for systems such as Autopoiesis that exhibit "emergent properties?" Participant/learners interacting with such systems are challenged to understand that cognition is less a matter of absorbing ready made "truths" and more a matter of finding meaning through iterative cycles of inquiry and interaction. Ironically, this may be what good teaching has always done. So would we be justified in building a "machine for learning" that does essentially the same thing that good teachers do? One argument is that by designing such systems we are forced to look critically at the current manner in which information is generated, shared, and evaluated. Further, important questions are surfaced such as "who can participate"; "who has access to the information;" and "what kinds of interactions are enabled?" The traditional "machine for learning" (the classroom) with its single privileged source of authority (the teacher) is hardly a perfect model. Most of the time, it is not a system that rewards boundary breaking, the broad sharing of information, or the generation of new ideas. It IS a system that, in general, reinforces the status quo. Intelligent machines such as Rinaldo's Autopoiesis can help us to draw connections between multiple forms of inquiry, enable new kinds of interactions between disparate users, and increase a sense of personal agency and self-worth. While intelligent machines will surely be no smarter than their programmers, pedagogical models can be more easily shared and replicated. Curricula (programs for interactions) can be modified or expanded to meet the special demands of particular disciplines or contexts. Most importantly, users are free to interact through the system in ways that are suited to particular learning styles, personal choices, or physical needs.
IMPLICATIONS FOR ART AND EDUCATION
Interactive artworks of the future will enable interactions that are at once personal and universal. These interactions will be characterized by a subtle reciprocity between the body and the natural environment, and an expanded potential for self-knowledg and learning. Truly interactive experiences are already empowering individuals (consider the "disabled" community or autistic learners, for example).
Returning to various theories of interaction (particularly those of Ellen Wagner), several recommendations for artists emerge that begin to trace a trajectory for the education of the interactive artist. They include training on and empowerment with various technologies; understanding media-structured feedback loops (1) and methods for enhancing "mutual recriprocity"; rethinking where meaning is constituted (cognitive theory is now suggesting that "meaning" is seen as something that happens between rather than inside individuals); and redefinition of the roles of educators and learners. Rapid evolution in the art profession as a whole is creating changes in the definitions and roles played by art teachers and prospective artists.
There is no question that the uses of technology outlined here need to be held against the darker realities of life in a hi-tech society. The insidious nature of surveillance and control, the assault on personal space and privacy, the commodification of aesthetic experience, and the ever-widening "digital divide" between the technological haves and have nots are constant reminders that technology is a double edged sword.
But there is at least an equal chance that a clearer understanding of the concept of interaction--specifically interaction enabled by technology--will yield a broader palette of choices from which human beings can come together to create meaning. In watching these processes unfold, educators will surely find new models for learning.
(1) "The feedback loop is perhaps the simplest representation of the relationships between elements in a system, and these relationships are the way in which the system changes. One element or agent (the 'regulator' or control) sends information into the system, other agents act based upon their reception/perception of this information, and the results of these actions go back to the first agent. It then modifies its subsequent information output based on this response, to promote more of this action (positive feedback), or less or different action (negative feedback). System components (agents or subsystems) are usually both regulators and regulated, and feedback loops are often multiple and intersecting (Clayton, 1996, Batra, 1990)." (Morgan, 1999)
Flax, Carol. (2000).
Hillman, D., Willis, D.J. & Gunawardena, C.N. (1994). Learner-interface interaction in distance education: An extension of contemporary models and strategies for practioners. The American Journal of Distance Education, 8(2).
Huhtamo, Erkki (1993). Seeking deeper contact: interactive art as metacommentary.
Moore, M. (1989). Editorial: Three types of interaction. The American Journal of Distance Education, 3(2), 1-6.
Morgan, Katherine Elizabeth (1999). A systems analysis of education for sustainability and technology to support participation and collaboration. Unpublished Master's Thesis at the University of British Columbia. http://www.interchange.ubc.ca/katherim/thesis/index.html
Penny, Simon (1996) Embodied agents, reflexive engineering and culture as a domain. p. 15. (talk given at the Museum of Modern Art in New York City, May 20, 1996)
Penny, Simon. (2000).
Rinaldo, Ken (2000).
Shannon, C.E. (1948). A mathematical theory of communication, Bell System Technical Journal, vol. 27, pp. 379-423 and 623-656, July and October. URL: http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html
Sommerer, Christa and Laurent Mignonneau (1998). Art as a living
system. Leonardo, Vol. 32, No. 3., pp. 165-173.
Sommerer, Christa and Laurent Mignonneau (2000).
Wagner, M. (1994). In support of a functional definition of interaction. The American Journal of Distance Education, 8(2).
Wilson, Stephen. (1993). The aesthetics and practice of designing interactive computer events.
Institute for Studies in the Arts
Arizona State University
Tempe, AZ 85287-3302
return to Dan Collins's bibliography