Notes for Jay David Bolter & Diane Gromala Windows and Mirrors: Interaction Design, Digital Art, and the Myth of Transparency

Key concepts: .

Review of digital art includes condensed argument repeated by Manovich and others.

Related theorists: Brenda Laurel, Brian Cantwell Smith.

Chapter 1. TEXT RAIN: The Digital Experience
The Art Gallery of SIGGRAPH 2000
(41) And digital art is all interface, defined entirely by the experience of its viewing or use. That is why digital art can provide such a clear test of the possibilities and constraints of digital design: it fails or succeeds unequivocally on the strength of its interface.
(41-42) Because each piece is a realization, an embodiment of radical design, it will embody the following three points:
1. The computer has become a new medium (a new set of media forms).
2. To design a digital artifact is to design an experience.
3. Digital design should not try to be invisible.
TEXT RAIN
(42) As the visitor immediately discovers, she herself becomes the show, when her face and figure are caught by the video camera and projected on the screen. Whenever the letters come in contact with the viewer's image, they cease to fall. Whenever the viewer moves, the letters that had collected resume their fall.
(43) TEXT RAIN is a text that is viewer-users help to create, a text that they write in the process of reading. Like the other digital installations in the gallery, TEXT RAIN is about the process of its own making. The letters of the text come from the poem “Talk, You” by Evan Zimroth (1993).
(44) As an experiment in the future of digital technology, it suggests that that future belongs to presentation and representation.
(45) Books, televisions, and computers stake out a cultural territory that is more varied and more mysterious than refrigerators. That's the territory that TEXT RAIN and the other installations of SIGGRAPH 2000 are exploring.

Compare Laurel lack of single essence to essence of technology theorized by Heidegger, then variable ontology of Smith, Bogost.

(45) But Laurel put too much emphasis on one rather specialized media form, the theater. In fact, the computer is not only a new stage for theatrical performance; it can also be a new cinema, a new television, and a new kind of book. The computer does not fuse all its representations into a single form, but presents them in great variety. If there was ever a technology that did not have a single essence, it is the digital computer.

The computer becomes a medium (a new set of media forms)
It's 1949.
(46) Alan Turing came to believe that the essence of human thought is symbol manipulation. . . . For Turing and others who followed him, the computer should not just be a channel for human messages; it should be a thinking machine, capable of producing its own messages.
It's 1954.
(47) Throughout the last half of the twentieth century, the enthusiasts kept insisting that the essence of both human and machine intelligence was symbol manipulation . . . . It was easier to think of the computer as an ersatz human being than as a medium like print, radio, film, or television.
It's 1962.
(47) Their article, “The Computer as a Communication Device,” written in 1968, was one of the first to label the computer as a medium. Licklider and Taylor had seen a very early version of the future, however; Engelbart was still years ahead of his time. In order to make the computer a communication medium for our culture as a whole, two technologies had to be developed and put into widespread use: electronic networks and the personal computer.
It's 1979.
(48) Kay was claiming that the computer was the ultimate medium and could make all other media obsolete. Using his Dynabook, we could create, edit, and store texts; we could draw and paint; we could even compose and score music.
(49) The graphics and sound capabilities of the Macintosh were the key to convincing us that the computer was a medium.
It's 1989.
It's 1993.
(50) The World Wide Web was the final element, creating in the 1990s an audience of millions of viewer-users for the digital experience that networked computers had to offer.

Did Turkle really endorse symbol manipulation as the essence of human being?

(50-51) For decades, AI specialists had fascinated us with the notion that the computer would change what it means to be human. (In 1984, Sherry Turkle summed up their vision in her book The Second Self.) . . . Collectively as a culture, we decide how computers are going to be used – whether as aids to human intelligence (calculators and word processors), replacements for human intelligence (AI applications), or expressive media.
(51) Computer graphics became more compelling to us than numerical analysis and logic programming. . . . Virtual reality (VR) replaced AI in our digital dreams and nightmares, and in VR the old debate about technology and humanity was again redefined. The supporters of AI had insisted that human beings, like computer programs, were information processors. The VR enthusiasts now offered a different definition of human identity that emphasized the senses rather than abstract information processing. They suggested that to be human was to be a bundle of perceptions, a moving and malleable point of view, just what we feel when we are wearing a VR headset.
(51) The Internet realized the vision of Nelson, Engelbart, and Licklider for the computer as a node in a (potentially) global network.
To design a digital artifact is to design an experience
(52) To design a digital artifact is to choreograph the experience that the user will have.
(53) The Web finally presented hypertext as a convincing digital experience. Such a digital experience does not simply enhance the delivery of information. The information itself becomes an experience. Even word processing programs and spreadsheets provide experiences.
(53-54) In a similar fashion, people like to share their adventure on the Web. William Gibson's often quoted description of cyberspace in Neuromancer (2000) really does seem to fit the Web: “a consensual hallucination experienced daily by billions of [human beings]” (51). . . . The form and content of Web pages are inseparable. Pragmatic explanations of the Web as an information system are not wrong, but they are insufficient. The same pragmatic explanation fails to account for the influence throughout the twentieth century of visual design in posters and magazines and on the television. It fails to explain why successful corporations have been willing to pay millions of dollars on branding and design programs for stationary or physical products and why they continue to do so on the Web.
(54) Interaction designers must keep in mind a world of if-then scenarios. . . . Works of digital art are experiments in interaction design. They can afford to be radical experiments because they do not have to meet the (often contradictory) demands of a client. As pure interfaces, they demonstrate the content and form are inseparable. A work of digital art can isolate and explore with clarity the relationship between itself and the user.
(55) The interaction between designer and user through the technology is what gives the experience its meaning. Experience design is also contextual, in the sense that designs must both respond to and shape the many contexts (personal, physical, and cultural) in which they function.
(55) Digital applications offer an experience like that offered by books, films, and photographs: a media experience that is also an “immediate” experience. The essence of digital design is to work on two levels at once – to be both mediated and immediate.
(56) If there is one reason that digital art is important for digital design, it is this: digital art reminds us that every interface is a mirror as well as a window.
Digital design should not try to be invisible
(56) HCI specialists and some designers speak as if that were the only goal of interface design: to fashion a transparent window onto a world of information. There are times, however, when the user should be looking at the interface, not through it, in order to make it function: to activate icons or to choose menu items, for example. At such moments, the interface is no longer a window, but a mirror, reflecting the user and her relationship to the computer.
(57) Popular interest in the process of making films, television shows, and music has increased in recent decades, so that we enjoy all of these media forms as mirrors as well as windows. The same is true of the computer, itself now a medium.
(58) Like other digital artists in the past two decades, those at SIGGRAPH 2000 want us to be aware of the contents in which their individual words in particular and digital technology in general function.
(58) Because computer designers so often assume that the interface should be a window, digital art insists that the interface can also be a mirror. And in the process, it demonstrates the other great strategy of digital design. Those who understand and master both strategies will be more effective designers.

Bolter, Jay David and Diane Gromala. Windows and Mirrors: Interaction Design, Digital Art, and the Myth of Transparency. Cambridge, MA: The MIT Press, 2005. Print.