Windows and Mirrors: Interaction Design, Digital Art, and the Myth of Transparency
Author: Jay David Bolter, Diane Gromala
Publisher: Cambridge, MA: MIT Press, 2003
Review Published: May 2005
This book will be misunderstood. (6)
Jay David Bolter is known to many for his seminal Writing Space: The Computer, Hypertext, and the History of Writing (1991), as co-author (with Richard Grusin) of the provocative Remediation: Understanding New Media (1999), and as co-creator (with Michael Joyce, John B. Smith, and Mark Bernstein) of the groundbreaking hypertext writing software Storyspace (1987-2005). Diane Gromala, a fellow professor of Bolter's at Georgia Tech, is a widely-exhibited digital artist. As chair of the SIGGRAPH 2000 Art Committee, she curated the conference's annual Art Gallery exhibition, whose works form the structural backbone of Windows and Mirrors.
Which brings us to the charming conceit: the whole thing is structured around a chapter-by-chapter walking tour, conducted by the authors, of the SIGGRAPH 2000 Art Gallery. At the beginning of each chapter they describe an exemplary work of digital art, then they spin various aspects of their argument off from that narrative thread. They enhance the text with a generous selection of photographs, and they sprinkle it with sidebars of anecdotes, parables, and historical tidbits. In the process, they engage topics such as the history of computer interfaces, ubiquitous computing, artificial intelligence, and virtual reality.
For example, Chapter 2 (Wooden Mirror: The Myth of Transparency) opens with a description of Wooden Mirror by artist Daniel Rozin:
To Bolter and Gromala, "the danger of transparency" can be quite literal. Somewhat melodramatically, they cite the 1979 Three Mile Island nuclear reactor meltdown as an example of bad interface design, because "the operators did not question their interface" when an indicator showed the amount of coolant increasing when it was actually decreasing (54). By contrast, the early Macintosh GUI was "a triumph of modernist clarity and simplicity" (46), although the goal of transparent, "natural" interfaces is increasingly unreachable as systems grow more complex. Virtual reality 3-D interfaces, too, are based on the myth of transparency, the impossible notion "that a technology can disappear completely and put the viewer or user in touch with reality" (52). Instead of striving for the undesirable and impossible, the authors argue, designers should heed the tension or oscillation between the transparent and the visible aspects of interfaces.
Other chapters follow a similar pattern. Chapter 4 (Magic Book: The New and the Old in New Media) begins with a description of Magic Book, by the Human Interface Lab at the University of Washington, a "digital pop-up book" that adds computer-generated augmentation (3-D effects) to the physical artifact of a children's book. Chapter 6 (T-Garden: The Materiality of New Media) opens with T-Garden, by the digital art group Sponge, a physical and virtual dance space in which participants' actions trigger audio and video changes to the environment. Each chapter uses the opening example as a springboard for engaging discussions of topics such as remediation or "the making of new media forms out of older ones" (83), or design philosophy in a world of tangible media and mixed realities.
The walking tour alone is worth the price of admission. It's hard to imagine a more thoughtful pair of guides than Bolter and Gromala, and the digital art works they describe are truly exciting, occasionally radically so. As for their argument, it accrues a kind of persuasiveness through repetition. As they apply their thesis across fields ranging from Web design to virtual reality, however, they add texture and detail but not very much depth. As a result, the reader may end up feeling as if the authors are turning a stone over and over in their hands, and while some of the facets and fissures make for fascinating digressions, it's still the same stone at the end.
Bolter and Gromala unfortunately weaken their thesis by making a very big deal, too big a deal it turns out, about ubiquitous computing and disappearing interfaces. As if right on cue for this review, the Communications of the ACM -- the flagship publication of the Association for Computing Machinery, the leading information technology group which also happens to be the parent organization of SIGGRAPH -- devoted much of its March, 2005 issue to "The Disappearing Computer." A dozen articles on current approaches to ubiquitous computing pay obeisance to Mark Weiser's pioneering ubicomp manifesto, published in Scientific American in 1991, which predicted that computers would soon fade into the background. The guest editors for the special section, Norbert A. Streitz and Paddy Nixon, write of interaction design:
To effectively "demonstrate the analytic power of the dichotomy" -- as Patrick J. Cook (1999) writes in his RCCS review of Bolter's previous book Remediation -- Bolter and Gromala need to overcome the false opposition they have set up of transparent windows/reflective mirrors. They never entirely succeed in doing so. If we grant for a moment the Us versus Them battle analogy, the authors acknowledge that "ubicomp seems to be winning" (107), and they're right. Forrester Research and IDC predict that "noncomputer objects will soon account for the vast majority of networked devices," and those noncomputer clients will shortly dominate network traffic (Thompson, 2005). Bolter and Gromala argue, though, that Weiser and the others are wrong to predict the demise of the intruding, visible interface. Most of our ubiquitous devices, they reasonably point out, are consumer appliances designed specifically to capture our attention, not diffuse it by "disappearing" (107-108).
It's a great leap, however, from camera phones and iPods to "visible media" that "reflect the world around them" in any way comparable to digital art like Wooden Mirror, Magic Book, or T-Garden. Is it even reasonable to ask that our PDAs or DV cameras function to heighten our awareness of cultural contexts, as the authors suggest that digital art should do? This conflation of consumer devices and entertainment appliances with the subversiveness of art goes to the heart of the weaknesses in Bolter and Gromala’s argument. Common sense tells us that we want our consumer devices to be transparent, meaning easy to use. HCI designers are not the enemy. The impossible-to-program VCR, anything but transparent, is only one famous example of bad interface design (hopefully Bolter and Gromala would not have replaced the "transparent" Three Mile Island reactor controls with the on-screen programming of the typical VCR; doing so might have precipitated a far worse disaster!). Maybe one day that poorly designed VCR will be preserved in a museum, but not as a work of art.
The authors say their purpose is to offer lessons from digital art to the community of "digital designers and technologists in general," and they define this audience specifically as
The fact is that as consumers -- even consumers "interested in the cultural implications of the digital revolution" -- we don't want VCRs or cell phones or the vast majority of the Web sites we visit to be experiments or works of art. We don't want them to have visible, reflective interfaces that turn us thoughtfully back upon ourselves; we vastly prefer the functionality espoused by Jakob Nielsen. As educational technologists, we don't want our technology-enhanced classrooms to function like T-Garden, and we don't want our course management systems to look like Fakeshop, with its "multiple windows, offering streams of information (text, stills, and video), reported through a richly layered interface" (96).
Bolter and Gromala regularly descend into common sense and allow that a large degree of transparency may indeed be necessary and desirable, that the point is actually to achieve a balance. Surely most readers will agree when the authors remind us of their thesis that "Good designs oscillate between hiding and revealing themselves" (68) or say that "Perhaps the only rule is that the design of each [Web] site should suit its envisioned community of users" (103). But their pendulum swings so far and so frequently against the poor designers that they obscure rather than clarify their overall argument. In doing so, they sometimes add to the confusion by using inconsistent terminology. In one version of the Us versus Them parable, "Designers" (in this case, those who began adding thoughtful graphical design to the Web) are the good guys, not the bad guys, and "Structuralists" (those who want the Web to focus on information content) are the enemies -- as if these two positions were incompatible.
The ambitiousness of Bolter and Gromala's project leads to some further confusions and inconsistencies. In one chapter they debunk "the myth of disembodiment" and "the long prejudice against the body" they claim is held by artificial intelligence aficionados and cyberspace utopianists (117-123), while seeming to forget that the whole enterprise of HCI and interaction design they criticize earlier is about designing things for humans as physical creatures. They write compellingly about new media forms, tools, artifacts, and appliances, but the distinctions among these tend to blur (e.g., "so-called information appliances are usually media forms," 112) and I often wished the authors had provided a more disciplined topology or set of working definitions. For example, the following distinctions come to mind: Wooden Mirror is a product or artifact, an artwork that arguably functions in a subversive way, by effectively (artistically) managing the balance between seeing through and reflecting back. To make Wooden Mirror, artist Daniel Rozin used various tools or media, whose degree of transparency, I'd suggest, may have little relevance to the final product. But Bolter and Gromala seemingly want us to apply their aesthetic -- yes, it is essentially an aesthetic -- to everything, that is, to all the tools and interfaces and media as well as to the artifacts made from them or using them.
In my own experience as a digital art practitioner, I would say that the beauty of Storyspace, the hypertext writing application co-created by Bolter and now the product of Eastgate Systems, is nearly proportional to its transparency -- the ease with which the Storyspace author can produce hypertextual nodes, complex linking of text and nodes, and a graphical map of the work's structure. It is then up to me as the artist, using that interface, to build the rhythm or tension between transparency and reflectivity that, I agree with Bolter and Gromala, characterizes the most successful art.
Or let's look at a medium or media form, to which the authors ascribe the same standards as an interface or software application. For the print medium in general, we don't want most of the books we read to call attention to the conventions of linear lines and pages and bound books; we prefer that interface to be essentially invisible nearly all the time. We read "through" the print to the content or meaning. We read forms like novels without questioning their conventions around plot or character or how the pages look. But we may occasionally love a work of art in book form that makes the interface visible, or plays with the tension between visibility and invisibility, the way that Lawrence Sterne's Tristram Shandy (1759-1765) systematically subverts the conventions of the novel. Or the way that Marc Saporta's so-called "book in a box," Composition No. 1 (1962), subverts the book medium itself by piling its unbound pages in a container and instructing the reader to shuffle them like a deck of cards before reading them.
I first learned about Composition No. 1 from Bolter in Writing Space. In that earlier book, he writes acutely about Saporta and Sterne and other writers who operate in that delightful aesthetic zone of oscillation between seeing through and reflecting back -- but in Windows and Mirrors this aesthetic is oddly dehistoricized. After all, the tension between "windows" and "mirrors" is really a dressed-up-for-the-digital-age version of Content versus Form, and art has arguably always been about the tension or interaction between content and form. But what does it really mean to design for both transparency and reflectivity, especially using new media? What are the deeper implications for both designers and participants? Many readers, like me, may be easily persuaded and genuinely seduced by the metaphor, but then hunger for more as it begins (to mix my own metaphors) to ring hollow.
This hollowness has been criticized by other reviewers of Windows and Mirrors as a lack of theoretical rigor. Bolter and Gromala explicitly eschew critical theory in this book because it "often strays too far from practice," but then, somewhat weirdly, they specify a list of critics with whom they are acquainted but whom they "choose not to discuss" (such as Haraway, Lacan, and Baudrillard) (x-xi). I did find myself wondering how they could spend most of Chapter 4 elaborating the concept of remediation without a single mention of McLuhan. Jan Baetens, in Electronic Book Review (2004), suggests that Windows and Mirrors is essentially a rephrasing or application of Remediation -- whose "main target . . . is definitely Marshall McLuhan" -- to the fields of digital art and design, mapping the earlier book's concepts of immediacy and hypermediacy onto the present book's transparency and reflectivity. But in Windows and Mirrors, Baetens suggests, the theory is taken for granted rather than analyzed. Similarly, Kelly McLaughlin, reviewing Windows and Mirrors for the Iowa Review Web (2004), writes, "Deciding that designers are to be the specific audience of this book in no way alleviates the need for critical and theoretical depth." I think the authors could have forestalled such reactions and still met their goal of addressing a more general audience by simply developing their argument in more depth on the journalistic or expository level, by relying less on the oversimplification of their primary metaphor, and by selectively contextualizing their argument with accessible references to major critics. Bolter and Gromala are too good as writers to succumb to the jargon of critical theory at its most unfathomable.
Readers of Windows and Mirrors may share with Baetens and McLaughlin a disappointment with Bolter and Gromala's final ambition -- expressed in a Colophon which really serves as a tenth chapter -- to have their book itself serve as a model of its thesis. In one way, the book's structure does reflect the theme by oscillating between "reflective" narrative engagements with digital art objects and "transparent" expository passages asserting an argument. The Colophon describes the final digital art exhibit, called Excretia, a "dynamic digital typeface" designed by Gromala herself. Excretia is a "biomorphic" font displayed on a computer screen while the reader is hooked up to a biofeedback device, so that its appearance constantly changes in response to the reader's physical state as measured by galvanic skin response. Thus with this typeface, the text itself, irrespective of the content, becomes a reflective interface, feeding back information about the reader during the process of reading.
In the Colophon, we learn that Excretia has been used in the book as the typeface for the main chapter headings. Well, sort of -- because of course print text cannot capture the dynamic properties of Excretia any more than Windows and Mirrors can measure electrodermal changes in the reader's sweat gland ducts. The book's design is otherwise entirely conventional (if beautifully laid out), so the authors' ambition that the book be self-reflexive in this way comes off as extravagant. The only really reflective interface here is the shiny-silver book jacket. To read the embossed text on the front of that reflective surface, the reader must tilt the book back and forth to catch the light at the right angle.
The joys and limitations of Excretia are indeed those of the book. The digital art, expertly portrayed and lovingly described, engages us in its radical experimentation. The ideas extrapolated from each exhibit are equally engaging, but the framework the authors build of those ideas doesn't quite hold the weight they pile upon it.
Baetens, J. (2004). A remediation's remediation? Electronic Book Review (online).
Bolter, J.D. and Grusin, R. (1999). Remediation: Understanding New Media. Cambridge, MA: MIT Press.
Bolter, J.D. (1991). Writing Space: The Computer, Hypertext, and the History of Writing. Hillsdale, NJ: Lawrence Erlbaum.
Cook, P. J. (1999, December). Review of Bolter and Grusin, Remediation: Understanding New Media. The Resource Center for Cyberculture Studies (online).
Eastgate Systems (1987-2005). Storyspace and Storyspace 2 [software].
McLaughlin, K. (2004). Review of Bolter and Gromala, Windows and Mirrors. The Iowa Review Web (online).
Norman, D. (1998). The Invisible Computer: Why Good Products Can Fail, the Personal Computer is so Complex, and Information Appliances Are the Solution. Cambridge, MA: MIT Press.
Streitz, N. and Nixon, P. (2005). The disappearing computer (introduction by guest editors). Communications of the ACM (vol. 48, no. 3), 33-35.
Thompson, M. J. (2005). Invisible computing is hard to miss. Technology Review 108:2 (February): 86.
Weiser, M. (1991). The computer for the 21st century. Scientific American 265, 3 (September): 94-104.
Richard Holeton is head of Residential Computing at Stanford University, where he formerly taught in the Writing and Critical Thinking program and English department for 12 years. He is the author of the hypertext novel Figurski at Findhorn on Acid (Eastgate Systems, 2001), short fiction, and scholarship, and editor of several college textbooks including Composing Cyberspace: Identity, Community, and Knowledge in the Electronic Age (McGraw-Hill, 1998). <email@example.com>
|HOME INTRO REVIEWS COURSES EVENTS LINKS ABOUT|
|©1996-2007 RCCS ONLINE SINCE: 1996 SITE LAST UPDATED: 12.10.2009|