HomeIntroducing CybercultureBook ReviewsCourses in CybercultureEvents and ConferencesFeatured LinksAbout RCCS

View All Books

Literacy in the New Media Age

Author: Gunther Kress
Publisher: New York: Routledge, 2003
Review Published: January 2006

 REVIEW 1: Karim A. Remtulla

    "The world told is a different world to the world shown" (1).
This statement succinctly expresses the central premise for Gunther Kress' Literacy in the New Media Age. The shift in modalities of communication and representation from writing to image, from the medium of the page to that of the screen, from meaning conceived through a logic of time to meaning perceived through a logic of space, and their cumulative consequences for power, politics, economics, culture, and epistemology, all are for Kress attributes pointing to the essential transformation happening in our conceptions of literacy.

As Kress puts it, "there is an urgent need for theoretical accounts that tell us how to understand communication in periods of instability" (11). Kress' intention is to provide a conceptual framework for thinking about literacy and how it is changing (8). Through a theoretical mapping of the transformation in literacy, Kress presents a detailed line of reasoning of how our interactions with each other and the world around us may change as a result of these new realities.

The foundations for Kress' rationale are laid out in Chapter One, "The futures of literacy: Modes, logics and affordances," Chapter Two, "Preface," and Chapter Three, "Going into a different world."

According to Kress, literacy "is the term to use when we make messages using letters as the means for recording that message," and "remains the term which refers to . . . the use of the resource of writing" (24). Making meaning involves modes and media. Modes are the resources for making meaning and include writing and image. Media disseminate meaning as messages and include book and computer-screen (22). Focusing on writing and image, with each mode's inherent affordances, Kress highlights how each mode is facilitated through traditional and new media.

Kress situates his discussion of literacy and how it is changing within the context of broader changes in society. Social changes are destabilizing forms of writing; economic changes are changing the uses and purposes of writing technology; communicational changes are increasingly relying on image instead of writing for representation and communication; and, lastly, technological changes are encouraging the screen and not the page as the primary medium for writing and image to flourish (9). Kress further adds that this shift in mode from writing to image is also intensified by globalization, computer-mediated communications, and a post-industrial world where information is both a resource and commodity.

In conceptualizing a multimodal framework for literacy and new media, Kress argues for a decidedly semiotic approach. Writing, with its origins in sound and alphabet, is materially different from image with its reliance on light and space. The logic of writing is temporal and sequential whereas the logic of image is spatial and simultaneous; writing follows "syntax" and image is ordered according to "display." The combining of writing and image ultimately results in what Kress suggests will be the "functional specialization" of modes, where neither writing nor image will carry the full weight of meaning and each mode will be used for only those tasks that it is best suited (21). The new technologies will then work with various modes and their affordances, through the dominance of the medium of screen, enabling interactivity and bidirectionality in representation and communication by converging at one site the resources of representation and dissemination.

Kress also deliberates whether a changing framework for literacy must include character-based writing systems, especially given the increasing use of image. This not only impacts the way we will communicate and represent our world, but also how we understand and structure knowledge as a society. Cultures that rely on character-based icons (e.g. Chinese hanzi and Egyptian hieroglyphs), where characters are inscribed with meaning, are markedly different for the West's conceptions of the alphabet as a technology for transcribing sounds:
    The imaginative work in writing focuses on filling words with meaning -- and then reading the filled elements together, in the given syntactic structure. In image, imagination focuses on creating the order of arrangement of elements which are already filled with meaning. (4)
What Kress strives to establish is the need for a new, multimodal vision of literacy as a system of transcription for sounds and ideas which builds on the materiality of multiple modes and responds to the complex and changing needs of society for meaning and meaning-making. Still, Kress concedes that there are those who counter his point of view. Some argue that book publishing and writing are more prominent now than ever before and much stands to be lost if writing disappears as the dominant and culturally favored form of representation and communication (7).

The discussion then turns in earnest to literacy and how to think about building a new theory of literacy and what elements such a theory might include. The basic questions such a theory will need to address are obvious: What is literacy? and what is writing?

Chapter Four, "Literacy and multimodality: a theoretical framework," addresses the confluence of literacy and its traditional grounding in the mode of language (i.e. speech and writing) with the many, newer forms of representation and communication. Kress captures his motivation by noting, "a linguistic theory cannot provide a full account of what literacy does or is; language alone cannot give us access to the meaning of the multimodally constituted message; language and literacy now have to be seen as partial bearers of meaning only" (35). Further, Kress contends, "we need a theory which is not specific to, or derived from, one mode but which applies to all modes" (41).

Semiotics enables, for Kress, a theory that can account not just for language but also speech, image, writing, color, music, and others. Of particular interest here is Kress' specific take on semiotics, form, and meaning, as it differs from those of Saussure and Pierce. Whereas Saussure takes the sign to be an arbitrary combination of form (signifier) and meaning (signified) enforced by social convention, Pierce concentrates on the relationship between the sign, the interpreter, and the object (i.e. that which is represented). Pierce distinguishes between iconic signs, indexical signs, and symbolic signs where each denotes a distinct type of relationship. For Pierce, there was no meaning without interpretation.

Kress rejects Saussure's idea of arbitrariness but draws on Pierce's notion of iconic signs, which in their form are analogous to the meaning of the signified, "as the model of all relations of signs to their referents" (42) and assumes that "the shape of the signifier . . . is chosen because of its aptness for expressing that which is to be signified" (42). The relationship between signifier and signified is motivated by the maker. The meaning to be expressed is shaped by what the maker understands as the needs and limitations of the socio-cultural environment within which communication is desired as well as the material affordances of the multiple modalities available to the maker.

Meaning must also be reconsidered given a multimodally, socially constituted message. Modes are inseparable from cultural, social, material, affective, and cognitive aspects of their relations with sign-makers. Time-based modes (e.g. speech and music) and their logic of temporal succession to infer causality and create meaning are different from space-based modes (e.g. image and sculpture) and their logic of the spatial spreading of simultaneously present elements to constitute meaning (45). Knowledge changes its shape when realized in materially different modes, implying that multimodality has deep implications for epistemology.

With this semiotic lens, Kress subsequently takes a cultural stance on modality and associated concepts such as modal specialization, functional load, materiality, and affordance including notions of transformation of forms and structures within a mode as well as the transduction of semiotic material across modes. Text is also revisited here including a review of the principles of organization and shaping of text. In this way, Kress further initiates new ways of thinking about learning and creativity as they relate to meaning, modality, and new theories of literacy:
    The theory demands that we assume that all aspects of a sign represent their maker's interest in representing that which they regard as most salient, at this moment, about the object or phenomenon to be represented. It entails all aspects of form are meaningful, and that all aspects must be read with equal care: nothing can be disregarded. (44)
In Chapter Five, "What is Literacy? Resources of the mode of writing," Kress takes a look at writing and its possible futures. Where writing goes, based on social and not technological factors, so too will our notions of literacy, "in that it always reflects social issues: who is in control, what is being controlled and what is that control being used for?" (73). In this part of his analysis, Kress looks specifically at the sentence as the resource for writing and as the result of textual and social processes:
    What all the examples illustrate is the responsiveness of the resource to the needs of the writers in their social and cultural environments, and the socially shaped characteristic of that often too abstractly thought about notion of the sentence. (83)
Writing systems serve two distinct purposes: an alphabetic system of transcription for sounds where meaning is subsequently attributed to combinations of sounds; and, image-based systems where meaning is attributed to characters first after which sound is attached to meaning. Accepting the growing dominance of the screen, Kress asserts that the textual entity is now treated as a visual entity and that images now accompany writing on the page as on the screen, thus giving spatial logic precedence in meaning-making over temporal logics. This leads to a substantial realization that "the placement of letter or word, the shape of the letter or its size, all these now need to be treated as signs" (66). Visually, writing will move away from speech and in the direction of image; textually, writing is likely to become more like speech (73).

Chapter Six, "A social theory of text: Genre," emphasizes the importance of understanding what text is, what it does, and that text is the result of social action (84). Genre is introduced as a social theory of text and the impacts that multimedia and multimodality will have on such a theory are articulated. Genre identifies the participants in this social action and their social relations and is understood as a social category in the organizing of text; it gives insight to the social relations of the participants in the making and the use of a text. As such, "When that social action is looked at from the point of view of representation, we invoke the category of text. Text is the result of the social semiotic action of representation" (84).

Seeing text and genre in this way also means that text, and the genre of which it is a part, may be structured in hierarchies of power with conditions of convention required for acceptance and participation in social life. However, the complete stability of genre is a virtually non-existent for Kress (98). As Kress explicates:
    we see text -- not letter, not word, not clause or sentence -- as the central category in literacy. Text is the result of social action, and so the centrality of text means that literacy is always seen as a matter of social action and social forces, and all aspects of literacy are seen as deriving from these actions and forces. (86)
The structure of the text reveals the social world involved in the making of the text. The social world is not a static place; as such, neither is text nor genre.

Chapter Seven, "Multimodality, multimedia and genre," continues this discussion of text, genre, and social action, and juxtaposes the issues of mode and media. If genre is a social theory of text, then can it equally be applied to other modes of representation and communication such as image? For Kress, "the answer is that the category of genre is essential in all attempts to understand text, whatever the modal constitution. The point is to develop a theory and terms adequate to that" (107).

The presence of multimedia affords the maker of text a plethora of choices with respect to modalities and genres and various combinations of both as mixed genres. The functional specialization of modes allows each modality to carry only that part of the message for which it is most aptly suited. Creativity is a normal process of representation necessary for a plural society where the generic forms of all cultures co-exist and intermingle made possible by the affordances of multimedia. The issue of a text as a mixed genre is not as important as are the social meanings behind the text that reveal the generic nature of the text and the motivation of the writer.

Kress rounds out his conceptual framework for discerning new theories of literacy with a discussion on semiosis. This includes Chapter 8, "Meanings and frames: Punctuations of semiosis," and Chapter Nine, "Reading as semiosis."

Here, the issue of how punctuation might fit into a multimodal theory of literacy is examined. "Punctuation is a semiotic resource available for making social meanings," used by the writer to "frame" the social world they conceive around them (124). The operative function here is framing and how it is materialized across multiple modes and what it might suggest about the social affiliation of the writer and their intended audience. Framing in the mode of speech, for example, is accomplished through clauses and materialized through voice. Framing in the mode of writing is accomplished through sentences and materialized through graphic devices such as capitals, spacing, and layout. Framing in multimodal communication and representation is more complex. Kress puts forward two notions to assist framing in multimodal communication: the notion of the "block" as a kind of visual arrangement, and the notion of "entry" as a principle of approach (136).

In the end, Kress turns to reading and explores increasing multimodal forms (e.g. image and writing) of text, dominance of the screen, and their impact on our notions of reading:
    As I have been demonstrating so far, many contemporary texts make use of image and of writing at the same time, using both to carry meaning in specific ways. In that context, a theory of reading which relates to the graphic material of "letters" alone is no longer able to explain how we derive meaning from texts. (141)
The difference lies in, "the world represented as a sequence of action or event versus the world represented as objects and their relations" (154). Kress brings in his idea of "reading path," which comprises the elements in a page (or screen) and the direction readers choose to follow to derive meaning, based on principles of relevance (162). Kress reinserts his notion of "functional specialization" here, such that objects existing in space in the real world are best communicated by image and events in action and time by writing. All this has profound consequences for currently accepted notions of reading:
    It will be essential to look with great care at the "semiotic affordances" of image, of writing and of speech, and of multimodal texts to see how the relative powers of makers and receivers of texts are reconfigured in this new dispensation. (166)
In the final chapter, Chapter Ten, "Some items for an agenda on further thinking," Kress briefly shares some of the more panoramic uncertainties that flow from this discussion on the future of literacy. He emphasizes continued investigation of the necessity of theories of meaning in a multimodal, pluralistic, multimedia world in social and economic flux; the assumptions of rationality and arbitrariness in language accepted as part of current linguistic theory; the role of the affective in interpretation; and the role of motivation behind form and meaning. He encourages further interrogation around consequences for notions of creativity, competence, and imagination; the implications for authorship, authority and knowledge; and the repercussions for the future of pedagogy, curriculum, and epistemology of all these issues.

Gunther Kress' Literacy in the New Media Age signifies an intricate and thorough elucidation of Kress' thinking. It speaks to the core of social-semiotic discourse and its future, and drafts a framework for researchers to think about as they develop new theories of literacy in response to the social, economic, technological, and cultural changes taking place in society. How literacy is understood within the context of these complexities will have a direct bearing on society's conceptions of knowledge and epistemology as well as perceptions of pedagogy, curriculum, communication, representation, skill, and competence. These multiple modalities, whether written or image-based or others, will increasingly translate the criss-crossing of global cultures, the interpretations of sounds and meanings, and the relevance of page and screen, ultimately bearing on self-expression and identity in a multitude of ways as of yet unexplored.

Karim A. Remtulla:
Karim A. Remtulla is a doctoral student at Ontario Institute for Studies in Education at the University of Toronto. His research focuses on the impact of the virtual on adult learning and identity.  <kremtulla@oise.utoronto.ca>

©1996-2007 RCCS         ONLINE SINCE: 1996         SITE LAST UPDATED: 12.10.2009