Multimodal Epistemologies

This volume develops a new multimodal semiotic approach to the study of communication, examining how multimodal discourse is construed transmedially and interculturally and how new technologies and cultural stances inform communicative contexts across the world. It contributes to current theoretical debates in the disciplines of semiotics, linguistics, multimodality, and pragmatics, as well as those aspects of pedagogy and film studies that engage with the notions of text and narrative by addressing questions such as: How do we study multimedia communication? How do we incorporate the impact of new media technologies into the study of Linguistics and Semiotics? How do we construe culture in modern communication? How useful are the current multidisciplinary approaches to multimodal communication? Through the analysis of specific case studies that are developed within diverse academic disciplines and which draw on a range of theoretical frameworks, the goal of this book is to provide a basis for an overarching framework that can be applied by scholars and students with different academic and cultural backgrounds.

This volume develops a new multimodal semiotic approach to the study of communication, examining how multimodal discourse is construed transmedially and interculturally and how new technologies and cultural stances inform communicative contexts across the world. It contributes to current theoretical debates in the disciplines of semiotics, linguistics, multimodality, and pragmatics, as well as those aspects of pedagogy and film studies that engage with the notions of text and narrative by addressing questions such as: How do we study multimedia communication? How do we incorporate the impact of new media technologies into the study of Linguistics and Semiotics? How do we construe culture in modern communication? How useful are the current multidisciplinary approaches to multimodal communication? Through the analysis of specific case studies that are developed within diverse academic disciplines and which draw on a range of theoretical frameworks, the goal of this book is to provide a basis for an overarching framework that can be applied by scholars and students with different academic and cultural backgrounds.

This collection highlights diverse epistemological perspectives in original research on the important role of multimodality in second language contexts. The volume explores a wide range of theoretical and methodological traditions toward foregrounding the notion that bodily action is not merely an add-on to the modality of talk but an integral part of second language teaching, learning, and interaction. Following an introductory chapter, 18 empirical chapters feature either classroom or non-classroom research, which shed light on different dimensions of multimodality in second language contexts, including learning reflected in gesture, learning gesture across languages, the role of bodily action in language teaching, and the role of movement in configuring space for effective communication. Each empirical chapter follows a consistent structure detailing the research focus, the background to each study, methodology, and findings. A concluding synthesis chapter braids the insights of these chapters, drawing parallels across different methods, and pointing toward crosscutting areas for future research. This book will be of interest to students and scholars in applied linguistics, multilingualism, bilingualism, gesture studies, cognitive science, and psychology. Chapter 10 of this book is available for free in PDF format as Open Access at www.taylorfrancis.com. It has been made available under a Creative Commons Attribution-NonCommercial-ShareAlike (CC-BY-NC-SA) 4.0 International license.

This book develops a new framework for describing the structure of multimodal documents: how language, image, layout and other modes of communication work together to convey meaning. Building on recent research in multimodal analysis, functional linguistics and information design, the book examines the textual, visual, and spatial aspects of page-based multimodal documents and employs an analytical model to describe and interpret their structure using the concepts of semiotic modes, medium and genre. To demonstrate and test this approach, the study performs a systematic, longitudinal analysis of a corpus of multimodal documents within a single genre: an extensively annotated corpus of tourist brochures produced between 1967-2008. The book provides multimodal discourse analysts with methodological tools to draw empirically-based conclusions about multimodal documents, and will be a valuable resource for researchers planning to develop and study multimodal corpora.

This volume presents innovative research on the multimodal dimension of discourse specific to academic settings, with a particular focus on the interaction between the verbal and non-verbal in constructing meaning. Contributions by experienced and emerging researchers provide in-depth analyses in both research and teaching contexts, and consider the ways in which multimodal strategies can be leveraged to enhance the effectiveness of academic communication. Contributors employ both quantitative and qualitative analytical methods, and make use of state-of-the-art software for analyzing multimodal features of discourse. The chapters in the first part of the volume focus on the multimodal features of two key research genres: conference presentations and plenary addresses. In the second part, contributors explore the role of multimodality in the classroom through analyses of both instructors’ and students’ speech, as well as the use of multimodal materials for more effective learning. The research presented in this volume is particularly relevant within the context of globalized higher education, where participants represent a wide range of linguistic and cultural backgrounds. Multimodal Analysis in Academic Settings contributes to an emerging field of research with importance to an increasing number of academics and practitioners worldwide.

This book is a first attempt to map the broad context of performance studies from a multimodal perspective. It collects original research on traditional performing arts (theatre, dance, opera), live (durational performance) and mediated/recorded performances (films, television shows), as well as performative discursive practices on social media by adopting several theories and methodologies all dealing with the notion of multimodality. As a mostly dynamic and also interactive environment for various text types and genres, the context of performance studies provides many opportunities to produce meaning verbally and non-verbally. All chapters in this book develop frameworks for the analysis of performance-related events and activities and explore empirical case studies in a range of different ages and cultures. A further focus lies on the communicative strategies deployed by different communities of practice, taking into account processes of production, distribution, and consumption of such texts in diverse spatial and temporal contexts.

This collection advances the study of context-dependent characteristics of argumentative discourse by examining a variety of media genres in which text and image (and other semiotic modes) combine to create meaning. The chapters have been written by an international group of senior and junior scholars researching multimodal argumentation in the last two decades. In each chapter, a specific approach to argumentation and rhetoric is combined with insights from visual studies, metaphor theory, scientific visualization, cognitive science, semiotics, conversation analysis, or (documentary) film theory in order to explain how multimodal genres function argumentatively and rhetorically. Together the chapters present a state-of-the-art in the analysis of multimodal argumentation in such diverse genres as print advertisements, news photographs, scientific illustrations, political cartoons, documentaries, film trailers, political TV advertisements, public debates, and political speeches. The volume will be of interest to advanced students and scholars in argumentation studies, rhetoric, and multimodal communication.

Metaphor and metonymy appeal to us because they evoke mental images in unique but still recognisable ways. The potential for figurative thought exists in everyone, and it pervades our everyday social interactions. In particular, advertising offers countless opportunities to explore the way in which people think creatively through metaphor and metonymy. The thorough analysis of a corpus of 210 authentic printed advertisements shows the central role of multimodal metaphor, metonymy, and their patterns of interaction, at the heart of advertising campaigns. This book is the first in-depth research monograph to bring together qualitative and quantitative evidence of metaphor-metonymy combinations in real multimodal discourse. It combines detailed case study analyses with corpus-based analysis and psycholinguistic enquiry to provide the reader with a prismatic approach to the topic of figurative language in multimodal advertising. Besides its theoretical contribution to the field of multimodal figurative language, this monograph has a wide number of practical applications due to its focus on advertising and the communicative impact of creative messages on consumers. This book will pave the way for further qualitative and quantitative research on the ways in which figurative language shapes multimodal discourse, and how it relates to our everyday creative thinking.