Building Bridges For Multimodal Research

The book takes differences in multimodality research as a starting point to discuss old and new theoretical, methodological as well as analytical ideas for building bridges between various disciplines and approaches.

Multimodality is one of the most popular and influential semiotic theories for analysing media. However, the application and conceptual anchoring of multimodality often remains geographically and disciplinarily grounded within local systems of thought. New Studies in Multimodality combines the expertise of multimodalists from around the globe, offering novel readings and applications of central concepts in multimodality and inviting innovative synergies between previously disparate schools. Combining perspectives from the most actively developing traditions of theory and research, this book progresses from classic concepts to more empirically and practice-motivated contributions. Contributors engage in mutual dialogue to present new theoretical perspectives and compelling applications to a variety of old and new media. Expanding the basis and scope of multimodality, this volume shows awareness and experience of this field in many disciplines and illustrates how versatile, pervasive and relevant it is for studying today's communication phenomena.

Literacy learning continues to be central to schooling, and is currently of major concern to educators, policy developers, and members of the public alike. However, the proliferation of communication channels in this digital era requires a fundamental re-thinking of the nature of literacy and the pedagogy of literacy teaching and teacher education. This text brings together papers by experts in teacher education, literacy, and information technology to help chart a way forward in this complex area. Because of their background in teacher education, the authors are realistic about what is appropriate and feasible – they do not just jump on a technology bandwagon – but they are also able to provide extended examples of how to embed technology in the practice of teacher education. “Taking a multi-disciplinary perspective (literacy, teacher education and digital technology) and informed by a range of empirical studies, policy analyses and scholarly reflection, this book makes a unique contribution to the literature on one of education’s most pressing challenges: how we prepare teachers of literacy at a time when understandings of literacy are expanding. Chapters by leading researchers are complemented by those offering illuminating vignettes of practice that, in turn, provide opportunities for interrogation by the rich theoretical toolkit that characterizes the field. The book is thoughtfully structured and manages a coherence that is rare in edited collections. An impressive and heartening read.” – Viv Ellis, Professor of Education at Brunel University, England and Bergen University College in Norway

Beyond the Visual is a survey of contemporary approaches to researching a wide range of visual and multimodal phenomena. Building on his earlier book, Reading the Visual, Serafini shares resources for conducting multimodal research across the social sciences. Beginning with a comprehensive overview of the theoretical foundations that support the analytical frameworks, the text is organized into two parts—texts and objects, events and spaces—with corresponding analytical approaches. Examples and outlines are provided to help novice and experienced researchers conduct their own studies. Vignettes by some of the most renowned scholars in the field of multimodality research take the reader behind the scenes of various projects to experience the thoughts and decisions that go into conceptualizing and applying the analytical frameworks presented in the book. This resource will enable both students and experienced scholars to acquire new research skills and designs resulting in more rigorous, high-quality research. Book Features: Assists researchers and educators to make better connections among theoretical orientations, analytical frameworks, and research designs. Showcases 16 models for conducting research on visual and multimodal phenomena across a variety of social, virtual, and physical contexts. Provides examples of how eminent researchers conceive, design, initiate, and conduct their studies. Explores the research methods cited in the author’s previous book, Reading the Visual.

This book develops a new framework for describing the structure of multimodal documents: how language, image, layout and other modes of communication work together to convey meaning. Building on recent research in multimodal analysis, functional linguistics and information design, the book examines the textual, visual, and spatial aspects of page-based multimodal documents and employs an analytical model to describe and interpret their structure using the concepts of semiotic modes, medium and genre. To demonstrate and test this approach, the study performs a systematic, longitudinal analysis of a corpus of multimodal documents within a single genre: an extensively annotated corpus of tourist brochures produced between 1967-2008. The book provides multimodal discourse analysts with methodological tools to draw empirically-based conclusions about multimodal documents, and will be a valuable resource for researchers planning to develop and study multimodal corpora.

This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today’s increasingly complex combinations of multimodal artefacts and performances of all kinds.

This book draws on visual data, ranging from advertisements to postage stamps to digital personal photography, to offer a complex interpretation of the different social functions realised by these texts as semiotic artefacts. Framed within the media environment of the city of Hong Kong, the study demonstrates the importance of social context to meaning making and social semiotic multimodal analysis. This book will be of interest to readers in the arts, humanities and social sciences, particularly within the fields of semiotics, visual studies, design studies, media and cultural studies, anthropology and sociology.

This book is a first attempt to map the broad context of performance studies from a multimodal perspective. It collects original research on traditional performing arts (theatre, dance, opera), live (durational performance) and mediated/recorded performances (films, television shows), as well as performative discursive practices on social media by adopting several theories and methodologies all dealing with the notion of multimodality. As a mostly dynamic and also interactive environment for various text types and genres, the context of performance studies provides many opportunities to produce meaning verbally and non-verbally. All chapters in this book develop frameworks for the analysis of performance-related events and activities and explore empirical case studies in a range of different ages and cultures. A further focus lies on the communicative strategies deployed by different communities of practice, taking into account processes of production, distribution, and consumption of such texts in diverse spatial and temporal contexts.