A Multimodal Language Faculty -

. .

Résumé :
Language has traditionally been held as an amodal system that flows into different forms like speech, writing, or signing; however, communication is multimodal by nature. We pair speech with gestures, use emoji with text, and combine writing with drawings and images in places from doodles to comics to advertising. Yet, the linguistic and cognitive theories maintaining the traditional amodal notion of language cannot account for the richness of this multimodal communication. What is needed is a new, multimodal paradigm of language. This book presents a model of a multimodal language faculty which heralds a re-organization of the structures of language and their guiding assumptions. It shows that the primary human expressive behaviors - speaking, signing, drawing-may seem distinct, but actually decompose into similar cognitive building blocks, which coalesce into a multifaceted multimodal communicative system. The result is an account of human cognition where all communication - whether speech, gesture, graphics, and their multimodal interactions - arises as emergent activation states out of a singular cognitive architecture. The architecture put forward provides a 'grand unified theory' of language and communication, accounting for the insights of traditional linguistic formalizations, conceptual semantics, gesture studies, Peircean semiotics, visual language theory, and both unimodal and multimodal lexicons, all integrated into a single model. This overall approach directly confronts the traditional notions of language that have stood for at least two centuries with a new paradigm of acknowledging the multimodal nature of human communication, forcing us to reimagine what language is and how it works.

Biographie:
Neil Cohn is Associate Professor of Cognition and Communication at Tilburg University, Netherlands. Cohn's books include The Visual Language of Comics (Bloomsbury, 2013) and the 2021 Eisner-nominated Who Understands Comics? (Bloomsbury, 2020), which establish the linguistic and cognitive study of graphic communication. Joost Schilperoord is Assistant Professor of Cognition and Communication at Tilburg University, the Netherlands. He is best known for his research on language, graphics, multimodality, and cognition.

Sommaire:
List of Figures List of Tables Part I: Foundations 1. Reimagining Language Part II. Modalities 2. What is a Modality? 3. Interfacing Between Modalities Part III: Meaning 4. Conceptual Structures for Multiple Modalities 5. Multimodal Semantic Interactions Part IV: Grammar 6. The Complexity Hierarchy 7. Interactions Between Combinatorial Schemas Part V: Multimodality 8. Unimodal Expressions in a Multimodal Model 9. Independent Multimodal Interactions 10. Substitutive Multimodal Interactions Part VI: A Multimodal Language Faculty 11. Consequences of a Multimodal Language Faculty 12. Evolution of a Multimodal Language Faculty 13. Towards a Multimodal Linguistics References Index