Multimodality, ethnography and education in south america


Narratology in the analysis of multimodal legitimation: an introduction



Yüklə 6,97 Mb.
səhifə5/16
tarix15.08.2018
ölçüsü6,97 Mb.
#62523
1   2   3   4   5   6   7   8   9   ...   16

Narratology in the analysis of multimodal legitimation: an introduction

Dimitrios Chaidas


The University of Edinburgh
dimitris_chaidas@yahoo.gr

Previous studies on legitimation, multimodality and political discourse by researchers, such as Van Leeuwen, Van Dijk and Mackay, have suggested different but supplementary methods of legitimation analysis by providing a number of analytical frameworks. Multimodal legitimation research, however, seems to be in need of a better conflation of the theoretical backgrounds of disciplines, such as narratology. This article focuses on the multimodal discourse of three political advertisements of the political party New Democracy, filmed for the needs of the Greek legislative election of January 2015. What is investigated is the multimodal means by which New Democracy’s president, and Prime Minister at the time, Antonis Samaras attempted to legitimise his candidacy. In the present article, I employ the six- layer framework proposed by Mackay (2015) for multimodal legitimation analyses and I argue that multimodal legitimation research can benefit and get enhanced from the use of narratology and its analytical categories, such as perspective.



References

Mackay, R. R. (2015). ‘Multimodal legitimation: Selling Scottish independence’, Discourse & Society 26(3): 323–348.



Using Multimodality in Science Learning: A Fijian Learning Space Perspective

Deepa Dewali Chand, John Kenny, Sharon Fraser


College of Arts, Law and Education (CALE), University of Tasmania
Deepa.chand@utas.edu.au

With changing Science Curricula, there is always a demand for context appropriate pedagogical approaches to the learning of science. This study aimed to explore the effectiveness of a multimodal approach used by students combined with the use of Talanoa. Talanoa is an informal conversation commonly used and culturally appropriate in a Pacific context. A total of 29 students ages 9-10 and their teacher participated in the study which collected data via learning space observation focusing on student interactions, activities and use of multimodality in their lessons. The findings indicated that doing drawings in science and Talanoa can encourage students to engage actively with the task. Additionally, use of culturally appropriate semiotic modes such as drawing, gestures, emotions and nonverbal communication cues such as the silence can enhance students’ ability to apply scientific learning to the local context and communicate their thoughts effectively. The overall findings highlighted the use of culturally appropriate semiotic modes and Talanoa can help to integrate diverse cultural experiences to challenge and extend students’ thinking.



Keywords: Multimodal, Talanoa, semiotic modes, science, culturally-appropriate

Investigating Note-taking in Consecutive Interpreting from A Visual Communication Perspective

Dr. Li-Wen Chang


The Chinese University of Hong Kong, Shenzhen
changliwen@cuhk.edu.cn

As an indispensable tool in supporting interpreters’ memory during consecutive interpreting, interpreters’ notes play a key role as a mediator between source language (SL) and target language (TL). The aim of this study is to explore how interpreters retrieve information in the transition between note-reading and interpreting. This study argues that there are two languages (SL&TL) embedded in the texts, with the interpreters’ notes acting as a third visual language with their own logic and patterning that needs interpreting. Given the importance of the interaction between written and visual modes in the note-reading process, this study draws on visual grammar (Kress and van Leeuwen 2006) to analyse interpreter’s notes with a view to gaining a better understanding of how the linguistic and visual semiotic resources are deployed in the process of note-taking. In this study, interpreters’ notes are viewed as multimodal texts and will be approached from a social semiotic perspective to investigate how interpreters deal with the multimodal nature of notes. The analysis focuses on (i) the function of vectors and geometrical shapes, and (ii) the spatial disposition of signs, including layout and salience, found in the dataset. Initial results suggest that structures of interpreters’ notes present meaning like linguistic structures do, even though the visual language is not as transparent and universally understood.



Keywords: Consecutive interpreting, note-reading, social semiotics, multimodal

Interpersonal representations of Syrian refugees in images from the UK national print press: A short-term diachronic perspective

Dr Ben Clarke


University of Gothenburg
ben.clarke@gu.se

Kress & van Leeuwen (2006: 140-143) proposed several systems of functional contrasts conveying interpersonal meanings in still-image: (i) relation between the represented participant(s) and the interactive participant along the horizontal axis (downwards [power-over], straight on [power-equal], upwards [power-under]); (ii) distance of represented participants in the image-frame (close-up [intimate], mid-shot, long-shot [impersonal]); and (iii) engagement (or not) with the represented participant(s) by the interactive participant (frontal angle [involved], oblique angle [detached]). These and others of Kress & van Leeuwen’s (2006) descriptive frameworks have been much critiqued, particularly for drawing some analogies to linguistic categories (e.g. Machin, 2007: ch. 8). Still, these descriptions have been found to be useful in research on still-image, particularly work in the critical tradition concerned with ideology and where informed by big data.

Here, a similar enterprise is undertaken: the representation of Syrian refugees during the recent Syrian refugee crisis is explored in terms of image depictions of Syrian refugees for the aforementioned interpersonal systems of Kress & van Leewuen (2006). The research is particularly concerned with short-term diachrony (Aarts et al., 2013), and therefore changes across time in patterns relative to interpersonal still-image systems. Using a dataset of thousands of images drawn from eight national newspapers in the United Kingdom, diachronic changes in patterns in interpersonal representations of Syrian refugees are found, in important part, to result from: (a) the occurrence of other and related prominent news stories (e.g. the death of Ala Kurdi; the UK’s referendum on EU membership); and (b) the political alignment of the newspaper. This said, more nuanced trends across time relative to individual interpersonal systems will also be discussed.

References

Arts, B., Close, J., Leech, G. & Wallis, S. (2013) ‘Introduction’. In Arts, B., Close, J., Leech, G. & Wallis, S. (eds.) The Verb Phrase in English: Investigating Recent Language Change with Corpora. Cambridge: Cambridge University Press, 1-13.

Kress, G. & van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design. 2nd edition. London: Routledge.

Machin, D. (2007). Introduction to Multimodal Analysis. London: Bloomsbury.




Playing the Archive: Multimodal Perspectives on Children’s Play

Kate Cowan


UCL Institute of Education
k.cowan@ucl.ac.uk

John Potter


UCL Institute of Education
j.potter@ucl.ac.uk

In their studies of children’s play throughout the 1950s-1990s, folklorists Iona and Peter Opie aimed to capture the ‘kaleidoscopic vitality’ of UK playgrounds through extensive observations and surveys of play (Opie, 1993). Recognising the ephemeral and multimodal nature of children’s play continues to be an issue of particular importance as play changes in light of digital media and technologies. Young children’s contemporary playworlds are often a complex interweaving of physical and digital dimensions, with the border areas between ‘real’ and ‘virtual’ becoming increasingly blurred. The growing prevalence of virtual reality, augmented reality and mixed reality creates further opportunities for hybrid spaces where play shifts across boundaries of space and time in new ways.

A multimodal social semiotic perspective offers insights into the affordances of modes, both physical and digital, and new possibilities for playful meaning-making. ‘Playing the Archive’ (2017-2019) is an interdisciplinary research project funded by the EPSRC exploring archives, technologies and spaces for play. This paper will share interim analysis of the project’s fieldwork conducted in the playground of an inner-city London primary school for children aged 5 to 11. Using a range of ethnographic, participatory methods, we will consider how contemporary play is experienced multimodally in playgrounds, and will speculate on key elements of continuity and change in children’s play since the 1950s and 1960s. This process will explore children’s own multimodal and curatorial practices in both the data collection and in negotiation with the archive itself. As the project unfolds, these insights will be used to shape a virtual play environment based on the Opie archive and the development of experimental ‘smart’ playgrounds in two UK cities.

References

Opie, I. (1993). The People in the Playground. Oxford: Oxford University Press.



Multimodal Digital Story-creation, Affordances and Students´ Agency

Helene Dahlström, PhD in Education


Mid Sweden University
helene.dahlstrom@miun.se

The availability of multiple ways of communicating and expressing ourselves has increased due to the digitization of society. These different ways of communicating also mean that we to a greater extent, use different modes as communication resources than we did before. Digital resources provide access to different modes within the same device. Given these changes in communicating, there is a need to explore how students in a school context are using these changing conditions for text creation. This study will explore how Swedish 11-12-year-old students use different modes while creating digital stories. The study will have a multimodal approach, which generally refers to the fact that representation, communication and interaction are more than verbal and treat the choice of modes as significant (Archer & Breuer, 2015). The existence of choice for students in text creation can be seen as students´ agency through choices (Bezemer & Kress, 2016). The unique affordances of modes give the sign maker the opportunity to choose which modes are best suited to conveying a special message, based on their interests and needs. This ongoing study will explore how students´ agency can be understood in relation to the affordances that the different modes offer when students create digital stories. The data collection consists of video recording of 26 students creating digital stories, and of the multimodal texts created by these students. Questions to be answered are: How do students use different modes in digital story-telling? What affordances do different modes offer and how can these be understood in relation to students´ agency?



References

Archer, A. & Breuer, E. (2015). Methodological and pedagogical approaches to multimodality in writing. I G. Rijlaarsdam (Series Ed.) & A. Archer & E. Breuer (Vol. Eds.), Studies in Writing: Vol. 30, Multimodality in writing (pp. 1–16). Leiden: Brill.

Bezemer, J. & Kress, G. (2016). Multimodality, learning and communication, A social semiotic frame. Oxon and New York: Routledge

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London: Routledge.




Playing games with multimodalities: the usage of typography in the construction of digital games characters

Cristiana Barbosa Nunes da Silva, graduate student


Universidade Federal do Piauí / Instituto Federal de Educação, Ciência e Tecnologia do Piauí – Brazil
cristiana@ifpi.edu.br

This Communication aims to present a pilot study investigating the construction of characters in digital games through the recognition of representative and interactive meanings in light of the Grammar of Visual Design by Kress and Van Leeuwen (2006). The research is exploratory and descriptive, utilizing qualitative criteria for the collection and screen shots analysis of the digital game Plants x Zombies. As a focus for this presentation, we will discuss the preliminary results of the analysis of five screen shots of the game Plants vs. Zombies, in which the typography is used as a semiotic way that favors the identification of the characters such as makers of the verbal processes displayed in these screen shots and the interaction between characters and players. In an educational scenario facing multiliteracies and multimodality, such results may indicate implications for English language teaching and learning mediated by digital games.



Keywords: Multimodality, multiliteracies, digital games

Designs for learning as political and social achievements: Recognising the use of semiotic practices in children’s visual representations:

Sophia Diamantopoulou, Senior Teaching Fellow


UCL Institute of Education

Drawing visual data from three museum education projects (Diamantopoulou, 2007; Diamantopoulou & Christidou, forthcoming; Kress & Selander, 2012) this paper aims to make a case for assigning prominence and recognition to the ‘rhetorical’ (political) aspect of communication outlined in Kress’s theory. This is in order to best account for learners’ interaction with the institutional context within which they operate, as well as broaden our understanding of designs for learning.

Most of the visual representations of children arise in institutional contexts. However, limited attention has been paid to the deeply political act of negotiating meaning making while operating within such contexts (Mavers, 2009). The paper presents instances of analysis of three visual representations prompted and framed by the institutional discourses of two educational programmes and one museum exhibition respectively. It argues for the need to use the same set of multimodal social semiotic tools to account equally for both the rhetorical and design aspect of multimodal representations, moving multimodal analysis away from descriptive accounts of how modes are coming together. It also foregrounds the need to account for those ‘semiotic practices’ that are involved in the negotiation of the interpersonal aspect of communication between the meaning maker and the institution.

Through the concept of ‘semiotic practices’ – suggesting the diverse responses to institutional practices framing communication – this paper explains the ways in which meaning makers exercise agency accounting for their varying positions to the institutional authority. Recovering the visual traces of ‘semiotic practices’ in these drawings, the paper discusses the importance of examining children’s designs for learning within the context that prompted and framed them (i.e. pedagogies, curricula, exhibition designs and educational activities).

The theoretical underpinnings and prompts for this exploration have been: (i) the concept of learning as a social achievement, as it arises in Kress’s work and the scholarly work of the New Learning Group (Bezemer et al, 2012), (ii) Kress’s concept of the dual aspect of communication as rhetoric and design (Kress, 2010) (iii) and the author’s PhD thesis on the semiotic practices of learners (Diamantopoulou, in prep).

References

Bezemer, J., Diamantopoulou, S., Jewitt, C., Kress, G., & Mavers, D. (2012) Using a Social Semiotic Approach to Multimodality: Researching Learning in Schools, Museums and Hospitals, National Centre for Research Methods (NCRM) Working Paper 01/12  http://eprints.ncrm.ac.uk/2258/4/NCRM_working_paper_0112.pdf

Diamantopoulou, S. (in prep) Designs for learning in museums and archaeological sites: A multimodal social semiotic approach to environments and signs of learning in the Museum of London and the Agora, Athens , PhD thesis, University College of London.

Diamantopoulou, S and Christidou, D. (forthcoming) Children’s eye view of an archaeological site: Multimodal social semiotic perspectives in the use of drawings. In Whitehead, G.D., Petrov, J. and Saunderson, H. ‘A child’s view of museums: Remembering Elee Kirk’. Museums and Society Special Issue.

Mavers, D. (2009) Student text-making as semiotic work, Journal of Early Childhood Literacy, 9 (2), 141-155.

Kress, G. and Selander, S. (2012) Introduction to the special issue on museum identities, exhibition designs and visitors’ meaning-making, Designs for Learning, 5 (1-2) 6-10.

Kress, G. (2010) Multimodality: A Social Semiotic Approach to Contemporary Communication, London: Routledge.

Diamantopoulou (2007) Ideas Factory Action Research Project Report: A multimodal approach to the ideas factory project’s impact on children’s literacy. London: Tate Britain, Available at: http://www.tate.org.uk/download/file/fid/46403, Accessed on 6 September 2017.



Semiotic artefacts in multimodal events: picture books and other props in library storytime sessions

Emilia Djonov


Macquarie University

Storytime sessions at public libraries are complex multimodal events that engage young children and their families in the shared reading of picture books, singing and dancing, reciting nursery rhymes, storytelling, and craft. Picture books and other objects (puppets, toys and musical instruments) are integral to many of these activities.

Studies of multimodality have examined both objects such as picture books or learning spaces and events such as classroom interactions. Multimodal studies of objects tend to treat these as ‘texts’ and ignore the ways in which an object may be integrated in specific events. Multimodal analyses of events, on the other hand, tend to account for this integration but pay surface attention to the integrated objects themselves.

In this paper, I view the picture books and other objects integrated in library storytime as ‘semiotic artefacts’ (Kress & Van Leeuwen, 2001; Van Leeuwen, 2005). A ‘semiotic artefact’ is a material semiotic resource that embodies selections from various modes (e.g. spoken language, visual design, texture) and media (e.g. aural, visual, tactile), and can itself be deployed for meaning-making alongside other semiotic resources. I argue that this perspective provides a promising foundation for studying and evaluating the interaction between the meaning-making potential of objects and its instantiation in semiotic practices such as storytime. To support this argument, I draw on the analysis of examples selected from a data set of 57 storytime sessions recorded at Australian public libraries, and extracts from interviews with their presenters, and consider the potential of picture books and other objects to support the key goals of storytime – to foster language and literacy learning, love of reading, and a sense of belonging to a community of readers.



References

Kress, G., & Van Leeuwen, T. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.

Van Leeuwen, T. (2005). Introducing Social Semiotics. London/New York: Routledge.

From static texts to personalized dynamic advertising: What has changed and how does it influence our understanding of the notion of text?

Martina Dodl, M.A.


English Linguistics, Universität Augsburg
martina.dodl@outlook.de

There is no denying that in linguistics, the notion of ‘text’ has undergone a major change within the 20th century. Visualizations and audio have gained influence not only in a cultural sense by means of digitalization and mass media but also in any scientific field having dealt with “plain texts” before. The iconic turn and multimodality as the “contemporary semiotic practice” (Kress & van Leeuwen) are just the outcome of what has been present in texts ever since.

A diachronic analysis of Coca Cola Advertising since the very beginnings in the 1880s until the 2000s shows how images have always been part of our communication system and how they developed over time changing the interplay between all modes in the ads enormously. Opening the borders for interdisciplinary research, the examination is based on linguistic methods chosen from semantics, pragmatics and syntax (e.g. Kress’ and van Leeuwen’s communicative function of colour or Stöckl’s sub-functions of persuasion), and complemented by cultural philosophical knowledge about images (e.g. Gombrich’s opposition of nature and convention). Yet in times of Smart Data and new technology, there still is more than meets the eye. Therefore, I would like to launch the discussion about the notion of text and semiotic resources all over with regards to modern, dynamic and data based text composing principles.

My study does not only challenge linguistic analyzing methods within the context of multimodal artefacts, it also reveals the power of the unsaid in terms of visuals as sublime carriers of core values and in terms of collected data for personalized, data driven texts.



Seeing knowledge through field: Multimodal meaning in science

Yaegan Doran


LCT Centre for Knowledge-Building and Department of Linguistics, The University of Sydney

The sciences are well known for their highly multimodal discourse. But characterising how such a range of semiotic resources build science’s highly technical knowledge has remained elusive. From a Systemic Functional perspective, scientific disciplines are largely devoted to field. They build large and elaborated ‘content’ meanings that enable description, explanation and prediction of physical phenomena. But despite their orientation to field, discussions of science to this point have largely focused on the grammatical, discourse semantic and genre-based features of science. In contrast, this paper introduces a recently renovated model of field in Systemic Functional Semiotics (Martin and Doran in press). This model has been explicitly developed to account for the range of field-specific meanings made across the highly multimodal discourse of science, including language, image, mathematics, chemical and nuclear equations, animations, gesture etc. (Doran in press). It suggests multimodal discourse may present meanings statically through relations between items, or dynamically through the unfolding of activities. Additionally, it models the gradable properties often ordered into large arrays, quantitatively gauged and/or positioned within a spatio-temporal frame. Through this model of field, we can build a semiotic understanding of content knowledge that is integrated with the elaborate Systemic Functional descriptions available for numerous semiotic resources, and offer a unifying description for considering why various semiotic resources are used for certain disciplines, with various genres and to construe various meanings.



References

Doran, Y. J. (in press) ‘Multimodal Knowledge: The use of language, mathematics and images in physics’, in K. Maton, J. R. Martin and Y. J. Doran (eds) Studying Science: Knowledge, Language, Pedagogy. London: Routledge.

Martin, J. R. and Doran, Y. J. (in press) ‘Seeing knowledge through field: Understanding scientific explanations’ in K. Maton, J. R. Martin and Y. J. Doran (eds) Studying Science: Knowledge, Language, Pedagogy. London: Routledge.

Complexity and context-dependence in multimodal texts: Semantic density and semantic gravity for images and language

Yaegan Doran, Karl Maton


LCT Centre for Knowledge-Building, The University of Sydney

Abstract-concrete, symbolic-iconic, simple-complex… such terms are often used to characterise multimodal discourse and the knowledge it organises. This variation is crucial to the unfolding of texts and the development of social fields, however presented as discrete categories they often obscure more than they reveal. Moreover how these variables are realised across a range of different semiotic resources and how we can ‘see’ them in multimodal texts remains relatively underexplored. To grasp this issue, this talk will introduce the dimension of Semantics from Legitimation Code Theory (LCT) (Maton 2014). In particular it will focus on the concepts of semantic density (SD) and semantic gravity (SG). Semantic density considers complexity of meaning; stronger semantic density (SD+) indicates higher complexity, whereas weaker semantic density (SD–) indicates less complexity. Semantic gravity conceptualises context-dependence of meaning, where stronger semantic gravity (SG+) indicates more context-dependence, and weaker semantic gravity (SG–) indicates less context-dependence. Importantly, rather than presenting simple binaries, semantic density and semantic gravity conceptualise complexity and context-dependence as continuous gradations; hence stronger and weaker semantic density (more and less complexity), and stronger and weaker semantic gravity (more and less context-dependence). These concepts are becoming widely enacted within linguistics, education and a range of other fields as a means of identifying both nuanced text variation and highly-valued practices. But as yet, much of the focus has been on monomodal linguistic texts. In contrast, this paper will consider shifts in SD and SG across language and image, using case studies from science teaching. In doing so, it will introduce a new ‘translation device’ for enacting semantic density and semantic gravity in images to complement that for English discourse introduce in Martin and Doran (2017 a, b). This tool will enable a more fine-grained understanding of the shifts in abstraction and complexity that occur through images, and will offer a principled basis for considering the role of language and image in building knowledge.



Yüklə 6,97 Mb.

Dostları ilə paylaş:
1   2   3   4   5   6   7   8   9   ...   16




Verilənlər bazası müəlliflik hüququ ilə müdafiə olunur ©genderi.org 2024
rəhbərliyinə müraciət

    Ana səhifə