Multimodality, ethnography and education in south america

Keywords: Multimodal discourse analysis, front page news reports, similarities and differences
A Study on Appraisal Resources in Argumentative Essays by non-English Major

Wang Hongli Chenying & Yan Kailun

School of Foreign Studies, Xi’an Jiaotong University

[e-mail]

Argumentative writing requires authors to analyze, discuss, and resolve controversies in a way that is clear, convincing, and considerate of diverse points of view. However, it is difficult for non-native English speaking students to construct interpersonal meaning in argumentative writing. Based on Appraisal theory, this study mainly investigates the employment of Appraisal resources in English argumentative essays written by Chinese non-English major postgraduates. To begin with, this study explores the overall distribution of Appraisal resources in argumentative essays and finds that Engagement resources, as the most frequently used ones, play a crucial role in achieving the rhetorical purpose of argumentative essays. The pattern of more Engagement than Attitude and Graduation could be regarded as the characteristic of the argumentative genre. Afterwards, this research compares and contrasts Appraisal resources across the high-rated and low-rated argumentative essays and explored how Appraisal resources co-articulate to achieve the rhetorical purpose of argumentative essays. The study finds that the two sub-corpora differ in the following aspects: the preference for Appraisal resources, the number of Appraisal resources, the realizations of Appraisal resources, and the interplay of Appraisal resources. The findings of this study can contribute to successful argumentative essay writing and offer some pedagogical implications.
Keywords: Appraisal theory; argumentative writing; non-English major MA students

A QUALITATIVE STUDY OF MULTIMODALITY IN MULTIMODAL LANGUAGE TEACHING AND LEARNING IN ENGLISH CLASSES OF CHINESE SCHOOLS

Jingjing Huang

University of Southampton

This research is about the qualitative study of multimodality based on multimodal language teaching and learning in English classes of Chinese schools. The author regards it as qualitative research based on the perspectives of educational research. The main aim of this investigation is to study the phenomena of the application of multimodality and visual semiotics in English classes in China to find out how different modes combine to work, what the processes of meaning made by modes are, and what they affect in teaching and learning processes. In terms of multimodal language teaching and learning, the author plans to take vocabulary teaching and learning as the main subject. According to the previous authoritative studies in the education field, the importance of multimodality attracts the attention of scholars. In China, the application of multimodal language teaching is popularly used in the classroom. The author will take fundamental concepts and theories of multimodality into vocabulary teaching and learning in English classes of Chinese schools.

Thus, the author plans to investigate and collect data from selected schools including a primary school, a secondary school and a high school. Both qualitative and quantitative methods will be used in the research data collection. But it mainly focuses on the qualitative methods including classroom observations, small empirical studies in the class, field notes or journals, and semi-structured interviews. Therefore, through this study, the purpose of this research is combining the application of multimodality with language education to investigate what kinds of modes are used, what meanings these modes make, how these modes work together, how about effects of teaching and learning in English classes of Chinese schools, and analyzing what the current situation of the application of multimodality in English classes nowadays in China, and its development in the future.

Keywords: Multimodality, modes, language education, meaning made, English vocabulary teaching and learning, English classes, Chinese schools, qualitative study, educational research

Contextualization of Compositional Modes: A multimodal study of traffic signs in China

Dai Hui, PhD, Guangdong University of Technology

Guangzhou, China, 510006
[e-mail]

This decade is hailed for a multiplication of multimodal investigation in languages, signs and discourses, yet little research zooms in on the semiotic modes of large-scale traffic signage in public space as a driving navigator (Yang, 2012; Dai, 2015). Meanwhile, in the traffic-engineering realm, traffic signs have been approached from a range of aspects including ergonomics (e.g. Park & Kim，2013; Ruta, Li & Liu, 2010; Jimenez et al., 2008), signing system (e.g. Shinar & Vogelzang, 2013; Liu, 2005) and readership propensities (e.g. Kirmizioglu & Tuydes-Yaman, 2012; Xie & Parker, 2002; Al-Madami & Al-Janahi, 2002). These aspects presuppose a level of decontextualized lab results than messages subject to reality and culture, neglecting their real-life visual effects as an ingredient of the meaningful whole.

This study explores the compositional features of traffic signs in Mainland China by adopting Geosemiotic System (Scollon & Scollon, 2003) and Visual Grammar Theory (Kress & van Leeuwen, 2006). Vector and line feed are analyzed to demonstrate the transporting visuality encountered by in-seat drivers and reading procedures possibly executed by them in the Chinese traffic context. The data have been collected in Guangzhou, a hub city and commercial center in China with colossal traffic volume and diverse sign users. It is found that multiformity in composition characterizes the overall Chinese traffic signing system where decontextualized information (e.g. morphological uniformity) is much preferred (MUTCD, 2009; RTSM, 2009).

This study suggests that while China with its high-end traffic signs makes inroads into a user-friendly linguistic landscape, compositional modes of signs may be contextualized and implemented willy-nilly by the traffic management. Specifically, in reducing proceeding length and enhancing semiotic legibility, multimodal uniformity of public signs shall be considered before people erect them in place.

References

Al-Madami, H. & Al-Janahi, A. R. (2002). Role of drivers’personal characteristics in understanding traffic sign symbols. Accident Analysis and Prevention, 34, 185-196.

Dai, H. (2015). A Geosemiotic Study of Communication Between Traffic Signs and Their Users in China. PhD dissertation. Guangdong University of Foreign Studies.
Jimenez P.G., Bascon, S. M., Moreno, H.G., Arroyo, S.L. & Ferreras, F.L. (2008). Traffic sign shape classification and localization based on the normalized FFT of the signature of blobs and 2D homographies. Signal Process, 88, 2943-2955.
Kirmizioglu, E., & Tuydes-Yaman, H. (2012). Comprehensibility of traffic signs among urban drivers in Turkey, Accident Analysis and Prevention 45,131-141.
Kress, G. & van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design (2^nd ed). London: Routledge.
Liu, Y. (2005). A simulated study on the effects of information volume on traffic signs, viewing strategies and sign familiarity upon driver’s visual search performance. International Journal of Industrial Ergonomics, 35, 1147-1158.
Park, J. G. & Kim, K. J. (2013). Design of a visual perception model with edge-adaptive Gabor filter and support vector machine for traffic signs detection. Expert Systems with Applications, 40 (9), 3679-3687.
Ruta, A., Li, Y., & Liu, X. (2010). Real-time traffic sign recognition from video by class-specific discriminative features. Pattern Recognition, 43, 416-430.
Scollon, R. & Scollon, S. W. (2003). Discourses in Place. London: Routledge.
Shinar, D. & Vogelzang, M. (2013). Comprehension of traffic signs with symbolic versus text displays. Transportation Research Part F, 18, 72-82.
Standardization Administration of China. (2009). Road Traffic Signs and Markings [RTSM]). Beijing: Standards Press of China.
US Department of Transportation, Federal Highway Administration. (2009). Manual on Uniform Traffic Control Devices [MUTCD]. Washington, D.C
Yang, X. (2012). Traffic Design. Beijing: China Communications Press.
Xie, C. & Parker, D. (2002). A social psychological approach to driving violations in two Chinese cities. Transportation Research Part F, 5, 293-308.


East Meets West? Identifying points of contestation when mediating understandings about Eastern picturebooks with children

Ngoc Tai Huynh, Angela Thomas, Vinh Thi To

University of Tasmania

Picturebooks draw on a reader’s knowledge about social and cultural meanings to contribute to the interpretive possibilities of the narrative in a variety of ways. They also rely on a reader’s background of understanding how images work to make meaning. However, in multicultural societies such as Australia, it is common for school children to engage with picturebooks from a multiplicity of cultures. As teachers mediate children’s understandings, one challenge they are faced with is understanding how best to help children read and interpret images from Eastern cultures. The Australian Curriculum (ACARA, 2016) draws heavily on Kress and van Leeuwen’s (2006) semiotic theory when teaching visual language. Yet, Kress and van Leeuwen clearly identify their framework as one devised from a Western perspective. This presentation takes as its starting point a study which examined the ways in which Vietnamese culture is represented in 3 children’s picturebooks. This study used an integrated framework developed from current western perspectives for visual analysis (Kalantzis & Cope, 2012; Kalantzis et al., 2016; Kress & van Leeuwen, 2006; Serafini, 2014). The process of analysing visual texts created by Eastern illustrators raised a number of critical issues about such Western frameworks. Our analyses illuminate the relationships among the narrative genre, picturebook images, and thematic interpretation of the texts, and how the latter can be problematic when the dominant Western “ways of looking” at images are the only ways teachers might mediate such texts with their students.

Keywords: Vietnamese culture, children’s picturebooks, semiotic theory, multimodality, non-western images

References

Australian Curriculum, Assessment and Reporting Authority (ACARA). (2016). The

Australian Curriculum: Literacy. Retrieved from https://www.australiancurriculum.edu.au/f-10-curriculum/general-capabilities/literacy/

Kalantzis, M., & Cope, B. (2012). Literacies: Port Melbourne, Vic. : Cambridge University Press, 2012.

Kalantzis, M., Cope, B., Chan, E., & Dalley-Trim, L. (2016). Literacies: Port Melbourne, VIC, Austalia : Cambridge University Press, [2016]. Second edition.

Kress, G., & van Leeuwen, T. (2006). Reading images : the grammar of visual design ( 2nd ed. ): London ; New York : Routledge, 2006.

Serafini, F. (2014). Reading the visual. [electronic resource]: an introduction to teaching multimodal literacy: New York : Teachers College Press, [2014].

TEACHERS AND LEARNERS CONSTRUCTING MEANING FOR NEW WORDS IN THE FOREIGN LANGUAGE CLASSROOM

Eva Ingerpuu-Rümmel

Institute of Estonian and General Linguistics, University of Tartu, Estonia
[e-mail]

Meaning construction for new words is one of the main activities in foreign language classes. There are still few studies on how meaning is constructed for new words with the help of modes (e.g. verbal expression, gestures, use of space and objects) in the multimodal communication of a classroom. The aim of the present study is to use analysis of communicative situations to identify the participation opportunities of teachers and learners in meaning construction for new words in the foreign language classroom. The present study is based on audiovisual material – two Estonian and two French classes (with the duration of 90 minutes each). In total, 110 communicative episodes were selected and transcribed for the analysis. The approach applied in this research is micro-level multimodal discourse analysis. The results show that the participants – the teacher and the learners – can be active or passive meaning constructors for a new word depending on how they use modes. Three models of communicative situations are created based on the results. The paper introduces the models by presenting some examples of communicative situations. The results of the study may help teachers to consider the possibilities of activation of participants in the classroom while planning their language classes.

References

Kress, Gunther 2015. “Semiotic work. Applied linguistics and a social semiotic account of multimodality”. AILA Review, 28, 49–71. John Benjamins Publishing Company.

Kress, Gunther, Theo van Leeuwen 2001. “Multimodal Discourse: The Modes and Media of Contemporary Communication”. London: Arnold.

Lazaraton, Anne 2004. “Gesture and speech in the vocabulary explanations of one ESL teacher: A microanalytic inquiry”. Language Learning: A Journal of Research in Language Studies, 54, 1, 79–117.

Gullberg, Marianne 2014. Gestures and second language acquisition. Body, language, communication: an international handbook on multimodality in human interaction. Müller, C., Cienki, A., Fricke, E., Ladewig, S. H., McNeill, D., Tessendorf, S. (eds.). Mouton de Gruyter, 2, 1868–1875.

Supporting students sense-making with images in History

Line Ingulfsen

Department of Teacher Education and School Research, University of Oslo, P.O. Box 1099 Blindern, N-0317 Oslo, Norway
[e-mail]

Images play a central role in the learning and teaching practices of History. Commonly found in textbooks, lectures and in students’ own texts and presentations, pictures are used to represent historical events, people, objects and settings. As historical sources often with a rich potential for multiple interpretations, images are also central in introducing students to interpretive practices such as historical analysis and source criticism (Wineburg, 1991). Moreover, with the move from page to screen, the availability of digital images increases, providing teachers with a wide range of visual resources to include in lectures and task design (Gillen et al., 2008; Twiner et al., 2013). Yet, despite the many uses of images in the teaching and learning practices of History, few studies have analytically scrutinized how they are attended to and used in classroom interactions.

Based on a sociocultural approach to multimodality (Ivarsson, et al., 2009; Wertsch, 1991), the current study will address this issue by directing analytical attention towards classroom interactions where images are the central artefact. Our aim is to provide insight into how one particular type of image - propaganda drawings – become structuring resources in interactions and how the teacher facilitates and supports students in their sense-making. The empirical basis of the study is a case study in which upper secondary school students worked on a project about the Interwar Period. Our primary data is transcribed video recordings of classroom interaction, and our analytical procedure is interaction analysis, implying the sequential analysis of interaction between interlocutors (Jordan & Henderson, 1995). Based on our (ongoing) analyses we will conceptualize students’ sense-making with images as a process of transduction (Bezemer & Kress, 2008). Further, we will discuss the role of the teacher in orchestrating multimodal connection making.

References

Bezemer, J., & Kress, G. (2008). Writing in Multimodal Texts - A Social Semiotic Account of Designs for Learning. Written Communication, 25(2), 166-195.

Gillen, J., Littleton, K., Twiner, A., Staarman, J. K., & Mercer, N. (2008). Using the interactive whiteboard to resource continuity and support multimodal teaching in a primary science classroom. Journal of computer assisted learning, 24(4), 348-358.

Ivarsson, J., Linderoth, J., & Säljö, R. (2009). Representations in practices: A socio-cultural approach to multimodality in reasoning. In C. Jewitt (Ed.), The Routledge handbook of multimodal analysis (pp. 201-212). London: Routledge.

Jordan, B., & Henderson, A. (1995). Interaction Analysis: Foundations and Practice. The Journal of the Learning Sciences, 4(1), 39-103.

Twiner, A., Littleton, K., Coffin, C., & Whitelock, D. (2014). Meaning making as an interactional accomplishment: A temporal analysis of intentionality and improvisation in classroom dialogue. International Journal of Educational Research, 63, 94-106.

Wertsch, J. V. (1991). Voices of the Mind: A Sociocultural Approach to Mediated Action. Cambridge, MA: Harvard University Press.

Wineburg, S. S. (1991). On the reading of historical texts: Notes on the breach between school and academy. American Educational Research Journal, 28(3), 495-519.

Representations of migration, borders and memories in exhibitions: a multimodal text analysis

Eva Insulander

Stockholm University, Sweden
[e-mail]

Museums are transforming the way they engage with their audiences and often adopt a critical approach towards existing collections and exhibition making. Having the potential to provide contesting perspectives on current topics, many contemporary exhibitions address migration stories (MacDonald, 2003). As exhibitions often combine several modes and media, they may be demanding for visitors to interpret. Building on a social semiotic approach to multimodality (Bezemer & Kress, 2016; Kress, 2010) and a model for analyzing multimodal texts (Danielsson & Selander, 2016), this paper explores how the conceptions of migration, borders and memory represent themselves multimodally in exhibitions-as-texts. I look at how different semiotic resources are used and discuss the potentials and constraints for making meaning – the meaning potential. Three Swedish exhibitions from 2015 were selected for systematic analysis and comparison, namely: The Legacy, Narratives About Escape; Borders; and Migrants and Refugees. These had in common a goal to discuss migration and to get visitors to critically reflect on their own perceptions and attitudes. The Legacy provided more neutral, even playful information about the migration of a particular group (cf. Lanz, 2016). Negative as well as positive minority identities were displayed primarily in metaphors. In Narratives About Escape, the ambition of breaking with prejudice against immigrants was visible at all levels of the text. Coherence was created between the different parts and modes, which meant that interpretation was potentially straightforward. Borders, Migrants and Refugees was more complex by comparison and challenged the visitor through multiple messages at all levels of the text, also through several numbers and complex maps that challenged social constructions (cf. Poehls, 2011; Sutherland, 2014). An abundance of abstract information was given, mixed with suggested solutions that demonstrated clear standpoints about the positive aspects of migration, but without avoiding conflicts. The paper demonstrates how exhibitions often tell several stories at the same time and provides insights into how values and political ideology are represented.

References

Bezemer, J. & Kress, G. (2016). Multimodality, learning and communication. A social semiotic frame. London & New York: Routledge.

Danielsson, K. & Selander, S. (2016). Reading multimodal texts for learning – a model for cultivating multimodal literacy. Designs for learning, 8(1), 25-36.

Kress, G. (2010). Multimodality. A social semiotic approach to contemporary education. London: Routledge.

Lanz, F. (2016). Staging migration in museums. A reflection on exhibition design practices for the representation of migration in European contemporary museums. Museum & Society, 14(1), 178-192.

Macdonald, S. (2003). Museums, national, postnational and transcultural identities. Museum and society. 1(1), 1-16.

Poels, K. (2011). Europe, Blurred: Migration, Margins and the Museum. Culture unbound. Journal of Current Cultural Research. 3, 337-353.

Sutherland, C. (2014). Leaving and longing: Migration museums as nation-building sites. Museum & Society, 12(1), 118-131.

Offense and Defense in German Populist Rhetoric: A Multimodal Case Study of AfD (Alternative für Deutschland)

Sylvia Jaki

Universität Hildesheim
[e-mail]

Tom De Smedt

Computational Linguistics Research Group, University of Antwerp
[e-mail]

The German federal elections of 2017 experienced a considerable rise in right-wing populism, closely linked to the political party AfD, which achieved a striking success with 12.6% of the votes. Two media that played an important role in the pre-election communication campaign are Twitter and political TV talk shows. In this case study, we focus on quantitative and qualitative methods to expose characteristics of recent German populist rhetoric on Twitter and in talk shows.

QUALITATIVE ANALYSIS

One of the most popular political talk shows on German public TV, “Hart aber fair”, demonstrates an oscillation of AfD politicians between defense (when confronted with former AfD hate speech¹) and offense (normalizing populist ideas). We analyse their performances as a mediated, face-to-face and hence multimodal interaction (Bateman et al., 2017: 239f), and we will show 1) whether their communication style differs (multimodally) from that of other politicians, and 2) how their performance differs between defense and offense.

QUANTITATIVE ANALYSIS

Social media platforms such as Twitter also play an important role in today’s political communication (Diekmannshenke, 2016). The pre-election period was often marked by aggressive discussion. We have collected a rich corpus of over 125,000 German tweets about politicians, by politicians, by their supporters, and by far-right influencers that spread hate speech. Using statistical AI techniques from Natural Language Processing (NLP) and Machine Learning (ML) we expose the differences in rhetoric.

Since social media are increasingly multimodal (Marx, 2017: 144ff), with users expressing themselves with emoticons, by sharing photos or videos, and so on, we also qualitatively explain what, why and how emoticons, memes and symbolism contribute to online German populist rhetoric.

References

¹ http://amp.dw.com/en/afd-politician-censored-under-new-german-hate-speech-law-for-anti-muslim-tweet/a-41992679

Bateman, John, Wildfeuer, Janina, Hiippala, Tuomo (2017): Multimodality. Foundations, Research and Analysis. A Problem-Oriented Introduction. Berlin/New York: de Gruyter.

Diekmannshenke, Hajo (2016): Massenmedien als Handlungsfeld III: digitale Medien. In: Martin Wengeler, Alexander Ziem, Kersten S. Roth (eds.), Handbuch Sprache in Politik und Gesellschaft. Berlin/New York: de Gruyter, 354-370.

Marx, Konstanze (2017): Diskursphänomen Cybermobbing. Ein internetlinguistischer Zugang zu [digitaler] Gewalt. Berlin/New York: de Gruyter.

Picture books and tourism

Ingrid K. Jakobsen, PhD student

UiT The Arctic University of Norway
[e-mail]

The Norwegian picture book Bim: Northern Light Dream (Bim og drømmen om nordlyset) was first published in 2013, and later came as a free app. It tells the story about the bear cub Bim, who decides to stay awake for the Arctic Night. All bears hibernate, but Bim wants to see the northern lights. The book is a co-production by sisters Stine Haugseth and Trude Haugseth Moe from Tromsø, Norway. Photos by Francesco Galbiati form the background whereupon print and drawings have been included.

This book is clearly targeted at the tourist market. It comes in German, Italian, English and Norwegian, and is sold in museums and tourist shops, in addition to local bookstores, and makes a nice souvenir or gift for tourists to bring back home to relive the northern lights experience. At the same time, it is a children’s book built upon known narrative patterns such as the home-away-home motif, and the helper who enables Bim to fulfil his dream. Moreover, there is a certain air of myth or fable when the other animals tell Bim how they perceive the northern lights.

I propose to use social semiotic multimodal analysis to explore selected spreads from Bim, asking first how modes separately and in interaction create a novel that straddles (adult) tourist interest and children’s interest. Secondly, I ask what the medium of the app changes in comparison to the book, in terms of choices in design afforded by technology. I will argue that the app is more clearly aimed at children than the book. I am considering interviewing one of the co-authors to get a glimpse behind the page and the screen. I could ask about the process of making the book, the choices made in assembling modes, and in developing the story through both photos, drawings, printed text and finally in the app with its animations, sounds, and the narrator’s voice.