Multimodality, ethnography and education in south america

PAPER ABSTRACTS

Vernacular semiotics in public places: The case of Leeds Kirkgate Market

Elisabetta Adami

University of Leeds, UK
[e-mail]

The visual landscapes of our public spaces signify through architectural design and all various signage and objects, as well as their colour, layout and materiality. Resources such as writing, color, typography, image, objects and layout have never been as significant and available to everyone as in today’s societies, yet their use is generally highly regulated in public spaces. Urban visual landscapes are increasingly shaped by aesthetic choices made by design professionals, both as institutional public signage and urban décor, and as corporate-led image-branding design of chain shops populating city centres globally. This limits the kinds of social agents who shape public spaces visually and influences our tastes more generally, through hegemonic patterned uses of resources such as font, writing, image, objects and layout.

A few places still exist that are still relatively semiotically unregulated, though. The paper presents a social semiotic ethnographic approach to the investigation of vernacular sign-making in place, by examining the visual landscape of Kirkate Market in Leeds (UK), as an example of a semiotically diverse unregulated place. By analysing traders’ use of resources in their stalls, it shows the disruption of mainstream aesthetic tenets emerging from vernacular sign-making practices, and the distinctive socio-cultural value of these practices in semiotically unregulated places.

Findings show that vernacular sign-making can be driven by different principles than corporate-led professional design aesthetics, revealing not only different tastes but also different conceptions of public space, even in a business-oriented place like a market. The conclusions derive implications for the investigation of vernacular semiotics in public space.

Visual Hate Speech and Popular Culture: The Case of Islamophobic Memes

Carmen Aguilera-Carnerero

Universidad de Granada
[e-mail]

Muslims have been the target of discriminatory attitudes – especially after 9/11- by certain sectors of society leading to what is known as ‘Islamophobia’. This feeling of exclusion has its most common manifestation in hate speech. Hate speech itself is a concept that has been approached from very different perspectives and despite the fact that no single universal definition of the notion exists, it could roughly be defined as all forms of expression that spread, incite, promote or justify forms of hatred towards individuals based on their membership in a group. In the technological society we live in, the most common channel of the expression of Islamophobia is, without doubt, social media giving rise to a new 2.0 variety― Cyber Islamophobia (Larsson 2007).

One of the most subtle ways to show hate speech is through humour, and especially humour on the Internet. In the realm of ‘Internet humour’, memes are a very frequently used resource. Memes are digitally created and shared images intended to be humorous that usually identify current events using popular culture references. However, memes are frequently used to reinforce social bonding rather than to share information, in a way in which one solidifies the in-group via the ridicule of others (Zappavigna, 2012) and, in this sense, they could be the perfect tool to spread online Islamophobia.

This paper analyses a corpus of 150 Islamophobic memes taken from social media (Facebook and Twitter) during the period 2014-2017 from a threefold perspective:

Describe the content of the memes applying the work on Visual Grammar Design (Kress and Van Leeuwen, 1996), Multimodal Analysis (Machin, 2007) and Visual Framing Theory (Rodríguez & Dimitrova, 2011; Geise 2017).

Analyse the ideology underlying them from a Critical Discourse Analysis perspective (Fairclough 1995)

Establish the connections between memes and hate speech: are memes a normalised way to disguise hate speech?

References

Baker, P., Gabrielatos, C., & McEnery, T. (2013). Sketching Muslims: A Corpus Driven Analysis of Representations Around the Word ‘Muslim’ in the British Press 1998-2009. Applied Linguistics, 34(3), 255-278.

Bunglawala, I. (2002). ‘British Muslims and the Media’, in Hamid, A., W. & Sherif, J. (eds.), The Quest for Sanity: Reflections on September 11 and the Aftermath (pp. 43-52). London: Muslim Council of Britain.
Fairclough, N. (1995). Critical Discourse Analysis. London: Longman.
Frost, D. (2008). Islamophobia: examining causal links between the media and ‘race hate’ from ‘below’. International Journal of Sociology and Social Policy, 28(11), 564-578.
Geise, S. (2017). ‘Visual Framing’. The International Encyclopedia of Media Effects, 1–12
Kress, G. and T. Van Leeuwen (1996). Reading Images: the Grammar of Visual Design. London & New York: Routledge.
Larsson, G. (2007) ‘Cyber-Islamophobia? The case of WikiIslam’. Contemporary Islam, 1, 53-67.
Machin, D. (2007). Introduction to Multimodal Analysis. London & Oxford: Bloomsbury.
Rodríguez, L. and D. V. Dimitrova (2011) ‘The Levels of Visual Framing’, Journal of Visual Literacy, 30, (1), 48-65.
Said, E. (1996). Covering Islam. How the Media and the Experts Determine How we see the Rest of the World. New York: Vintage.
Werbner, P. (2005). Islamophobia: Incitement to Religious Hatred: Legislating for a New Fear? Anthropology Today, 21(1), 5-9.
Zappavigna, M. (2012). Discourse of Twitter and Social Media. How we Use Language to Create Affiliation on the Web. London: Continuum.


Using variation and unpacking to help students decode disciplinary-specific semiotic resources

John Airey

Stockholm University and Uppsala University
[e-mail]

In this presentation I will describe a social semiotic approach (Halliday 1978; van Leeuwen 2005) to the multimodal teaching and learning of a discipline that takes variation theory (Marton & Booth 1997; Runesson 2005) as its theoretical framing. Following Airey and Linder (2017:95) I define social semiotics as “the study of the development and reproduction of specialized systems of meaning making in particular sections of society”.

Learning at university level involves coming to understand the ways in which disciplinary-specific semiotic resources can be coordinated to make appropriate disciplinary meanings (Airey & Linder 2009). Nowhere is this more true than in undergraduate physics where a particularly wide range of semiotic resources such as graphs, diagrams, mathematics and language are essential for meaning making. In order to learn to make these disciplinary meanings, students need to discover the disciplinary affordances (Fredlund et al. 2012, 2014; Airey & Linder 2017) of the semiotic resources used in their discipline.

Fredlund et al. (2015) propose a three-stage process that lecturers can use to help their students:

1. Identify the disciplinary relevant aspects needed for a particular task.

2. Select semiotic resources that showcase these aspects.

3. Create structured variation within these semiotic resources to help students notice the disciplinary relevant aspects and their relationships to each other.

However, many disciplinary specific semiotic resources have been rationalized to create a kind of disciplinary shorthand (Airey 2009). In such cases the disciplinary relevant aspects needed may no longer be present in resources used, but are rather implied. In such cases the resources will need to be unpacked for students (Fredlund et al. 2014). Such unpacking increases the pedagogical affordance of semiotic resources but simultaneously decreases their disciplinary affordance.

References

Airey, J., & Linder, C. (2017). Social Semiotics in University Physics Education. In D. F. Treagust, R. Duit, & H. E. Fischer (Eds.), Multiple Representations in Physics Education (pp. 95-122). Cham, Switzerland: Springer.

Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. Journal of Research in Science Teaching, 46(1), 27-49.
Fredlund, T., Airey, J., & Linder, C. (2012). Exploring the role of physics representations: an illustrative example from students sharing knowledge about refraction. European Journal of Physics, 33, 657-666.
Fredlund, T., Airey, J., & Linder, C. (2015). Enhancing the possibilities for learning: Variation of disciplinary-relevant aspects in physics representations. European Journal of Physics, 36(5), 055001.
Fredlund, T., Linder, C., Airey, J., & Linder, A. (2014). Unpacking physics representations: Towards an appreciation of disciplinary affordance. Phys. Rev. ST Phys. Educ. Res., 10 (020128).
Halliday, M. A. K. (1978). Language as a social semiotic: The Social Interpretation of Language and Meaning. London: Arnold.
Marton, F., & Booth, S. (1997). Learning and awareness. Mahwah, NJ: Lawrence Erlbaum Associates.
Runesson U (2005) Beyond discourse and interaction. Variation: a critical aspect for teaching and learning mathematics Cambridge Journal of Education 35, 69–87
van Leeuwen, T. (2005). Introducing social semiotics. London: Routledge.
Volkwyn, T. Airey, J. PERC proceedings

Multimodal Science and Engineering Teaching: Perspectives from 8ICOM

John Airey

Department of Physics and Astronomy, Uppsala University, Sweden

Zachary Simpson

Department of Mathematics and Science Education, Stockholm University, Sweden

The previous international conference on multimodality – 8ICOM – featured two sessions devoted to multimodal, social semiotic approaches to science teaching and learning (c.f. Halliday1978; van Leeuwen 2005, Airey & Linder 2017). What the papers in these sessions shared was the argument that such perspectives on science, and science teaching, can, at least in part, respond to calls to ‘democratize’ science education by recognising diverse sets of semiotic resources and, in so doing, seeking to address impediments to equal participation (Burke et al., 2017). 

The 8ICOM science sessions were particularly noteworthy given the backdrop against which 8ICOM had been organised. In the months leading up to the conference, South Africa (and Cape Town, in particular) had experienced campus unrest aimed at ‘decolonizing’ higher education in that country. As part of this movement, the phrase #ScienceMustFall briefly trended on social media. This emanated from the claim that ‘science’ is a western, colonial construct that needs to be dismantled and replaced with the teaching of indigenous, African knowledge. Although the #ScienceMustFall slogan has since departed from the wider public consciousness, the questions it raises nonetheless remain: why, and how, should science be taught?  Is science more than just a western colonial construction and, if so, why? And, what can the concept of multimodality offer by way of answering these questions? 

In this paper, we offer an overview of the multimodal science papers presented in the two sessions at 8ICOM in the light of these questions. This is done with a view to assessing where the multimodality community finds itself regarding science education, and how it might address questions of the legitimacy of western science in the future. It is thus an attempt, as the conference theme suggests, to ‘move the theory forward’.       

References

Airey, J., & Linder, C. (2017). Social Semiotics in University Physics Education. In D. F. Treagust, R. Duit, & H. H. Fischer (Eds.), Multiple Representations in Physics Education (pp. 95-122). Cham, Switzerland: Springer.

Burke, P. J., Crozier, G. and Misiaszek, L. I. 2017. Changing Pedagogical Spaces in Higher Education: Diversity, Inequalities and Misrecognition. London: Routledge.

Halliday, M. A. K. (1978). Language as a social semiotic: The Social Interpretation of Language and Meaning. London: Arnold.

van Leeuwen, T. (2005). Introducing social semiotics. London: Routledge.

“It can cry, it can speak, it can pee”: Modality Values and Playing Affordances in Contemporary Baby Dolls’ Discourse

Danielle Barbosa Lins de Almeida

Universidade Federal da Paraíba/CNPq

[e-mail]
Baby dolls have been in the toy market for more than a hundred years, since French firm Jumeau entered the toy industry in the 19^th century and started producing ‘bébés’, considered the greatest phenomena of the toy market (FLEMING, 1996). The aim of this analysis is to shed some light on the multimodal properties provided by the aural, verbal and visual texts of the packages of Brazilian baby dolls through a careful look at their textual and contextual meanings, anchored on Kress & Van Leeuwen’s (2006) subsystem of modality (reality value), within the interpersonal visual metafunction. The analyses of the baby dolls’ packages point to roles suggested to young girls from very early age, varying from parenting roles they are asked to fullfill later in life as future mothers to medical abilities they are encouraged to master in order to care and nurture for their “children”.
Exploring academic argument in data visualizations

Arlene Archer

University of Cape Town, South Africa
[e-mail]

In an age where more and more data are produced and circulated visually, and digital environments make the production of data visualizations increasingly accessible, it is important to develop critical tools for students to engage with these texts. Data can be represented through a range of modes (such as writing, visuals and numbers) and different information graphics (such as tables, charts, graphs). There are design choices to be made in terms of size, shape, colour and composition in order to represent a particular argument to a particular audience in the most apt way. Using a multimodal social semiotic approach, this presentation investigates rhetorical and semiotic strategies for realizing argument in data visualization texts. The strategies investigated include aspects such as use of colour, typography, graphics (like using a bar rather than a point on a graph); as well as the relation between academic argument and citation in information graphics. The effect of the underlying basis for comparison of data is examined, as is the selection and processing of data. The presentation investigates i) the semiotic encoding of ideational material and ii) the ways relationships are established within the discourse communities constructed in particular data visualizations. This has implications for the way we teach these text-types in Higher Education in order to produce critical citizens; both in terms of production and critical analysis.

Discourses in and around upcycled artefacts as instantiations of discourse

Arlene Archer

University of Cape Town, South Africa
[e-mail]

Anders Björkvall

Örebro University, Sweden
[e-mail]

This presentation looks at artefacts from a social semiotic perspective in order to explore global / local discourses instantiated in and circulating around ‘upcycled’ artefacts. Upcycled products tend to gain not only economic value in the process of being transformed from used, old things, but also ethical value through being created out of a responsibility for the environment and resistance toward mass consumerism. The aesthetic and functional values that are added in upcycling address the demands of different local and global markets. The focus of this presentation is how discourses in and around an upcycled artefact make it possible for it to move between cultural and geographical spaces, whilst both maintaining and transforming the meaning potential of the artefact. We look at discourses in artefacts as instantiated through explicit semiotic resources (specifically material, shape and colour). In thinking about discourses ‘around’, we apply Pietikäinen and Kelly-Holmes’ (2011) notions of authenticity and mobility, and use the concept of provenance (Adami, 2015; Björkvall 2018; Djonov and van Leeuwen 2011) to discuss how upcycled artefacts relate to other texts, semiotic domains, and practices. The data explored include artefacts made from different kinds of waste plastic bottles. We highlight the ways in which these artefacts draw on such varied phenomena as environmentalism, poverty reduction, Western functionalism, and trophy hunting. We demonstrate how an analysis of discourses and semiotic resources in an artefact as well as the discourses, narratives and practices around it can provide us with a number of insights regarding local authenticity, global mobility, and other meaning potentials of upcycled objects.

References

Adami, E. (2015) Aesthetics in digital texts beyond writing: A social semiotic multimodal framework. In A. Archer and E. Breuer (eds.) Multimodality in Writing: The State of the Art in Theory, Methodology and Pedagogy. Leiden & Boston: Brill.

Björkvall, A. (2018) Critical genre analysis of management texts in the public sector: Towards a theoretical and methodological framework. In C. Seiler Brylla, G. Westberg and D. Wojahn (eds.) Kritiska textanalyser. (Södertörn Discourse Studies 6.) Huddinge: Södertörns högskola.

Djonov, E and van Leeuwen, T. (2011) The semiotics of texture: From tactile to visual. Visual Communication 10 (4), 542-564.

Pietikäinen, S. and Kelly-Holmes, H. (2011) The local political economy of languages in a Sami tourism destination: Authenticity and mobility in the labelling of souvenirs. Journal of Sociolinguistics 15 (3), 323-346.


Truth, image and politics: the visual discourse of the "Alianza Cambiemos"

Ana Aymá

Universidad Nacional de Quilmes, Buenos Aires, Argentina
[e-mail]

One of the strategies of the presidential campaign (Garcia Beaudoux, V. et al., 2005) of 2015 in Argentina, of then candidate and now President, Mauricio Macri, was appealing to a "new way of making politics" based on the slogan: "talk less and do more." In this paper, we propose to investigate the sign making process of the visual discourse of the Alianza Cambiemos, focusing on the spots of the general election campaign in 2015. We research the function of images in Cambiemos discourse, used as a semiotic resource in confrontation with the common usage of speech in the political field.

There are different ways of making representations in visual communication, according to the multimodal approach within social semiotics (Kress, 2010; van Leeuwen, 2008; Kress & van Leeuwen, 2001; Kress & van Leeuwen, 2004). As Kress and van Leeuwen have said, the representational metafunction is the one that refers to the participants represented, that is to say, the people, places and objects that can be seen within the image. Within this metafunction and depending on whether the participants are taking action or not, two types of structures can be distinguished: narrative and conceptual. The narrative images allow observers to create a story about the participants, in a more personal approach, since they include motion vectors; the conceptual images instead do not include vectors, and participants are usually grouped together to present to the observer the "concept" of who or what they represent, in more impersonal terms. These elements of analysis allow us to recognize semiotic strategies of identification (Fairclough, 2004) in the framework of a political structure that explicitly confronts and avoids the use of speech as semiotic resource.

A first approach indicates that a relationship is established between the images, the facts and the truth, in open confrontation with political speech, which is seen as a device of deceit.

References

Fairclough, Norman (2004) Analysing discourse. Textual analysis for social research. Londres: Routledge.

Garcia Beaudoux, V. Et al. (2005) Comunicación política y campañas electorales. Barcelona: Gedisa

Kress, Gunther. (2010) Multimodality. A social semiotic approach to contemporary communication. Londres. Routledge.

Kress G. & Van Leeuwen, Theo (2001) Multimodal discourse. The modes and media of contemporary communication. Londres. Arnold

Kress G. & Van Leeuwen, Theo. (2004) Reading Images. The Grammar of Visual Design. Londres, Routledge.

Van Leeuwen, Theo (2008) Introducing Social Semiotics. Londres: Routledge.
The Resemiotization of the Notion of Educational Inclusion from the Official Voice: Multimodal Critical Analysis of the Website of the Chilean Ministry of Education

Patricia Baeza

Pontificia Universidad Católica de Valparaíso, Research Center for Inclusive Education, Fondecyt 1180472
[e-mail]

Dominique Manghi

Pontificia Universidad Católica de Valparaíso, Research Center for Inclusive Education, Fondecyt 1180472
[e-mail]

Liliana Vásquez

Universidad Andrés Bello, Fondecyt 1180472

In 2016 the Chilean Government enacted the Inclusion Act, which eliminated for-profit education and established free and non-selective access to schools. The official voice resemiotizes the Act (Iedema, 2003) and its power is present in semiotic practices by means of the promotion of some signs and meanings as well as the annulment of others (Bezemer & Kress, 2016). This study aims to describe the meaning making within the website of the Chilean Ministry of Education, in which the Inclusion Act is resemiotized. The qualitative methodology incorporates analytical tools of Multimodal Critical Discourse Analysis, Social Semiotics and the Grammar of Visual Design (Kress & van Leeuwen, 2006). The corpus is comprised of static and dynamic multimodal texts (videos) of the web page, which are analyzed manually from a multimodal approach in order to denaturalize the semiotic ensembles through which the meanings of inclusive education are made. Among the preliminary results, on the one hand, the web page of the Ministry of Education resembles the concept of inclusion in an evoked way, through non-permanent banners that recontextualize four conditions in which inclusion occurs: a) free of charge education is expressed by means of an intermodal ensemble; b) the strengthening of public education, which is instanced through writing; c) gender equity that is represented multimodally; and d) the diversification of teaching, which is presented through speech. On the other hand, the web page shows a low prominence of the concept of inclusion, which contradicts the spirit of the Inclusion Act that seeks to expand the beneficiaries of educational inclusión and to promote structural changes of the educational system.

Keywords: Multimodality, public policy discourse, social semiotics, resemiotization

References

Bezemer, J. & Kress, G. (2016). Multimodality, Learning and Communication: A Social Semiotic Frame. London: Routledge.

Fairclough, N. (2010). Critical Discourse Analysis: The Critical Study of Language. London & New York: Routledge.

Kress, G. & van Leeuwen, T. (2006). Reading images: A grammar of visual design. London: Routledge.

Iedema, R. (2003). Multimodality Resemiotization: Extending the Analysis of Discourse as a Multisemiotic Practice, Visual Communication 2 (1): 29-57.

Slee, R. & Allan, J. (2001). Excluding the included: A reconsideration of inclusive education. International Studies in Sociology of Education, Volume 11, Number 2,173-191.