Multimodality, ethnography and education in south america

Action competence by way of a representation focus

Heidi Kristensen, PhD student

Erik Knain, professor
Department of Teacher Education and School Research, University of Oslo
[e-mail]

Whereas there is much research on teaching and learning in school science focusing on science as a body of knowledge (i.e. concepts, models and theory complexes), less research is done into science as part of complex environmental issues when focusing on representation practices. A key goal of education for sustainable development is for students to develop action competence (Jensen & Schnack, 1997; Sinnes, 2015) and agency in transformation processes (O'Brien & Sygna, 2013). In transformation processes, students have the potential of developing reflexivity leading to a capacity for change. Identity is an important part of action competence. Individuals' access to different types of knowledge, and different experiences and interpretations of these, contribute to the identity “work” across acts of meaning in representation practices. The presentation will address the issue of how action competence can be developed and analyzed when students are working with environmental issues supported by a multimodal representation focus. Thus transformational learning is investigated by transformations of semiotic resources. The main research question is: How can agency and action competence be analyzed by a representation focus? Representations need to be construed in the intersection between culture and self, and through intersubjective dialogue. This also means transforming and negotiation social norms as genres in the situation (Bezemer & Kress, 2015). Key rationales for the study is UNESCO’s (2017) framework for implementing ESD in education and design principles developed by Tytler et.al. (2013). These principles are developed into principles for working with socio-scientific issues (Sadler, Klosterman, & Topcu, 2011) by Knain et al. (2017). A framework for analyzing action competence is presented with examples of analysis from upper secondary classrooms. Analysis of interaction trajectories focusing on students’ transformations of and interaction with multimodal texts are presented. The study is part of the REDE project (Representation and participation in school science) at the University of Oslo.

References

Bezemer, J., & Kress, G. (2015). Multimodality, Learning and Communication. A socialsemiotic frame. London: Routledge.

Jensen, B. B., & Schnack, K. (1997). The action competence approach in environmentaleducation. Environmental Education Research, 3(2), 1-22.

Knain, E., Fredlund, T., Furberg, A., Mathiassen, K., Remmen, K. B., & Ødegaard, M. (2017). Representing to learn in science education: Theoretical framework and analyticalapproaches. Acta Didactica Norge, 11(3).

O'Brien, K., & Sygna, L. (2013). Responding to climate change: The three spheres ofTransformation. Proceedings of Transformation in a Changing Climate, 19-21 June 2013, 16 23.

Sadler, T. D., Klosterman, M. L., & Topcu, M. S. (2011). Learning Science Content and Socio-scientific Reasoning Through Classroom Explorations of Global Climate Change. In T.D. Sadler (Ed.), Socio-scientific Issues in the Classroom (pp. 45-77). Dortrect: Springer.

Sinnes, A. (2015). Utdanning for bærekraftig utvikling - hva, hvorfor og hvordan? Oslo: Universitetsforlaget.

Tytler, R., Prain, V., Hubber, P., & Waldrip, B. (Eds.). (2013). Constructing Representations to Learn in Science. Rotterdam: Sense Publishers.

UNESCO. (2017). Education for sustainable development goals - Learning planet objectives.

Digital collage as play material for students’ composing of multimodal picture books

Kirsten Linnea Kruse, associate professor and PhD-candidate

University College of Southeast Norway (USN)
[e-mail]

In this project, contemporary picture book aesthetics has been an inspiration for designing reading and text creating processes at a third-grade level in a Norwegian primary school. In this presentation, I will investigate what kind of potential a digital collage technique has for students’ meaning making processes in their shaping of narrative picture books.

Many contemporary picture books contain digital collages, which means that the creators experiment with and combine different kinds of visual material, like for example photography and drawings in an interplay with written text. Druker (2008) defines the picture book collage as «a broad aesthetic principle of combination and synthesis» (s. 46).

The study is conducted as a qualitative Educational design research project (EDR). This means that knowledge about students’ multimodal meaning making is gained through a process of related and iterative interventions, developed and reflected in close collaboration between researcher and teachers.

Based on the main theoretical frameworks for the study (multimodality and social semiotic theory and picture book theory), I will provide multimodal analyses of selected spreads from the student’s picture books – combined with a view on the sign making process and the affordances of the digital artefact used. The students use a digital drawing tool for iPad, which offers the opportunities to integrate photography and digital drawings in different layers, to cut out, copy and enlarge objects, and modify backgrounds – all in a dynamic text creation process.

The analyses of the final picture books show that playing with and combining different materials create surprising expressions and work as catalysts for fantasy. It also leads to develop a metalanguage about narrative and multimodal composing. To create multimodal characters and environments in this way, gives the students an opportunity to communicate atmospheres and create emotionally engaging stories, which they would not have been able to communicate by writing alone.

References

Druker, E. (2008). From Avant-garde to Digital Images: Collage in Nordic Picturebooks. Bookbird: A Journal of International Children’s Literature, 46(3), 45-51.

Kress, G. R., & van Leeuwen, T. (2006). Reading Images: The Grammar of Visual Design. London: Routledge.

Painter, C., Martin, J. R., & Unsworth, L. (2013). Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox.

Pantaleo, S. (2015). Exploring the artwork of young student’s multimodal compositions. Education 3-13. International Journal of Primary, Elementary and Early Years Education, s.1-19.


Recontextualizing literacy. ABC-apps as semiotic technology

Gunhild Kvåle

University of Agder, Norway

Tablets, such as the iPad, have become popular amongst pre-school children, and many are marketed for their potential for learning to read and write. The semiotic technology of the various apps can be imbued with various views on literacy and support the literacy development of children in various ways and to various degrees. This paper will investigate a corpus of commercially produced ABC apps aimed at Norwegian pre-school children for use outside of educational institutions. It will analyze and critique how literacy is recontexualized in these apps, including what dimensions of literacy that are included/excluded, what roles children are offered, and what social values that are promoted. The study will be theoretically informed by social semiotic technology studies (Zhao, Djonov and van Leeuwen, 2014; Kvåle, 2016), sociocultural literacy studies (Barton, 2007; Barton & Hamilton, 1998), and previous studies of ABC resources for pre-school children (e.g. Vollan, 2010). The paper aims at bringing forward knowledge and critical awareness of what assumptions of literacy that commercial software producers subscribe to and inscribe into the apps, at discussing how quality may be understood, and at expanding and strengthening the theoretical-methodological basis for social semiotic multimodal approaches to technology.

References

Barton, D. (2007). Literacy: an introduction to the ecology of written language. 2nd ed. Malden, Mass: Blackwell publishing.

Barton, D. & Hamilton, M. (1998). Local literacies: reading and writing in one community. London: Routledge.

Kvåle, G. (2016.) Software as Ideology. A Multimodal Critical Discourse Analysis of Microsoft Word and SmartArt. In Journal of Language and Politics – Special Issue on Multimodality, Politics and Ideology 15 (3): 259–273.

Vollan, M. (2010). Med geit for g – kritiske blikk på læremidler for de minste. In D. Skjelbred & B. Aamodtsbakken (Eds.). Faglig lesing i skole og barnehage, s. 261–278. Oslo: Novus forlag

Zhao, S., Djonov, E. & van Leeuwen, T. (2014). Semiotic technology and practice: a multimodal social semiotic approach to PowerPoint. Text and Talk 34 (3): 349-375.

Multimodal narratives on teaching Spanish as a foreign language to students with Autistic Spectrum Disorder

Alex Bezerra Leitão

University of Brasília, Brazil
alexb.leitã[e-mail]

This study aims to analyze a multimodal narrative (van LEEUWEN, 2005; ALANEN, R.; KALAJA, 2013) of a Spanish student as a foreign language learner, diagnosed with Autistic Spectrum Disorder. The study was carried out within the framework of Conceptual Metaphor theory, drafted primarily by Lakoff and Johnson (1980) and subsequently expanded by Kövecses (2005, 2010) and Faith (2007), in relation to the cultural variability and the Idealized Cognitive Models, respectively, and by Forceville (2009), about the Multimodal Metaphor. This research is a qualitative case study (CHADDERTON and TORRANCE, 2015) and the procedures for this data collection are: researcher's observation (STAKE, 2011), multimodal narrative collection (MELO-PFEIFER, 2015; P, 2016) and semi-structured interview (ARNOLDI, 2006). For data triangulation (STAKE, 2011), the following instruments are used: fieldnotes (FLICK, 2009), audio recordings (STAKE, 2011) and reflective sessions (ABRAHÃO, 2006). The results of this study show that the student with Autistic Spectrum Disorder produces visual-verb metaphors and multimodal metaphors, which indicates that source and target domains are conceptualized and produced in different ways. Therefore, the use of multimodal narratives in teaching a foreign language to learners who have symptoms of Autistic Spectrum, through this study, indicates that the work with a multimodal approach in a foreign language classroom produces multiple senses that go beyond semiotic resources focused only on oral expression or written production. This practice, therefore, contributes to the process of school and social inclusion and the reduction of attitudinal asymmetrical barriers (AMARAL, 1998), which prevent the establishment of an inclusive education.

Key words: Multimodal narratives, multimodal metaphor, autistic spectrum disorder., teaching and learning foreign languages


A social semiotic perspective on multimodal language practices for teaching Coriolis acceleration in engineering dynamics

Kate le Roux, Bruce Kloot
University of Cape Town, South Africa

This paper describes and explains lecturers’ multimodal language practices in an undergraduate engineering dynamics course at a South African university. Engineering dynamics uses physics and mathematics knowledge to understand motion and forces in the context of engineering mechanisms. This paper draws on a wider study of language use in the first of two compulsory dynamics courses for students enrolled in a four-year professional bachelor degree in mechanical engineering or mechatronics. Consistent with reports in the literature (e.g. Fang, 2014), students describe the course as ‘horrible’ and ‘difficult’, with difficulties having significant implications for progression and subsequent curriculum coherence.

The analytic focus of this paper is lectures on the topic of ‘Coriolis acceleration’, a key dynamics concept. While the ‘digital turn’ affords new possibilities for using time, space and texts in education (Mills, 2016), the lecture  and lecture recordings made available to students on course websites  remain central in undergraduate education in contexts characterised by resource constraints (Thesen, 2007), diverse student intakes and disruptions to classes. Transcribed recordings of two lecturers’ classes are analysed from a social semiotic perspective, using the concepts of mode, the three metafunctions of language (ideational, interpersonal and textual), rhetorical move, and contexts of culture and of situation (e.g. Kress, 2014; Kress, Jewitt, Ogborn and Tsatsarelis, 2001; Morgan, 2006). The micro-level analysis explores how different language modes act textually to produce coherent conceptual and interpersonal meanings. Contextual documents and interviews with the lecturers are used to explain lecturers’ choices. This understanding of lectures’ multimodal language practices  which are key to the ‘semiotic landscape’ (Kress & van Leeuwen, 1996) of the dynamics course  poses questions in the wider study about who may be included in/excluded from participation in engineering dynamics.

References

Fang, N. (2014). Difficult concepts in engineering dynamics: Students’ perceptions and educational implications. International Journal of Engineering Education, 30, 1110-1119.

Kress, G. (2014). Design: The rhetorical work of shaping the semiotic world. In A. Archer & D. Newfield (Eds.), Multimodal approaches to research and pedagogy: Recognition, resources and access (pp. 131-152). New York: Routledge.

Kress, G., Jewitt, C., Ogborn, J., & Tsatsarelis, C. (2001). Multimodal teaching and learning: The rhetorics of the science classroom. London: Continuum.

Kress, G. & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London: Routledge.

Mills, K.A. (2016). Literacy theories for a digital age: Social, critical, multimodal, spatial, material and sensory lenses. Bristol: Multilingual Matters.

Morgan, C. (2006). What does social semiotics have to offer mathematics educationresearch? Educational Studies in Mathematics, 61, 219-245.

Thesen , L. (2007). Breaking the frame: Lectures, ritual and academic literacies. Journal of Applied Linguistics, 4(1), 33-53.

Transduction practices in the learning of stereochemistry: towards developing a multimodal theory of emergent learning

Cedric Linder

Department of Physics and Astronomy, Uppsala University, Sweden
[e-mail]

Susanne Wikman

Department of Chemistry and Biomedical Science, Linnaeus University, Sweden
[e-mail]

Anne Linder

Department of Physics and Astronomy, Uppsala University, Sweden
[e-mail]

In disciplinary learning classrooms, access to the intended object of learning is constituted through the affordance of discerned disciplinary relevant aspects, which are distributed across semiotic resources (Airey & Linder 2009; 2017; Eriksson et al. 2014; Fredlund et al. 2015). From a multimodal perspective this means that such learning needs to be seen in terms of getting to be able to interpret and use the meaning potential of these disciplinary-specific semiotic resources (Kress 2010; Jewitt 2008). The aim of this presentation is to use this framing to make a theoretical link to the complex system notion of emergence as characterized for educational practices by Davis & Sumara (2006). The data environment is interactive learning with stereochemical molecular-structure identification exercises, which takes place in a five-week introductory level organic chemistry course. The data environment is chosen because of the appresent dynamics (Linder 2013) that the stereochemistry curriculum presents – the disciplinary relevant aspects are microscopic and thus their discernment and affordance requires semiotic mediating tools to facilitate visualization transduction (Bezemer & Kress 2008; Kress 2010). The analysis shows how, through transduction, students in group-work situations combine disciplinary convention with own alternative invention to create semiotic resources that they are able to engage with in a meaningful way, both concretely and visually.

References

Airey, J. & Linder, C. (2017). Social Semiotics in University Physics Education, in Treagust, D., Duit R., Fischer, H. (eds) Multiple Representations in Physics Education: Springer.

Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. Journal of Research in Science Teaching, 46(1), 27-49.

Bezemer, J., & Kress, G. (2008). Writing in multimodal texts: a social semiotic account of designs for learning. Written Communication, 25(2), 166-195.

Davis, B., & Sumara, D. (2006). Complexity and education: Inquiries into learning, teaching and research: Erlbaum.

Eriksson, U., Linder, C., Airey, J. & Redfors, A. (2014). Introducing the anatomy of disciplinary discernment: An example from astronomy. European Journal of Science and Mathematics Education, 2(3), 167-182.

Fredlund, T., Linder, C. & Airey, J. (2015). A social semiotic approach to identifying critical aspects. International Journal for Lesson & Learning Studies, 4(3), 302-316.

Jewitt, C. (2008). Multimodality and Literacy in School Classrooms. Review of Research in Education 32; 241, DOI: 10.3102/0091732X07310586

Kress, G. (2010). Multimodality. A Social Semiotic Approach to Contemporary Communication. London: Routledge.

Linder, C. (2013). Disciplinary Discourse, Representation, and Appresentation in the Teaching and Learning of Science. European Journal of Science and Mathematics Education, 1(2), 43-49.

Signs of multimodal genre awareness in young YouTubers' online engagements

Fredrik Lindstrand, professor

Department of Visual Arts and Sloyd Education, Konstfack University College of Arts, Crafts and Design, Stockholm, Sweden
[e-mail]

In Sweden, big efforts are currently being made in what is referred to as the "digitization of school" (cf. Government of Sweden, 2017) and in the work to support media and information literacy (MIL) at a more general level. Since many young people today are deeply engaged in various forms of digital media on the one hand (Swedish Media Council, 2017) while lacking commitment to their education on the other, a focus on digitization and media literacy could perhaps bridge these motivational gaps – at least to some extent, for some pupils. However, instead of listening to and acknowledging children's knowledge and experience within this field there seems to be a tendency to frown upon their engagements in digital media and to describe their activities on digital platforms as potentially harmful.

This paper presents results from of an on-going pilot project, Learning in Digital Wastelands (Lindstrand, 2018), on children's learning and designs for learning (cf. Selander & Kress, 2010; Bezemer & Kress, 2008) in digital arenas outside school. The aim of the project is to investigate meaning-making and designs for learning in digital contexts outside school where children and young people are engaged in multimodal sign-making practices (Kress, 2003; 2010). An incentive is, perhaps naïvely, that this may offer new perspectives on resources and designs for learning suitable for children today.

More specifically, the paper presents a multimodal analysis of the opening sequence of a video posted on YouTube by a nine-year old. By contextualising the video and its modal configuration and orchestration through comparisons with other YouTube videos referred to by this young producer, the paper claims that the video indicates a high level of multimodal genre awareness. As a conclusion it is suggested that a curiosity in what children do outside school could give great leads in terms of how to take pedagogy further and work within genres, modes and media that are relevant to children today. Who knows what genres will be dominant tomorrow (cf. Hyon, 1996; Johns, 2002)?

References

Bezemer, J. & Kress ,G. (2008) Writing in Multimodal Texts. A Social Semiotic Account of Designs for Learning, Written Communication 25(2).

Government of Sweden (2017) "Action on digital transformation", retrieved 20180110 from http://www.government.se/press-releases/2017/06/action-on-digital-transformation/

Hyon, S. 1996. Genre in three traditions: Implications for ESL. TESOL Quarterly 30(4):693-722.

Johns, A.M. (Ed.). 2002a. Introduction. Genre in the classroom. Multiple perspectives. London: Lawrence Erlbaum, pp. 3-13.

Kress, G. (2003) Literacy in the new media age. London: Routledge.

Kress, G. (2010) Multimodality. A social semiotic approach to contemporary communication. London: Routledge.

Lindstrand, F. (2018, in press) "Relevanta utsvävningar på digitala arenor - om YouTube, literacy och genremedvetenhet"[Relevant dissipations on digital arenas - on YouTube, literacy and genre awareness] in Forsgren Anderung, K. & Folkesson, E. (eds.) Att mötas kring berättelser i olika former [Meeting with stories in different forms] [preliminary title]. Sandvikens kommun: Kulturcentrum.

Selander, S. and Kress, G. (2010) Design för lärande. Ett multimodalt perspektiv. [Designs for learning. A multimodal perspective]. Stockholm: Norstedts.

Swedish Media Council [Statens Medieråd] (2017) Ungar och medier 2017 [Kids and media 2017]. Retrieved 20180111 from https://statensmedierad.se/publikationer

Student teachers’ perspective of representations in science teaching and learning

Mai Lill Suhr Lunde, Ketil Mathiassen, Tobias Fredlund, Erik Knain

Department of Teacher Education and School Research, University of Oslo
[e-mail]

Representations such as writing, speech, graphs, diagrams, gestures and simulations are important tools for teaching and learning in science (Knain, 2015). Representations are also valuable tools for making student understanding visible for sharing, discussion and mentoring.

In this study, our aims were to study student teachers’ conceptualization of representations as tools for student learning, the importance of representations related to their own field of science and their experiences with representations during teaching practice. We performed focus group interviews with student teachers before and after teaching practice, along with group discussions on selected representations. We also studied exam papers from a small number of student teachers focusing on representations. The data was analysed by thematic analysis using software for qualitative analysis, ATLAS.ti.

Preliminary findings suggest that before teacher practice the student teachers were familiar with the concept of representations and the importance of representations as tools for learning in science. They were aware of challenges related to interpretation of representations, and that different representations and combinations of representations can support student learning in science. During teacher practice the student teachers seem to have developed a greater awareness of the nature of representations, what students need to know and that they should be able to interpret and make their own representations. However, student teachers also report on limited possibilities to focus on representations during their teacher practice.

We conclude that working with representations for teachers and student teachers is related to the development of an awareness of representations as fundamental tools and forms of expression in science learning. An important task is to enable the student teachers to study their own teaching practice by building a bridge between subject, pedagogical content knowledge and teaching practice, creating a “third space” as an arena for the student teachers’ professional development as teachers.