Multimodality, ethnography and education in south america

.

COPE, Bill; KALANTZIS, Mary. [s.d.] Ubiquitous learning: an agenda for educational transformation. In: Proceedings of the 6th International Conference on Networked Learning, 576-582. Avaliable in: . Accessed in: December 24, 2017.

KRESS, G. (2010). Multimodality. A social semiotic approach to contemporary communication. New York: Routledge, 2010.

Commercial illustration – a process of adaptation of meanings

Leena Raappana-Luiro

University of Lapland
[e-mail]

This presentation is based on an article, which is in process and is aimed to be a part of my doctoral thesis in graphic design.

Compared with (fine/free) art, illustration as an applied art has a less independent nature. As an image, illustration has a certain object, which it illustrates. It has to go through a process with several actors. An illustrator as a designer modifies her/his ideas in co-operation with a client. This study is focusing on the way in which a designer’s original, often relatively intuitive ideas are adapted for a client’s purposes in the case of a postage stamp design process.

Concerning my position as a researcher, the point of view is a practitioner’s one: I examine this particular process from within by making detailed notes during the sketching rounds and the changes made in stamp designs. These different drafts and versions of designs function as empirical data for this research.

As a researcher-designer, interested in semiotics, these theories are inevitably a part of my knowledge, even if some stages of the design process seem to be more spontaneous and even accidental. From a semiotic point of view, all the design elements convey meanings – in the case of graphic design, they can be supposed to be more or less intentional and controlled. Theories of visual grammar and multimodal semiotics offer the lens through which the design solutions and changes in them are examined: What are the meanings aroused and how are they created by different semiotic resources such as colours, composition and typography, and by wider concepts of style and technique as well as the objects depicted. As a result, it seems that in the case of a commercial illustration, the meanings are pushed towards unambiguous, soft and neutral ones.

Keywords: Illustration, graphic design, practice led design research, postage stamps, multimodal semiotics, semiotic meanings

Moving in and out of being “on record” by gaveling

Nathalie Schümchen, Maria Vanessa aus der Wieschen

Department for Design and Communication, University of Southern Denmark, Sønderborg
[e-mail]

We analyze video recordings of an experimental set-up in which two participants at a time work together in order to make sense of different visualizations of German prosody. The participants are instructed to familiarize themselves with and evaluate several types of prosody visualization systems. The participants are learners of German as a Foreign Language and read example sentences aloud paying special attention to producing the correct intonation as indicated by the visualization. The participants move through different subtasks; 1) discussing their understandings of the notation system in question, 2) practicing reading the sentences, and 3) taking turns producing an “on-the-record”-version of each sentence. The participants themselves negotiate when to move from one subtask to the next. Intended to facilitate the phonetic analysis by providing a clear auditive signal prefacing the on-the-record talk, the participants are asked to mark their “on-the-record”-talk with the strike of a hammer (“the gavel”).

We are particularly interested in the way the participants move in and out of subtask three, producing “on-the-record” utterances, and in finding out what the phonological and embodied characteristics of “on-the-record-talk” in an experimental setting are. Additionally, we aim to describe the practical methods participants use to move in and out of “on-the-record-talk”.

Our analysis revolves around the coordination of verbal and embodied resources participants use to mark when “on-the-record”-talk is being done, focusing is on the use of said hammer. Our preliminary analysis shows that the hammer is used as a tangible tool to perform several actions, such as functioning as a “talking stick” that signals whose turn it is to talk and as a type of “metronome” that establishes and maintains a rhythm for reading aloud and turn-taking.

Security and danger – how to foster public behaviour by way of visual representations

Staffan Selander, Stockholm University

In her book, Purity and danger (1966), Mary Douglas analysed the concepts of pollution and taboo, which she related to thinking about “us” and “outer boundaries”. I will here discuss visual representations of security and danger for air flight passengers. My starting point will not be “concepts”, but the leaflets one would find on any aeroplane today.

Transport concerns serious aspects of security, and passengers are thus fostered in relation to certain aspects. In most leaflets on aeroplane security, one will find information about how to:

close and open the security belt

sit on the chair

put on life vests

use the oxygen masks

– and when you are allowed to – use electric devices

act if you have to leave the aeroplane in case of a severe situation/accident

The information has to be serious, objective, general, trustworthy, and relevant. From this point of view, my focus will be on what is emphasized.

Another aspect that I will discuss is how the information is presented. The representation of something – the information linking – entails aspects like: What is given and what is new? What is seen as central? (Kress & van Leeuwen, 1996; van Leeuwen, 2006). I will also discuss what the information wants the reader to do during certain circumstances, what I call activity linking (Selander, 2017, 2018). How can we for example interpret significant differences between the following pictorial information on how to behave if smoke occurs in the aeroplane?

The third level of analysis will be how this information relates to over-arching ideas about security and danger in our societies, and to how the society is divided in terms of power, status, and life-styles etc. (Douglas, 1986, 1996).
Heroes and Saviours: The Bengali Hero in Children’s Comics during the Bangladeshi Struggle for Liberation, 1971

Tiyasha Sengupta

Department of Language and Communication, University of Southern Denmark, Odense

Heroic comics targeted towards children have been frequently used in conflictual situations, especially since the Second World War (DiPaolo, 2011; Riches et.al, 2009; Rifas, 2017; Watkins, 2000). This paper undertakes a comparative multimodal examination of the Bengali hero in two children’s comics, published in West Bengal, India, during the Bangladeshi War of Liberation 1971. The volatile relationship between India and Pakistan suffered another blow when the pogrom in East Pakistan by the West Pakistani army was vehemently condemned by India, ultimately leading to the Indo-Pakistan War of 1971, concluding with the birth of Bangladesh. The Indian state of West Bengal extended its solidarity to the ethnolinguistically similar East Pakistan’s plight (Ahmed, 2004; Pramanik, 2010; Shamshad, 2017). The phenomenon was significantly reflected in West Bengal’s contemporary mass media (Pramanik, 2010).

The paper investigates and compares two comic strips - Batul the Great and Poober Akash Laal (the eastern sky is red) published in the special festival issues of popular Bengali language children’s periodicals Shuktara and Kishore Bharati respectively. With the help of a combination of the Social Actors Approach of Critical Discourse Analysis and Social Semiotics, the various visual and linguistic methods employed in the production of the Bengali hero in the two comics have been examined. The analysis revealed the use of attires, physical attributes, languages and dialects in the delineation of the social actors. Additionally, visual devices like fonts, speech bubbles, frame, mis-en-scene, etc. have also been extensively used.

The comics, produced by Indian Bengali artists for Indian Bengali children (primarily residing in West Bengal), served to raise awareness about the conflict among their young readers, and endow them with a sense of ethnolinguistic pride overlooking borders. In addition to contributing to the limited knowledge of Bengali language comics in the context of conflicts, the study also offers an insight to the complex functioning of identity in a diverse, multicultural canvas, like that of India.

References

Ahmed, S. (2004). Bangladesh: Past and Present. New Delhi: A.P.H.

Di Paolo, M. (2011). War, politics and Superheroes: Ethics and Propaganda in Comics and Film. Jefferson: McFarland & Co., Publishers.

Pramanik, B. (2010). Bangladesher Muktijuddhe Bharoter Bhumika. Kolkata: Patralekha.

Riches, A., Parker, T. and Frankland, R. (2009). When the Comics Went to War: Comic Book War Heroes. Edinburgh: Mainstream.

Rifas, L. (2017). War Comics. In: F. Bramlett, R. Cook and A. Meskin, ed., The Routledge Companion to Comics. New York: Routledge, pp.183-191.

Shamshad, R. (2017). Bengaliness, Hindu nationalism and Bangladeshi migrants in West Bengal, India. Asian Ethnicity, 18(4), pp.433-451.

Watkins, T. (2000). Piloting the Nation: Dan Dare in the 1950s. In: D. Jones and T. Watkins, ed., A Necessary Fantasy?: The Heroic Figure in Children's Popular Culture. New York: Psychology Press, pp.153-176.

Multimodal Content Analysis
Dr. Frank Serafini

Stephanie Reid

Traditional content analysis allows scholars to conduct systematic analyses across a broad, yet manageable corpus of selected, or meaningful, matter (Krippendorf, 2004). Once a research question has been established, Rose (2001) asserted there are four basic steps to traditional content analysis: 1) establishing the corpus and its rules for inclusion/exclusion, 2) the development of coding categories, 3) the coding of all data in the corpus, and 4) the analysis of the results. The researcher searches across the corpus, looking to infer nonmanifest characteristics and construct patterns common across the data corpus (Krippendorff, 2004).

Although this description seems rather linear in fashion, the actual development and employment of coding categories is actually a recursive and iterative process requiring revisions and further analysis to be successful.

For this paper presentation, two literacy researchers will present an adaptation of traditional qualitative content analysis that works in an iterative fashion to construct categories and themes across a significant body of multimodal phenomenon. Focusing on a selected corpus of metafictive, or postmodern picturebooks, the presenters will describe and share their analytic techniques that blends grounded theories and other interpretive analytical procedures with content analysis using a study on metafictive children’s picturebooks as an example.

Multimodal content analysis, as described in this presentation, requires researchers to draw upon specific analytical frameworks and an analysis of the data itself to construct a series of templates that help researchers systematically observe and analyze multimodal phenomenon at the site of the phenomenon itself. The process of coding, constructing categories, and inferring nonmanifest themes requires researchers to revisit, discuss, rethink, and analyze a selected data corpus over and over before arriving at any trustworthy conclusions or results.

Interpersonal and interactive meanings in TED speech videos

Shao Shan, lecturer

College of Foreign Languages, Nanjing University of Aeronautics and Astronautics
[e-mail]

At present, most researchers focus on the study of multimodal text, while they give less attention to the study of dynamic multimodal discourse. TED speech video is a typical dynamic multimodal discourse with great communicative meaning. The purpose of this study is to explore how the interpersonal and interactive meanings are constructed and what the relationship is between different modalities in TED speech videos. Based on the interpersonal function of systemic functional grammar and the interactive function of visual grammar, this study adopts both quantitative and qualitative methods to analyze the research data. Through investigating the relations between the different modalities in TED speeches, it can be found that the auditory modality formed by language is the main modality. Visual modality plays a supporting role in TED speeches. The relations between visual modality and auditory can be divided into three kinds: reinforcement, extension and interpretation. Visual modality and auditory modality work together to construct meanings in TED speeches.

Brand Image Projection from Multimodal Metaphor and Metonymy - A Comparative Analysis of HUAWEI’s Chinese and Overseas Video Advertisements

Chen Shujun, Phd candidate

Guangdong University of Foreign Studies, China
[e-mail]

Wei Zaijiang, Phd. Doctoral Supervisor

Guangdong University of Foreign Studies, China
[e-mail].

Most of the multimodal metaphor and metonymy analyses have been concerned with their cognitive construction, while few people have carried out comparative studies from the intercultural perspective. Based on Cognitive Linguistics, this paper analyzes how multimodal metaphors and metonymies project HUAWEI’s distinct brand images within four video ads of HUAWEI mobile phones (two Chinese and two overseas versions). By taking Hofstede’s theory of cultural dimensions into consideration, the result reveals that the Chinese versions place emphasis on collectivism and morality while the overseas ones focus on individualism, innovation and achievement. HUAWEI build different brand images to meet cultural demands of local markets, which provides implications for international enterprises to employ effective speech strategies in trans-cultural communication. The combination of Cognitive Linguistics and intercultural theory makes contributions to the fusion of different perspectives in multimodal discourse analysis.

Keywords: Video ads, multimodal metaphor, multimodal metonymy, brand image projection, intercultural communication


The exhibition ‘Gestures – in the past, present, and future’

Martin Siefkes

Technical University of Chemnitz

The exhibition presents some results from the MANUACT research project. It is the first ever exhibition that traces human gestures as a means of communication, starting from their relations to traditional crafts to the future of gesture control. The exhibition presents current gesture research in its many facets and practical applications, explains the function of (both symbolic and co-speech) gestures in human communication, and outlines the rapidly growing importance of gesture control (e.g. in smart homes, interaction with self-driving cars, and human-robot interactions). It also discusses risks and fears related to these technological developments, for example of robots replacing humans at work.

The exhibition centres on the scientific perspectives of the MANUACT project, but combines them with artistic installations, and fascinating interactive exhibits (specifically developed for this exhibition by the Ars Electronica Futurelab, a worldwide leader in media art). Difficult and little-known academic research is presented so that it can be (literally) grasped. Forms of presentation have been found that appeal to both children and adults. Artistic research forms an integral part of the exhibition.
http://manuact.org/en/

http://www.gesten-im-museum.de/home/en/

https://www.tu-chemnitz.de/tu/pressestelle/aktuell/8233/en

https://www.tu-chemnitz.de/tu/pressestelle/aktuell/8464/en

https://www.smow.com/blog/2017/11/gestures-past-present-and-future-the-sachsische-industriemuseum-chemnitz/


Interim Representation as a Pedagogic Strategy in Civil Engineering Education

Zach Simpson

University of Johannesburg, South Africa
[e-mail]

Academic disciplines draw on particular forms of meaning-making; in many cases, including the science and engineering disciplines, these forms of meaning-making are highly abstract. One of the challenges facing university students who aspire to join such disciplines is mastery of these abstract forms of meaning-making. Often, the ‘semiotic leap’ which students must take in order to access these abstractions is vast and acts as a hindrance to student success.

It is thus necessary for educators to seek out and deploy strategies for facilitating students’ access to these abstract forms of disciplinary meaning-making. One such strategy is the use of interim representations. Interim representations rely on the semiotic notions of transduction (Kress, 2000) and re-semiotization (Iedema, 2003). They draw on less abstract forms of meaning-making in order to lessen the semiotic leap expected of students, thus acting as a semiotic bridge that facilitates students’ access to the more abstract representational mode.

This paper discusses three examples of the pedagogic use of interim representations in a tertiary diploma in civil engineering offered by a university in Johannesburg, South Africa. In the first example, an educator uses tabulation as a bridge to accessing complex mathematical expressions. In a second example, another educator uses pictorial drawings to help students access a tabulated representation. In the final example, it is demonstrated that textbooks, too, make use of interim representations in order to facilitate their student-readers’ access to highly abstract forms of meaning-making.

The significance of these interim representations is then discussed. This significance lies in their use as a pedagogical strategy aimed at enhancing student access to educational practices within higher education, in that they “materialize, preserve, and facilitate the intermediate steps in knowledge production” (Juhl and Lindegaard, 2013: 25).

References

Iedema, R. 2003. “Multimodality, Resemiotization: Extending the Analysis of Discourse as a Multi-Semiotic Practice”. Visual Communication, 2 (1), pp. 29 – 57.

Juhl, J. and Lindegaard, H. 2013. “Representations and Visual synthesis in Engineering Design”. Journal of Engineering Education, 102 (1), pp. 20 – 50.

Kress, G. 2000. “Design and Transformation: New Theories of Meaning”. In: Cope, B. and Kalantzis, M. (eds.) Multiliteracies: Literacy Learning and the Design of Social Futures. London: Routledge. (pp. 153 – 161).

Fake news and memes

Christopher A. Smith, PhD student

Applied Linguistics and Discourse Studies, School of Linguistics and Language Studies Carleton University, 1125 Colonel By Drive, 246 Paterson Bldg., Ottawa, ON, Canada, K1S 5B6
[e-mail]

While the use of ‘Fake News’ may crisply delineate political factions, appearance of the expression inside internet memes (IM) may deliver a more widespread and effective assault on agencies reared on public trust. Hence, ‘Fake News’ may be an iconoclastic emergence of weaponized propaganda delivered, in part, by IM’s to mass audiences. IM’s are multimodal artifacts featuring ideological singularities designed for ‘flash’ consumption, often composed by numerous voices echoing popular, online culture (Milner, 2013). However, some IM’s may be ‘weaponized’ by targeting specific recipients or agencies to be deliberately and continuously associated with a particular ideology for the purpose of damaging that target’s public credibility. This study proposes that IM’s when used for attacking agencies of public trust constitute a form of ‘weaponized’ propaganda or what I call weaponized iconoclastic multimodal propaganda (WIMP) discourse. Additionally, this study attempts to define ‘Fake News’ internet memes (FNIM) as model examples of WIMP discourse by asking the following questions: 1) What power relations and ideologies do FNIM’s harbor? 2) How might the manifest ideologies and power relations in FNIM’s qualify as WIMP discourse? 3) What specific agencies have been targeted by FNIM’s and by whom? A corpus linguistics assisted critical multimodal analysis of a pool of FNIM’s drawn from three social media platforms revealed targeted agencies and from what political canons they likely emerged. Findings indicate that FNIM’s were specifically directed, implying that other forms of WIMP discourse could be used as polestars for influencing online trajectories of public discourse.

Keywords: Fake News, critical multimodal analysis, internet memes, propaganda
Multimodality and Persuasion: A Mixed Methods Study of How Students Interpret Multiple Modes in a Digital Video

Blaine E. Smith, Ph.D.

University of Arizona
[e-mail]

Carita Kiili, Ph.D.

University of Oslo
[e-mail]

Miika Marttunen, Ph.D.

University of Jyväskylä
[e-mail]

Although a majority of research on youth’s multimodal literacies focuses on how they orchestrate modes to achieve a various goals and designs (e.g., Gilje, 2010; Hull, Stornaiuolo, & Sahni, 2010; Smith, Kiili, & Kauppinen, 2016), recent attention has also been devoted to understanding how youth interpret multimodal texts (e.g., Rowsell & Burke, 2009). This research is particularly important for exploring the ways youth analyze the avalanche of multimodal persuasive messages they encounter in their daily lives.

The proposed presentation builds upon this research by examining how upper secondary school students (ages 16 to 20) identified and interpreted the multimodal means used in a persuasive video. Students (n = 404) analyzed a YouTube video where two comedians demonstrated the importance of vaccinating children. Students were asked to identify the main message of the video and to indicate the three most important communicative modes for meaning making from five options (sounds; facial expressions and gestures; movements; texts; and visuals). Next, they were asked to describe how these three modes were used in the video to communicate its main message. In order to compare students’ answers, those (n=217) who selected the modes regarded as the most important for persuasion (e.g., sounds, visuals, and movements) were selected for further analysis. Through mixed methods, students’ multimodal analyses were approached from three angles: 1) richness of description of multimodal means, 2) inclusion of interpretive or evaluative elements, and 3) interplay between the modes. Examples of students’ multimodal analyses will be presented illustrating patterns and how they varied in their richness and depth. Considering that a majority of research from a social semiotics (Kress, 2010; van Leeuwen, 2005) perspective is qualitative, this presentation contributes by expanding analysis to mixed methods and a large sample of participants. Furthermore, this presentation will discuss empirical, theoretical, and methodological implications.

References

Gilje, Ø. (2010). Multimodal redesign in filmmaking practices: An inquiry of young filmmakers’ deployment of semiotic tools in their filmmaking practice. Written Communication, 27(4), 494-522.

Hull, G. A., Stornaiuolo, A., & Sahni, U. (2010). Cultural citizenship and cosmopolitan practice: Global youth communicate online. English Education, 42(4), 331-367.

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. New York: Routledge

Rowsell, J., & Burke, A. (2009). Reading by design: Two case studies of digital Reading Practices. Journal of Adolescent & Adult Literacy, 53(2), 106-118.

Smith, B. E., Kiili, C., & Kauppinen, M. (2016). Transmediating argumentation: Students composing across written essays and digital videos in higher education. Computers & Education, 102, 138-151.

van Leeuwen, T. (2005). Introducing social semiotics. Psychology Press.

Multimodal Interplay of Tropes in Tourism Advertising

Vlado Sušac

University of Zadar, Croatia
[e-mail]

Ivana Lozo

[e-mail]
University of Zadar, Croatia

This paper is a continuation of previous research based on the semiotic analysis of slogans in tourism advertising (Sušac, 2013), where dominant metaphorical concepts in the destination branding were isolated on a global scale. Here, it represents a shift from the previous monomodal to a multimodal approach to the analysis of the selected corpus, with a particular focus on the interplay of verbal and pictorial metaphors and metonymies, from contextual to hybrid ones. In other words, the objective is to examine to what extent there is a compliance of the already analysed verbal signifiers and the pertaining concepts with pictorial resources, which also include a variety of typographic solutions. Two types of relationship are identified, the functional one, directly connected to the verbal part of a mutimodal message and the expressive one, which features a much looser connection between them. The largest number of the analysed advertisements make part of the promotional activities of Croatian Tourist Board in recent years. In addition, particular emphasis will also be placed on the paradigmatic analysis of marked and unmarked signs that support or break the dominant values of the social environment in which particular promotional texts are created.

Keywords: Metaphor, metonymy, typography, slogans, tourism advertising, markedness

References

Forceville, C. (2016). Pictorial and Multimodal metaphor. In: NinaMaria Klug and Hartmut Stöckl, eds, Handbuch Sprache im multimodalen Kontext]. Linguistic Knowledge series. Berlin: Mouton de Gruyter, (241-260)

Kress, G., & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London, UK: Routledge Falmer

Machin, D. (2007). Introduction to multimodal analysis. London: Hodder Arnold.

Nørgaard, N. (2009). The Semiotics of Typography in Literary Texts. Orbis Literarum. 64(2) 141–160.

Serafini, F. et al. (2012). Typography as Semiotic Resource. Journal of Visual Literacy. 31 (2) 22-38.

Sušac, V. (2013). Conceptual Mappings in Tourism Advertising - a Semiotic Approach. Acta turistica 25,(2); 147-164

van Leeuwen, T. (2011). The language of color: An introduction. London: Routledge.

Making meaning, making signs: Semiosis through the lens of deaf-hearing interaction

Professor Ruth Swanwick

School of Education, University of Leeds, UK
[e-mail]

Dr Elisabetta Adami

School of Languages, Cultures and Societies, University of Leeds, UK
[e-mail]

This paper investigates ways in which deaf and hearing interactants communicate and understand each other when diverse cultural backgrounds, different experiences of sign and spoken language, and of being deaf or hearing are in play. We analyse ways in which multimodal resources are used in a situated way, in contexts where there are sensory and communication ‘asymmetries’ in terms of language access, experience and skills and the use of hearing technologies.

Using a social semiotic approach to multimodality, analysis of video-recorded data of interactions among deaf children and hearing adults identifies how actors use semiotic resources and communicative strategies to co-construct situated understanding beyond cultural and linguistic barriers, and to fulfil their communicative needs.

The analysis of interaction in a context where participants have different visual/gestural and auditory/oral experience of representation of meaning enables to identify (more immediately than in spoken interactions) communicative strategies that interactants use to co-create signs by drawing on a range of multimodal resources; these co-created signs then become part of the interactants’ shared repertoire for meaning-making.

We discuss ways in which these findings deepen understandings of (1) deaf/hearing communicative practices and (2) principles of semiosis enabling human communication in general. We outline the practical implications for education in particular and for facilitating communication practices in shared deaf/hearing spaces and multilingual/multicultural contexts.
Multimodality in Social Practice: Unravelling Contextual Ambiguity of Fashion in Museums

Ken Tann, Lyn Teo

University of Queensland Business School
[e-mail]

Museums have been exhibiting fashion accessories since the 1930s as a way to preserve and document cultural history. However, the recent development from ‘dress museology’ with a focus on societal representations to ‘fashion museology’ (Melchior, 2014) with a focus on contemporary designers has stirred considerable controversy about corporate sponsorship, curatorial independence, and historical accuracy (Steele, 2008). By collaborating with designers, museums seek to reach a wider audience and increase media attention, while the fashion corporations working with them seek to represent themselves in a less commercial context. The result is a ‘visitor-oriented’ and ‘sensuous experience’ (Melchior, 2011), with an ‘increased emphasis on contextualization’ (Anderson, 2000).

This paper seeks to demonstrate that the multimodal experience fashion museology presents to visitors reflects a shift in the social practice of museum exhibition, creating an ambiguity that critics are responding to. Spaces that were previously associated with educational contexts have been recontextualized as part of a branding activity. Consequently, ‘museum myths’ (Skjulstad, 2014) are appropriated by retail culture, and the narrative of fashion as told through museum exhibits has been transformed.

By comparing these narratives as multimodal discourses (Kress & van Leeuwen, 2001; Ravelli & McMurthrie, 2015), our analysis shows how dress museology and fashion museology construe very different experiences, and hence their contexts for visitors, despite being located in very similar spaces. The paper then proposes a stratified model of context (Martin, 1992; Tann, 2017) to provide a theoretical framework for understanding contextual ambiguities and their implications for discourse, as well as a way to locate multimodal integration in social practice.

References

Anderson, F. (2000) Museums as Fashion Media. In S. Bruzzi (ed.) Fashion Cultures: Theories, Explorations and Analysis. London: Routledge, pp.371–389.

Kress, G. & van Leeuwen, T. (2001) Multimodal Discourse: the Modes and Media of Contemporary Communication. London: Arnold.

Martin, J.R. (1992) English Text: System and Structure. Philadelphia & Amsterdam: John Benjamins Publishing.

Melchior, M.R. (2014) Introduction. Understanding Fashion and Dress Museology. In Fashion and Museums. Theory and Practice. London: Bloomsbury Academic, pp.1-16.

Ravelli, L.J. & McMurthrie, R.J. (2015) Multimodality in the Built Environment: Spatial Discourse Analysis. New York & London: Routledge.

Skjulstad, S. (2014) Exhibiting Fashion: Museums as Myth in Contemporary Branding and Media Culture. Paper presented at NODEM: Engaging Spaces, Design and Digital Strategies, Warszaw.

Steele, V. (2008) Museum Quality: the Rise of the Fashion Exhibition. In Fashion Theory 12(1), 7-30.

Tann, K. (2017) Context and Meaning in the Sydney Architecture of Systemic Functional Linguistics. In The Routledge Handbook of Systemic Functional Linguistics. New York: Routledge, pp.438-456.
The value of viewing modalities in a digital story résumé

Jane Tanner, Doctorate Adult Education Candidate

University of Wollongong
[e-mail]

Multimodalities within a digital story are created for the specific aim of educating the audience. The construction of a digital story has many educational benefits; but what are the benefits from viewing a digital story?

This study investigated the viewing of a digital story as a digital story résumé in a mock job interview; what happened after young adults who have a learning disability created their digital story résumé? It explores an important gap in digital storytelling; what meaning does the viewer extract from the digital story after it is viewed?

The research is presented using the theoretical framework of a new literacy and of a community of inquiry to investigate how the community, namely the viewer (employer) and the creator (applicant) in a mock interview setting, use a digital story résumé to stimulate communication between them as a community.

Using the lens of the inquiry cycle (Bruce & Bishop, 2008), a link is afforded between theory and pedagogical practice by “exploring the intersection between learners’ lived experiences and community engagement” (Bruce and Lin 2009, p. 14).

The study concludes that the digital story résumé enabled the viewers to make a comprehensive more informed employment decision about the creator.

The opportunity for prospective employers to view the modalities within the digital story résumé, using an iPad tablet sitting alongside the creators, provided them with background information (that they both heard and saw) that acted as an important stimulus for a more in-depth conversation. This purpose built experience enabled both the viewers to exercise their professional recruitment knowledge, and the creators to present themselves as viable candidates for employment. The viewers were able to gain a better sense of the character of each creator as they gathered insight into their aptitude and preferences towards particular employment positions.

This format afforded a much quicker relationship structure to be built between the interview participants than in a more traditional interview format where the participants typically have a short time to build a relationship and communicate with each other.

References

Bruce, B. and Bishop, A. (2008) New literacies and community inquiry, in Handbook of Research on New Literacies, Taylor and Frances, New York, pp. 699-742.

Bruce, B. and Lin, C.C. (2009) Voice of youth: podcasting as a means for inquiry-based community engagement, E-Learning, vol. 6, no. 2, pp. 230-241.


Teaching vocabulary to EFL learners: a multimodal corpus approach

Qu Tao
Sichuan University, Chengdu, China

[e-mail]

This research aims at exploring the possibility and effectiveness of using a multimodal corpus approach to teach vocabulary to EFL learners. It is a case study carried out in a top Chinese university, with 32 freshmen from an English listening and speaking course. The researcher provided students with a small film corpus to learn vocabulary. The way of teaching is based on the social semiotic framework of multimodal communication, which means that students are taught to pay attention to the representational, interactional, and compositional meanings realized by various semiotic resources, including language, image, gesture, facial expression, music, space and so on. It is found that the multimodal corpus approach is much more popular with the students than the traditional linguistic approach because it creates a variety of contexts and impresses students with vivid meaning-making practices. The effectiveness of this approach depends on systemic design of the multimodal corpus, focused teaching of the targeted vocabulary, and persistent input from students. Based on this research, the presentation will consist of three parts: first, a brief introduction to the theoretical framework; second, illustration of the research data and methodology; and the last part is about the research findings and implications.

References

Ackerley, K., & Coccetta, F. (2007). Enriching language learning through a multimedia corpus. ReCALL, 19(3), 351-370.

Kress, G. Multimodality: a social semiotic approach to contemporary communication. London ; New York: Routledge, 2010.

Van Leeuwen, T.(2005), T. Introducing social semiotics. London: Routledge 2005.

Aesthetical Taste Discourses in Web Design

Lisbeth Thorlacius, Associate Professor, Ph.D.

Roskilde University, Denmark
[e-mail]

The main aim of this article is to conceptualize four main taste discourses, which flourish in contemporary web design, taking a point of departure in the visual semiotic resources that form the groundwork of aesthetical and stylistic expressions. The sociologist Pierre Bourdieu defined in 1979 in the book La Distinction. Critique sociale du jugement three main taste discourses: ‘Legitimate taste’, ‘Middle-brow taste’ and ‘Popular taste’. These categories are still relevant to utilize, however in a moderated form in regard to the conceptualization of aesthetical taste discourses in contemporary web design. However, the criteria of what is considered good and bad taste within the various taste discourses have changed during the last 40 years and the borders between them have been subject to smooth transitions. In addition to the taste discourses in the sense of Bourdieu I have included one more: ‘Postmodern avant-garde taste’. In regard to this fourth taste discourse I draw on the reflections of Jean Francois Lyotard on ‘the sublime’.

Through my research I have arrived at a number of characteristic aesthetical styles within contemporary web design, which form the groundwork of exemplifying these four predominant taste discourses. This research is a further development of the styles, which were conceptualized by design researcher Ida Engholm’s Ph.D. dissertation of the history of Design from 2003 and Curt Cloninger’s Fresh Styles for Web Designers from 2002. The new styles that I consider predominant in the 2010’s consist among others of three mainstream styles: ‘Digital modernistic style with trash or retro elements’, ‘Grid style’ and ‘Cell phone style’, which all can be considered middle taste in the sense of Bourdieu. Among the styles that deviate stylistically from mainstream design I discuss ‘Modernistic Swiss style’ and ‘Branding style’, which are conceptualized in regard to the legitimate taste. Moreover, I introduce a newer sub cultural style: ‘Web Brutalism’ in regard to both popular taste and post modernistic avant-garde taste.

References

Arcement, Katherine (2016). ”The hottest trend in Web design is intentionally ugly, difficult sites.” <https://www.washingtonpost.com/news/the-intersect/wp/2016/05/09/the-hottest-trend-in-web-design-is- intentionally-ugly-unusable-sites/?utm_term=.f0875105a15f > (Viewed 15.10. 2017).

Bashooka.com: <http://bashooka.com/inspiration/20-beautiful-examples-of-brutalist-web-design/> (Viewed 20.10.2017).
Bourdieu, Pierre (1979). La Distinction. Critique sociale du jugement. Paris: Minuit.
Bourdieu, Pierre (2010). Distinction. A social Critique of the Judgement of Taste. Translated by Richard Nice. London and New York : Routledge. [First translation in English 1984].
Cloninger, Curt (2002). Fresh Styles for Web Designers - eye candy from the underground. Indianapolis, Ind.: New Riders.
Engholm, Ida (2003). WWW’s designhistorie – website udviklingen i et genre- og stilteoretisk perspektiv. (IT University of Copenhagen dissertation series). Copenhagen: IT-University. [Ph.D.-dissertation].
Ganci, Aaron og Ribeiro, Bruno (2017). ”On Web Brutalism and Contemporary Web Design.” Dialectic., 1,1, s. 91-110. <https://quod.lib.umich.edu/d/dialectic/14932326.0001.107?view=text;rgn=main> (Viewed 22.10.2017).
Kress, Gunther and Theo van Leeuwen (1996). Reading Images: The Grammar of Visual Design. London / New York: Routledge. [2. edition. 2006. 3rd edition, march 2018].
Lyotard, Jean Francois (1994). Lessons on the Analytic of the Sublime. (Kant’s ‘Critique of Judgment’, sections 23-29). Translated by Elizabeth Rottenberg. Standford, CA: Stanford University Press, (Meridian). [Originally published in French in 1991].
Serafini, Frank; Clausen, Jennifer and Fulton, Mary Lou (2012). ”Typography as Semiotic Resource.” Journal of Visual Literacy. Volume 31 (2)., s. 1-16.