Research on Japanese impact captions which shake, pulse and capture your attention on TV screen
Research on Japanese impact captions which shake, pulse and capture your attention on TV screen
Combining research expertise in Asian Studies, Cognitive Pragmatics and Audiovisual Translation, Dr. Ryoko Sasamoto and Dr. Minako O’Hagan at the Centre for Translation and Textual Studies are investigating a distinct form of same-language captions widely used on TV in Asia. Originating in Japan in the 1980s, these “impact captions” typically displayed in multiple-colours occupy a large proportion of the TV screen (see the screenshot), often with further added effects such as pulsing and shaking. These open captions (ie the viewers cannot turn them off) are added during the post-production editing process by TV producers and directors who choose particular verbal and non-verbal elements of their programmes to be highlighted in order to draw the attention of the “hearing” viewer. As such, they convey information that is highly selective and are completely different in nature from closed captions (i.e. the viewer can turn them on and off) designed for the deaf and hard-of-hearing. Despite the prevalent use of these captions across different programmes and genres, including news, there are currently no official guidelines established. Their use is largely controlled by particular TV stations or even by particular TV producers and directors while the viewers are constantly exposed to them without a choice. Our interview and survey data shows that the generation of viewers in Japan who grew up with these captions finds it difficult to fully enjoy entertainment programmes without them while others who are displeased with the visible invasion by these captions regard them as a manipulative and dumbing down mechanism. Furthermore, the relative ease of adding these captions has on occasions led to glitches that allowed unchecked captions to be aired to the distress and anger as well as amusement of the viewers. Motivated by such unfolding situations and the remaining lack of academic interest in these captions, we have been conducting a series of detailed studies. We have been analysing in some depth the exact composition of these captions in selected Japanese TV programmes by applying a multimodal analysis with the help of qualitative data analysis software while carrying out empirical studies with viewers, using eyetracking. Our PhD student, Eline Sikkema, funded by this project is investigating the effect of these captions on learners of Japanese to provide empirical evidence into what has so far been only anecdotally suggested. It will form a first such study. The lack of research interest in these captions reflects the regional nature of these captions which have so far been confined largely to certain Asian countries. However, more recently, same language titles have started to emerge on TV in the rest of the world, as demonstrated by dramas such as the BBC series Sherlock; their use of the same language titles, known as integrated or authorial titles, as a diegetic element forming part of the drama’s narrative structure even came to characterise the drama.
The impact caption project has attracted a number of fundings from RIS, ISCA and the Faculty of Humanities and Social Sciences. Such support allowed us to conduct eyetracking experiments in Japan and visit Japanese TV stations to talk to TV producers and caption makers as well as a visit to the UK to meet BBC’s R&D group that have been using eyetracking to determine the optimum placing of the captions in Sherlock from the veiwers’ perspective. The impact of the research dissemination so far includes a number of invited lectures in Ireland, the UK, Italy and Japan and an invitation to participate in the international workshop “Language and Global Media” in Australia in November 2016 organised by Dr. Claire Maree through the Asia Institute at the University of Melbourne as part of her research project funded by the Australian Research Council (ARC-DP:DP150102964).