Hi! This is a poster on eye-tracked evaluation of subtitles in immersive VR 360 video, with Marta Brescia-Zapata and Pilar Orero from Univsitat Autonoma of Barcelona,
Krzysztof Krejtz at SWPS University in Poland, and Chris Hughes from Salford
in the Uk and myself from Clemson in South Carolina, the U.S.A. All right.
So subtitles are important for not only multilingual translation,
but also for accessibility services.
And while there are standards that exist for 2D immersive media little has been explored in VR/AR,
in 360 video, for example, this is what our poster is about.
So we designed a very controlled experiment where we
compared head-locked and fixed subtitles. So that's position versus monochrome and color subtitles in a very controlled manner such that each viewer would only see one type of video content once.
So they either saw head locked
in color or fixed in monochrome, but from the 2 different videos, either the one on the left there, or the one on the right.
So to do this we wanted to
eye-track the experiment, to see what people are looking at inside VR 360, and to get a sense of how well did the viewers actually look at
the subtitles. So our framework for evaluating this in VR is the novelty really of our contribution where we looked at psychophysiological measures, eye movements, performance metrics, and so
reported
that kind of statistics. Our framework is based on an HTC Vive Pro Eye, where we use a data manager to display the subtitles using a particle system like implementation
from computer graphics which runs in its own thread
so that we can capture our movements at 120 Hertz.
And then we record everything, including video position, caption, position by the manager, and then later we can play back or output
the results that we had collected.
So we hypothesized the headlocked subtitles, the ones that are always in front of the viewer, would be better for content comprehension, measured by lower task, load,
and in other words, lower cognitive, demand, self-reported in this case, and also that
they'd be easier to read in some sense. So we'll look at that in a minute. A 2 by 2 mixed design, where we use the position
as a between subjects factor and the color was within subjects factor. So like, I said, viewers would only see one type of content
at a time, counter-balanced. 24 participants played a part in our study, and most of our digital savvy
the digital media savvy.
Here's the experiment. We started with a demographic questionnaire, then calibrated the eye tracker inside the HMD,
provided instructions, played the clip,
and then obtained the NASA TLX, so self-reported load questionnaire,
Video clip 2, counterbalanced, so that you know the order of presentation was always different
or balanced across subjects, or participants.
And then a debrief session to explain what was going on. So results show that head-locked self-reported task load was indeed lower than the fixed subtitles. So fixed subtitles are fixed in space
within the video, and so the user has to rotate their head and essentially find the subtitles that are positioned next to the speaker.
Ambient-focal attention, our K coefficient, which is a the metric that uses both the fixation duration minus the subsequent fixation saccade,
you can see the poster for further details, showed that for headlocked subtitles
viewers were mainly focal, so they could read the subtitles, and then they were
ambient, so they can move their eyes around afterwards. The observation was somewhat reversed in the fixed subtitles. They were ambient first,
so they were most likely looking for the subtitles first, and then
focal in time to be able to read the subtitles. Total fixation time
on the headlock subtitles
between scene and subtitle was pretty much the same. So, in other words, it allowed participants to see this as much of the scene in subtitles in a balanced kind of way; whereas the fixed subtitles
allowed more viewing of the scene and fewer time on the subtitles. Now that, on the one hand, that is good because you get to see more of the scene as intended, and not spending most of your time on the subtitles. On the other hand, it could also mean that you're basically having a hard time finding
the subtitles that are positioned somewhere in the scene.
So head-locked are better which was expected, and it allowed better content comprehension,
as expected. We did not find much evidence in terms of impact of color in this particular instance.
But I think this is a a landmark study in the sense of
processing eye movements as the participants are viewing subtitles in VR.
Those are scarce in 2D, those kinds of studies, and even more scarce in VR. And hopefully further work on this, including more controlled experiments, will lead to better content comprehension and better use of subtitles
in the video, whether it's for accessibility purposes, or translation.
That's about it. Thank you very much. As I said, this is a a large
cooperation, and is also funded by the LEAD-ME cost European cooperation, science, technology funded by the horizon 2020 program,
and involves several groups of people from Poland, the UK, Spain, and the U.S.A. Thank you so much.