Work

A model of face selection in viewing video stories

Year: 2014
Type: dissertation
Abstract: When typical adults watch TV programs, they show surprisingly stereo-typed gaze behaviours, as indicated by the almost simultaneous shifts of their gazes from one face to another.However, a standard saliency model based on low-level physical features alone failed to explain such typical gaze behaviours.To find rules that explain the typical gaze behaviours, we examined temporo-spatial gaze patterns in adults while they viewed video clips with human characters that were played with or without sound, and in the forward or reverse direction.We here show the following: 1) the ''peak'' face scanpath, which followed the face that attracted the largest number of views but ignored other objects in the scene, still retained the key features of actual scanpaths, 2) gaze behaviours remained unchanged whether the sound was provided or not, 3) the gaze behaviours were sensitive to time reversal, and 4) nearly 60% of the variance of gaze behaviours was explained by the face saliency that was defined as a function of its size, novelty, head movements, and mouth movements.These results suggest that humans share a face-oriented network that integrates several visual features of multiple faces, and directs our eyes to the most salient face at each moment. When typical adults watch TV programs or movie scenes, they show surprisingly stereo-typed gaze behaviours in time and space [1][2][3] , as typically indicated by almost simultaneous shifts of their gazes from one face to another 4 .Recent studies have shown with the aid of multidimensional scaling that such typical temporo-spatial gaze behaviours were not shared by adults with autism 4 or by monkeys 3 and were quite different from those generated artificially based on low-level physical features 3 .These previous findings suggest that temporo-spatial gaze behaviours of typical adults are driven by some human specific social ''saliency'', which must be different from the standard saliency model that depends solely on the low-level physical features 5,6 .In the present study, we propose a rule that determines the dynamic choice among the faces while viewing motion pictures.First, we placed a face detection mechanism along a cascade of other factors in our model for face selection (Figure 1a) because typical adults spend most of their time viewing a face, whether they viewed motion pictures 4,7,8 or still pictures 9,10 .However, we also looked at other items in a scene; for example, text typically attracts attention 4,9,11 .Thus, in the first part of this study, we tested whether the essence of actual gaze behaviours was retained after disregarding non-face objects while they viewed short video clips that featured two or more human characters.We then examined gaze patterns when the video clips were played without sound, to evaluate if verbal semantics plays a critical role in determining our gaze behaviour.If verbal semantics in conversation is critical, our stereo-typed gaze behaviours should be significantly altered by not providing sound.We also presented the video clips in reverse, to further test whether the normal context other than the verbal semantics is critical for the generation of gaze behaviours.If the gaze behaviours are depending solely on the low-level physical features, the behaviours should be symmetric in time reversal.We then hypothesized that the saliency of each face is determined by a combination of its size, novelty, head movement, and mouth movement (Figure 1a).We chose size because a recent neuroimaging study has shown that responses in the fusiform face area were clearly modulated by the size of the face stimuli 12 .We added the novelty component because the amygdala, which is assumed to be a member of face detection circuits, is reported to respond strongly to novel faces [13][14][15] .We chose head and mouth movements because a number of face areas over the occipitotemporal cortex are responsive both to biological motion and face OPEN
Authors: 須田, 悠紀, スダ, ユウキ
Language: English
Open Access status: green
OpenAlex ID: W7011590910
Cites: 34
Cited by:
Related to: 10