Decoding and Reconstructing Orthographic Information from Visual Perception and Mental Imagery Using EEG and fMRI
Year: 2023
Type: dissertation
Abstract: Vision plays a crucial role in reading by supporting the processing of visual-orthographic representations, which are essential for social interactions and daily activities. In recent years, there has been a lot of interest in investigating such processes across multiple methodological domains, including electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). For instance, studies have shown the importance of the visual word form area (vWFA) in reading, but its exact role remains unclear. On the other hand, mental imagery is a unique form of visual experience where, in contrast to perception, no external stimuli are present. The current thesis aims to investigate the neural representational structure of visual-orthographic information in both visual perception and mental imagery by considering both EEG and fMRI signals during word recognition and by applying state-of-the-art pattern analysis techniques to such signals. Specifically, Chapter 2 established the feasibility of decoding and reconstructing whole-word images from EEG data. The examination of the time course of decoding provided insight into the temporal dynamics of early word processing, identifying critical time points for word recognition. Chapter 3 examined the relative contribution of different time points from different EEG channels, revealing time and frequency features that underlie decoding for words and faces. Chapter 4 investigated the neural representational structure of words in mental imagery using fMRI and successfully decoded and reconstructed whole words from activity patterns in both vWFA and early visual cortex. These findings provide a deeper understanding of the neural mechanisms underlying visual perception and mental imagery of orthographic information. They demonstrate the feasibility of retrieving fine-grained visual features from neural signals that support the distinction of within-category exemplars, which offers considerable potential benefits for the development of new communication methods for individuals with communication impairments.
Source: TSpace (University of Toronto)
Author: Ling, Shouyu
Institution: University of Toronto
Language: English
Subfield: Developmental and Educational Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7132889655
