RESEARCHER SPOTLIGHT: WENYU HUANG

Although neuroscience has become an important part of understanding education and its association with brain development, few previous studies have applied neuroimaging techniques to multiliteracies research. To address this gap, my study used functional near infrared spectroscopy (fNIRS) to investigate the association of multimodality and multiliteracies learning on adult English Second Language (ESL) students’ performance on multimodal tasks and brain activity. Emotional videos were selected as stimulus because they contain multimodal information and language learning was considered highly related to emotions.

Participants’ multimodality background was collected through a technology questionnaire. Behavioural and fNIRS data were collected in two sessions of experiment (i.e., before and after multiliteracies learning) in which participants need to answer questions after viewing and listening emotional videos. The behavioural data showed that there was no significance change in behavioural responses while the model for predicting the correctness and reaction time changed after multiliteracies learning. The fNIRS data showed that multimodality and multiliteracies learning is associated with activation of the learning network in the brain including the superior temporal gyrus (STG), premotor cortex (PMC), and the intraparietal sulcus (IPS).