The effect of semantics on perception – investigating synaesthesia and multisensory perception

Synaesthesia is a rare neurological phenomenon by which a sensory input triggers sensation in other modalities. Despite the large variety of reported experiences (Day, 2005), most triggering inputs are acquired concepts (e.g., graphemes, weekdays, pitches), setting learning as mechanism of interest.
Multimodal learning is a complex process modulated by factors that are both idiosyncratic and also environmental (e.g., type of information, teaching method). The present project is divided into 3 different subsections that investigate those aspects.
Regarding the general influences of environment, spontaneous multimodal associations involving basic concepts such as shapes, pitches, graphemes, colours, months and space will be investigated in both Europe and Asia.
The influence of teaching methods on multimodal knowledge will be explored through the musical teaching technique of the (i.e., ABC song) in childhood. Additionally, the possibility of enriching the knowledge of letter identities at a non-linguistic level in expert adult readers using music will also be investigated. Finally, reading performances and colour naming processes will be compared in synaesthetes and non-synaesthetes.
The various subsections involve collaborations with museums in Denmark and Japan, along with data collections in Beijing. The aim of the project is to better understanding the role of senses in the learning process.