Crossmodal correspondence between tonal hierarchy and visual brightness: Associating syntactic structure and perceptual dimensions across modalities

Neta B. Maimon, Dominique Lamy, Zohar Eitan

Research output: Contribution to journalArticlepeer-review

Abstract

Crossmodal correspondences (CMC) systematically associate perceptual dimensions in different sensory modalities (e.g., auditory pitch and visual brightness), and affect perception, cognition, and action. While previous work typically investigated associations between basic perceptual dimensions, here we present a new type of CMC, involving a high-level, quasi-syntactic schema: music tonality. Tonality governs most Western music and regulates stability and tension in melodic and harmonic progressions. Musicians have long associated tonal stability with non-auditory domains, yet such correspondences have hardly been investigated empirically. Here, we investigated CMC between tonal stability and visual brightness, in musicians and in non-musicians, using explicit and implicit measures. On the explicit test, participants heard a tonality-establishing context followed by a probe tone, and matched each probe to one of several circles, varying in brightness. On the implicit test, we applied the Implicit Association Test to auditory (tonally stable or unstable sequences) and visual (bright or dark circles) stimuli. The findings indicate that tonal stability is associated with visual brightness both explicitly and implicitly. They further suggest that this correspondence depends only partially on conceptual musical knowledge, as it also operates through fast, unintentional, and arguably automatic processes in musicians and non-musicians alike. By showing that abstract musical structure can establish concrete connotations to a non-auditory perceptual domain, our results open a hitherto unexplored avenue for research, associating syntactical structure with connotative meaning.

Original languageEnglish
Pages (from-to)805-836
Number of pages32
JournalMultisensory research
Volume33
Issue number8
DOIs
StatePublished - 2020

Keywords

  • Crossmodal correspondences
  • Implicit Association Test
  • Musical syntax
  • Tonality
  • Visual brightness

All Science Journal Classification (ASJC) codes

  • Experimental and Cognitive Psychology
  • Ophthalmology
  • Sensory Systems
  • Computer Vision and Pattern Recognition
  • Cognitive Neuroscience

Fingerprint

Dive into the research topics of 'Crossmodal correspondence between tonal hierarchy and visual brightness: Associating syntactic structure and perceptual dimensions across modalities'. Together they form a unique fingerprint.

Cite this