Putting the image back into the frame: Modeling the linkage between visual communication and frame-processing theory

Given the rising use of visual and multimodal information, text-oriented framing research is at risk of losing traction with current media reality. We propose applying frame processing theory as a general framework for understanding how coherent meaning is constructed from complex stimuli, regardless of their modality: Both visual and textual information processing follow a recursive sequence of (a) selective perception/structuring, (b) decoding, (c) the construction of relations, and (d) their integration into coherent meaning. The specifics of visual and textual modalities provide varying degrees of structuring and salience within a fundamentally unified information processing process. Integrating advances from framing and visual communication research, we discuss implications for the empirical analysis of multimodal news contents, and sketch an agenda for research.