Schloss Visual Reasoning Lab

Associations between visual features and concepts are at the core of visual reasoning. Evidence suggests that color-concept associations are the basis on which people (1) evaluate preferences for colors, (2) evaluate preferences for entities, and (3) interpret meanings of colors in information visualizations. The link between color-concept associations and these three seemingly different types of judgements can be understood within the Color Inference Framework (Schloss, 2018). The framework posits that people continually form and update their associations between colors and concepts through color-related experiences in the world. These associations can be represented in a network that stores associations between all possible colors and concepts. Different kinds of inference operations are computed on the color-concept association network to produce different kinds of judgments: pooling produces preferences for colors, transmitting influences preferences for entities, and assigning determines interpretations of the meanings of colors in visual encoding systems. 

In this line of research we have two key objectives. First, we aim to understand how people form color-concept associations through their experiences in the world. Second, we aim to develop efficient ways of quantifying color-concept associations by leveraging image databases and computational modeling. With good estimates of color-concept associations, we will be able to produce more comprehensive predictions about how these associations contribute to visual reasoning.

Automatically estimating color-concept associations

Quantifying color-concept associations is a central part of our research, but obtaining those judgments from human participants is costly in time and effort. Building on prior work using large-scale databases, we are working on new ways to automatically estimate color-concept associations. So far, we have developed a hybrid approach using image statistics and human judgments. We trained and tested models using human ratings on a specific set of colors, and once the models were trained, they could be used to estimate color-concept associations for new concepts and new colors without humans in the loop. The most effective model used features that were relevant to human perception and cognition, aligning with perceptual dimensions of color space and extrapolating within color categories (Rathore, Leggon, Lessard, & Schloss, 2020). 

These methods will advance the field’s understanding of visual reasoning for visual communication in a few ways. First, they will help make visual reasoning research more efficient by circumventing the need to collect human ratings each time we to quantify color-concept associations to use for experiment designs on interpreting information visualizations. Second, these models provide insights into how humans form color-concept associations. Third, they will help achieve a long-term goal of automatically producing optimal color palettes for semantically interpretable visualizations.

Understanding how color-concept associations map onto dimensions in color space

Both in the scientific literature and in popular culture, it is common to find claims like color x means y (e.g., red means anger). However, color-concept associations are not so discreet or unitary. Color-concept associations are graded and continuous across color space, not all-or-none. Researchers can quantify the associations between a given concept and all possible colors, and represent them in what we call a “color-concept association space”. We contend that every concept has a color-concept association space, which means every color is associated with every concept to some degree, even if that degree is near zero. 

From this perspective, we can approach characterizing color-concept associations by quantifying how they map onto dimensions within color space (e.g., lightness, chroma, redness vs. greenness, and yellowness vs. blueness). This approach has been shown to dispel commonly held notions about color-concept associations. In particular, it is commonly held that yellow hues are associated with happiness whereas blues hues are associated with sadness. We found that happiness/sadness of colors was dominated by lightness and chroma. When lightness and chroma were controlled statistically or colorimetrically, yellow hues were no happier than blue hues and in some cases blue hues were happier (Schloss, Witzel, & Lai, 2020). Although the origin of these color-emotion associations is still unclear, having a more accurate description of the phenomena will help constrain possible accounts of where color-emotion associations come from and why they exist.