Category: Announcement

Associations between visual features and concepts are at the core of visual reasoning. Evidence suggests that color-concept associations are the basis on which people (1) evaluate preferences for colors, (2) evaluate preferences for entities, and (3) interpret meanings of colors in information visualizations. The link between color-concept associations and these three seemingly different types of judgements can be understood within the Color Inference Framework (Schloss, 2018). The framework posits that people continually form and update their associations between colors and concepts through color-related experiences in the world. These associations can be represented in a network that stores associations between all possible colors and concepts. Different kinds of inference operations are computed on the color-concept association network to produce different kinds of judgments: pooling produces preferences for colors, transmitting influences preferences for entities, and assigning determines interpretations of the meanings of colors in visual encoding systems. 

In this line of research we have two key objectives. First, we aim to understand how people form color-concept associations through their experiences in the world. Second, we aim to develop efficient ways of quantifying color-concept associations by leveraging image databases and computational modeling. With good estimates of color-concept associations, we will be able to produce more comprehensive predictions about how these associations contribute to visual reasoning.

Automatically estimating color-concept associations

Quantifying color-concept associations is a central part of our research, but obtaining those judgments from human participants is costly in time and effort. Building on prior work using large-scale databases, we are working on new ways to automatically estimate color-concept associations. So far, we have developed a hybrid approach using image statistics and human judgments. We trained and tested models using human ratings on a specific set of colors, and once the models were trained, they could be used to estimate color-concept associations for new concepts and new colors without humans in the loop. The most effective model used features that were relevant to human perception and cognition, aligning with perceptual dimensions of color space and extrapolating within color categories (Rathore, Leggon, Lessard, & Schloss, 2020). 

 

These methods will advance the field’s understanding of visual reasoning for visual communication in a few ways. First, they will help make visual reasoning research more efficient by circumventing the need to collect human ratings each time we to quantify color-concept associations to use for experiment designs on interpreting information visualizations. Second, these models provide insights into how humans form color-concept associations. Third, they will help achieve a long-term goal of automatically producing optimal color palettes for semantically interpretable visualizations.

Understanding how color-concept associations map onto dimensions in color space

Both in the scientific literature and in popular culture, it is common to find claims like color x means y (e.g., red means anger). However, color-concept associations are not so discreet or unitary. Color-concept associations are graded and continuous across color space, not all-or-none. Researchers can quantify the associations between a given concept and all possible colors, and represent them in what we call a “color-concept association space”. We contend that every concept has a color-concept association space, which means every color is associated with every concept to some degree, even if that degree is near zero. 

From this perspective, we can approach characterizing color-concept associations by quantifying how they map onto dimensions within color space (e.g., lightness, chroma, redness vs. greenness, and yellowness vs. blueness). This approach has been shown to dispel commonly held notions about color-concept associations. In particular, it is commonly held that yellow hues are associated with happiness whereas blues hues are associated with sadness. We found that happiness/sadness of colors was dominated by lightness and chroma. When lightness and chroma were controlled statistically or colorimetrically, yellow hues were no happier than blue hues and in some cases blue hues were happier (Schloss, Witzel, & Lai, 2020). Although the origin of these color-emotion associations is still unclear, having a more accurate description of the phenomena will help constrain possible accounts of where color-emotion associations come from and why they exist.

To interpret information visualizations, people must determine how perceptual features (e.g., color, shape, size, texture) map onto concepts. This process is easier when the encoded mapping between perceptual features and concepts in visualizations matches people’s expectations. The questions is, what determines people’s expectations? Answering this question will enable the design of visualizations that are easier to interpret.

The UW Virtual Brain Project™ is making education more engaging through virtual reality (VR). The immersive learning environments spark excitement, and  improve student’s understanding of how the brain works.

To demonstrate the potential of VR-based learning, we have developed two lesson plans which can be downloaded below:  The Virtual Visual System™ and Virtual Auditory System™. The lesson plans immerse people in a model of the brain based on real brain scans, allowing them to follow the path from sensory input to cortex. Information stations along the way describe key topics at each stage of neural processing.

Our perspective on VR education is that VR is a lens, analogous to a microscope or telescope, through which students  experience content that would otherwise be difficult to see. We believe that the future of VR in the classroom is to provide enriched experiences that are integrated within the larger course structure, rather than supplant traditional education. Just as students do not spend entire classes with microscopes or telescopes attached to their face, they also need not to spend entire classes wearing VR headsets. VR acts as a springboard to facilitate class discussion and activities, rather than isolate students from each other and the instructor. Thus, the UW Virtual Brain Project™ lessons are brief (about 5 min.) and can be built into regular lessons on neural structure and function.

In the classroom. In Spring 2019, Professor Karen Schloss implemented the Virtual Visual System™ and Virtual Auditory System™ in her lecture course Psychology of Perception (Psych 406) at the University of Wisconsin-Madison. With 25 Oculus Go headsets in hand, approx. 80 students cycled through a VR experience in about 20 min. This timing enabled setting up the topic using traditional lecturing methods, completing the VR experience, and then conducting group activities to practice content learned in VR and re-engage with the class community.

In the lab. The UW Virtual Brain Project™ team is  conducting research to demonstrate the efficacy of VR-based education and  to identify the aspects of VR that are especially beneficial to learning outcomes.

UW Virtual Brain Project™ team: Karen Schloss • Bas Rokers • Chris Racey • Simon Smith • Ross Treddinick • Nathaniel Miller • Melissa Schoenlein • Chris Castro

Download the UW Virtual Brain Project™