“Vision is the primary means through which we experience the world around us,” says vision scientist Sholei Croom, a doctoral candidate in the Department of Psychological and Brain Sciences. Croom’s research in the Hopkins Perception & Mind Lab explores how we see, how we think, and how the two interact to produce sophisticated behavior, like reaching to pick up a cup or shaking a box to figure out what’s inside. We rely on a variety of visual cues to interpret what we see, but with mixed success rates.

Visual perception and action

Croom’s early research as an undergraduate at Brown explored how visual perception influences action. For example, if we see a cup on a table, we get a sense of how full the cup is, how far away it is, and how to reach out and grab it. At Hopkins, Croom built on that research to study questions of intuitive physics, the way we interpret the environment in terms of physical dynamics and properties.

Croom got the idea for an experiment involving knots while working on an embroidery project. To fix a mistake on the front side of an embroidered cloth, one must turn it over to untangle the errant stitch on the back side, explains Croom. “It made me wonder, what are the underlying processes or mechanisms that are enabling me to do this? Is it something that I’m just parsing visually… or am I also using some sort of physical reasoning?”

To find out, Croom designed an experiment where participants were shown four knots that are visually similar but vary in terms of strength. People were asked to look at the knots and choose the strongest one. They were consistently incorrect. “It suggests that there’s a gap in what is otherwise a really robust ability to understand the physics of our environment,” says Croom, whose findings were published in Open Mind.

Understanding that relationship

Now, Croom is turning to how people intuitively understand the relationship between perception and action, and how that plays out in observing other people. “Most research on social cognition focuses on how other people interpret the behavior they see in terms of thoughts, beliefs, and desires,” explains Croom. “The topic of my dissertation is how we can also interpret someone’s behavior in terms of visuomotor experiences as well.” In one experiment, participants watch a video and guess whether a person is actually performing an action or just pantomiming it.

While all these experiments may seem very different, Croom says, “It’s all part of this overarching story about the limits of what we can understand just by visual observation… perception, physical knowledge, the motor system—how does this all interact to help us understand the world around us?”