Most women have experienced the awkward boob ogle. One moment you’re having a perfectly civil conversation with a guy—a friend, a romantic interest, a colleague, a complete stranger—the next, his eyes drift downward and glaze over. In the best of circumstances, catching a guy in the act can lead to a sarcastic rebuff such as the stereotypical, “Hello, I’m up here.” But in some situations, these social faux pas may have more dire impacts on the victim. Scientific studies have shown that the so-called objectifying gaze, for some women, makes them uncomfortable with their bodies in public, keeps them from speaking up and is even linked to decreased cognitive performance.

Stopping some of those roving eyes depends on understanding when and why people choose to focus on a woman’s chest rather than her face. In an October 2013 study published in the journal Sex Roles, researchers asked 29 female and 36 male students to examine 10 photos of college-aged women. The participants did not know that the researchers had manipulated the images to accentuate or deemphasize certain body parts, such as breast size or hip-to-waist ratios. The researchers asked some of the participants to evaluate the woman pictured for her personality, and the others to judge her appearance. While the participants considered the photos, the researchers tracked their gaze using a device called the EyeLink II system.

Surprisingly, when asked to evaluate the women on their appearance, the female participants were just as guilty as the males of allowing their gaze to linger on the female figures’ chests and waists. The more hourglass the woman in the photo, the longer participants dwelled on those bodily features. “Both men and women were objectifying women—that is, they were both directing their attention to the body more and the face less when they were looking at curvaceous bodies or asked to focus on appearance,” says Sarah Gervais, a psychologist at the University of Nebraska-Lincoln and lead author of the study.

READ MORE: We See Women as Parts, Men as Whole People

The pattern changed slightly in the personality-focused group. Participants of both genders first looked at the woman’s face, although men allowed their attention to wander to other body parts more quickly than female participants. But overall, Gervais says, “We found that the [objectifying] gaze was significantly tempered when people focused on women’s personalities.”

Finally, while men and women both tended to stare, men viewed photos of women with idealized hourglass figures more positively than those showing women with average or smaller breasts and hip-to-waist ratios. This held true even when the men were supposed to be judging the woman for her personality, not her looks. Women, on the other hand, did not show significant favoritism either way.

Rather than despair that women are forever doomed to be objectified by both men and each other, the researchers are applying a more positive interpretation to their results, especially in light of the findings about personality. “The gaze is certainly not inevitable and easy interventions such as talking about women’s personalities or more humanizing qualities can easily temper these effects,” Gervais thinks.

Of course, the participants in this experiment were looking at photos of strangers. The researchers didn’t test whether the same things happen when people look at friends or acquaintances—that is, people whose personalities and human-ness are already front and center in our minds. Gervais posits that the results would bear out differently and that both men and women would be far less likely to undress, say, a co-worker, with their eyes.

But there are two exceptions to that, she warns: if the woman is dressed or acting seductively, or if you’re hoping to undress her with more than your eyes.