This is a striking development:
Researchers from the University of Wisconsin at Madison, Harvard, and the University of Virginia examined 499 studies over 20 years involving 80,859 participants that used the IAT and other, similar measures. They discovered two things: One is that the correlation between implicit bias and discriminatory behavior appears weaker than previously thought. They also conclude that there is very little evidence that changes in implicit bias have anything to do with changes in a person’s behavior. These findings, they write, "produce a challenge for this area of research."
That’s putting it mildly. "When you actually look at the evidence we collected, there’s not necessarily strong evidence for the conclusions people have drawn," says Patrick Forscher, a co-author of the paper, which is currently under review at Psychological Bulletin. The finding that changes in implicit bias don’t lead to changes in behavior, Forscher says, "should be stunning"....
Everyone agrees that the statistical effect linking bias to behavior is slight. They only disagree about how slight. Blanton’s 2013 meta-analysis found less of a link than a 2009 meta-analysis by Banaji and Greenwald. Blanton sees the correlation as so small as to be trivial. Banaji and Greenwald, in a 2015 paper, argue that "statistically small effects" can have "societally large effects."
The new analysis seems to bolster Blanton’s less-sanguine take. It found that the correlation between implicit bias and behavior was even smaller than what Blanton had reported. That came as a surprise, the researchers write.
Another surprise is that one of the co-authors of the paper is Brian Nosek, who is — along with Greenwald and Banaji — one of the three founders of the IAT. Nosek, best known these days as the director of the Center for Open Science and an advocate for better research practices, is well aware that this paper will provide aid and comfort to critics of the test he helped create....
He does defend the IAT, noting that it’s engaged millions of people in a conversation about the science of bias. He points to the test’s successes, like experiments that show how it can predict who someone would favor in a presidential election by tracking their associations. But what he calls the "very weak overall" connection between implicit bias and discriminatory behavior should, he believes, put researchers on notice. "You would think that if you change the associations, and the associations predict behavior, then the behavior would change too," Nosek says. "But the evidence is really limited on it."
Patrick Forscher, who shares the title of first author of the paper with Calvin Lai, a Harvard postdoc, thinks that there’s been pressure on researchers over the years to make the science of implicit bias sound more definitive and relevant than the evidence justifies. "A lot of people want to know, How do we tackle these disparities?" says Forscher, a postdoc at the University of Wisconsin at Madison. "It makes us feel important to say, Aha, we have these measures that can tell us what the problem is, and, not only that, we can tell them how to fix the problem."
That’s essentially Blanton’s argument as well. Public discussion about implicit bias has been based largely on the results from one particular test, and that test, in his view, has been falsely sold as solid science. "They have engaged the public in a way that has wrapped the feeling of science and weight around a lot of ‘cans’ and ‘maybes,’" Blanton says. "Most of your score on this test is noise, and what signal there is, we don’t know what it is or what it means."
I trust the philosophers who jumped on this bandwagon will now rethink their positions in light of the actual evidence. If one is looking for an explanation for the obstacles women face in philosophy, perhaps explicit bias and sexual harassment deserve the real attention.
(As a sidenote, and a reminder of the totalitarian environment in some departments, the PhD student who sent this to me asked not to be named since, "Folks in my department take this stuff as gospel, so I'd rather not be targeted as a skeptic." After this newest research, anyone who still takes it as "gospel" will have a lot in common with those who take the actual gospel literally.)