Rutgers University professor and social psychologist Lee Jussim recently posted a link on Twitter to a study that found “neither an overall effect of stereotype threat on math performance, nor any moderated stereotype effects”:

He did so in response to a Harvard University graduate student expressing surprise that there are people who think “stereotype threat” doesn’t exist:

Dr. Robin DiAngelo would also be quite surprised to hear such doubts. In her book, What Does It Mean to Be White? Developing White Racial Illiteracy, DiAngelo defines “internalized oppression” as “[t]he result of socialization in which members of a minoritized group are conditioned into their roles,” which “causes them to see themselves as naturally inferior to the relationally dominant group and less deserving of the resources of society.”

She cites the research of Claude Steele on stereotype threat. “Claude Steele’s work in stereotype threat,” DiAngelo writes, “demonstrates internalized oppression in action.” The idea is “that a person’s social identity as defined by group membership—age, gender, religion, and race—has significance in situations in which attention is drawn to that identity.” If this identity has negative stereotypes attached to it, then the person with that identity “will tend to underperform” because anxiety about that stereotype causes the person to “confirm the stereotype.” DiAngelo cites “a study in which black students were asked to identify their race before taking a standardized test.” In this study, the black students “consistently scored lower than black students who were not asked to identify their race before the test.” The scores “did not correspond with their previously identified abilities.”

“These results,” she writes, “have been consistent in many other studies Steele and his colleagues have conducted.” She then translates stereotype threat into social justice terms by calling it “internalized oppression”, a state “in which a person believes the negative information about their group and acts on it, fulfilling society’s expectations.” We are asked to “consider the beliefs many teachers hold about children of color and the role these beliefs play in children’s school performances.” The “[c]onstant focus on what is termed the achievement gap and pressure to perform on high-stakes standardized tests—or else!—surround these students and create a vicious circle.”

As is typical of her work, however, DiAngelo only considers the research that supports her theory. University of Chicago economist John List, on the other hand, found the theory eminently plausible but decided to test it anyway. In an interview with the Federal Reserve Bank of Richmond, he elaborates:

I believe in priming. Psychologists have shown us the power of priming, and stereotype threat is an interesting type of priming. Claude Steele, a psychologist at Stanford, popularized the term stereotype threat. He had people taking a math exam, for example, jot down whether they were male or female on top of their exams, and he found that when you wrote down that you were female, you performed less well than if you did not write down that you were female. They call this the stereotype threat. My first instinct was that effect probably does happen, but you could use incentives to make it go away. And what I mean by that is, if the test is important enough or if you overlaid monetary incentives on that test, then the stereotype threat would largely disappear, or become economically irrelevant.

List says that he and his team designed an experiment to test this theory, and found that they could not induce stereotype threat, no matter how hard they tried. His team would tell the test takers, “Women do not perform as well as men on this test and we want you now to put your gender on the top of the test.” List notes that “other social scientists would say, that’s crazy—if you do that, you will get stereotype threat every time.” But, contrary to expectations, it did not materialize. List maintains that priming works, but has concluded that “stereotype threat has a lot of important boundaries that severely limit its generalizability.”

Stereotypes Are Often Harmful, and Accurate

Stereotyping begets many social problems, but you seldom solve a problem by mischaracterizing its nature.

QuilletteNoam Shpancer

So what explains the popularization of a theory this resistant to replication? “I think,” List says, “what has happened is, a few people found this result early on and now there’s publication bias.” He continues:

…when you talk behind the scenes to people in the profession, they have a hard time finding it. So what do they do in that case? A lot of people just shelve that experiment; they say it must be wrong because there are 10 papers in the literature that find it. Well, if there have been 200 studies that try to find it, 10 should find it, right? This is a Type II error but people still believe in the theory of stereotype threat. I think that there are a lot of reasons why it does not occur. So while I believe in priming, I am not convinced that stereotype threat is important.

Thoughtful consideration of type I and type II errors (also known as false positives and false negatives, respectively) is not something you are going to find much of in radical contemporary theorizing about race. As I have written here and here, this kind of thinking falls prey to the pitfalls of any hypothesis that disdains statistical analysis. “[M]any critical race scholars,” the authors of one paper note, “are fundamentally skeptical of (if not simply opposed to) quantitative data and techniques to begin with.” Of course, this doesn’t mean that statistics provide a highway to truth. But statistics do offer robust tools with which we can test whether or not claims hold up against evidence. As I have stated before, “one of the strengths of the scientific method is that its emphasis on methodological rigor is a robust defense against any presumption of infallibility.”

None of this is to say whether one side or the other is definitively right, of course. But Professor Jussim is surely right to be skeptical about stereotype threat. As a review of the literature notes, “Two decades of research have demonstrated the harmful effects that stereotype threat can exert on a wide range of populations in a broad array of performance domains. However, findings with regards to the mediators that underpin these effects are equivocal.” In other words, even if there is correlation between stereotype threat and measures of performance, the causal relationship is far from being well understood, and may simply not be there.

Di Angelo’s writing about stereotype threat, white fragility, structural oppression and so on, on the other hand, is more like dogma than scientific theory. So it is not surprising that she considers only the evidence that confirms her a priori view of how racism works, rather than conflicting evidence that might give her pause for further reflection. If the popularity of the concept of “stereotype threat” is simply a product of publication bias, it may not be clear evidence—or indeed any evidence at all—of “internalized oppression.”